Windows problems with network locations

[jlw4049]

FileBot seems to be hanging onto the directory somehow after renaming the file, even once closed when manipulating files over a network connection.

The files reside on UnRaid. They are accessed via SMB shares. I have tried this with FileBot on a Virtual machine within UnRaid and a Physical machine.



I can replicate it locally by CD into the directory with CMD.exe. Change the file name with "rename". If I keep command prompt open or stay in that directory, I cannot change it. As soon as I close command prompt/leave that directory I can do what ever I want with the file that's over the network/locally.

This is the same behavior once I rename a file with FileBot over the network. Once the the file has been renamed, I cannot move/rename the directory the file was in at all what so ever until I reboot the computer.

Code: Select all

# C:\Users\jlwServerWin\AppData\Roaming\FileBot\logs\error.log
Oct 05, 2022 8:56:33 PM net.filebot.DiskStore acquireDiskStore
WARNING: Initialize new disk cache: C:\Users\jlwServerWin\AppData\Roaming\FileBot\cache\0
Oct 05, 2022 8:57:01 PM net.filebot.UserInteraction configureLicense
SEVERE: Bad License Key: PGP SIGNED MESSAGE not found (34 chars, 1 lines)
Oct 05, 2022 8:57:01 PM net.filebot.UserInteraction configureLicense
SEVERE: Bad License Key
~~~
Year should be 1975, repack needed
~~~
Oct 05, 2022 8:57:01 PM net.filebot.UserInteraction configureLicense
WARNING: You have selected 1 line. A complete License Key has no less than 20 lines.
Oct 05, 2022 8:57:35 PM net.filebot.License lambda$verifyLicense$1
WARNING: Activate License [P41846617] on [Wed Oct 05 20:57:35 EDT 2022]
Oct 07, 2022 12:37:10 AM net.filebot.media.XattrMetaInfo setMetaInfo
WARNING: Failed to set xattr: Access Denied: F:\downloads\The Handmaid's Tale - 5x01 - Morning.mkv:net.filebot.metadata
Oct 07, 2022 12:37:10 AM net.filebot.media.XattrMetaInfo setMetaInfo
WARNING: Failed to set xattr: Access Denied: F:\downloads\The Handmaid's Tale - 5x02 - Ballet.mkv:net.filebot.metadata
Oct 07, 2022 12:37:10 AM net.filebot.media.XattrMetaInfo setMetaInfo
WARNING: Failed to set xattr: Access Denied: F:\downloads\The Handmaid's Tale - 5x03 - Border.mkv:net.filebot.metadata
Oct 07, 2022 12:37:10 AM net.filebot.media.XattrMetaInfo setMetaInfo
WARNING: Failed to set xattr: Access Denied: F:\downloads\The Handmaid's Tale - 5x04 - Dear Offred.mkv:net.filebot.metadata
Oct 07, 2022 12:37:10 AM net.filebot.media.XattrMetaInfo setMetaInfo
WARNING: Failed to set xattr: Access Denied: F:\downloads\The Handmaid's Tale - 5x05 - Fairytale.mkv:net.filebot.metadata
Oct 16, 2022 9:41:06 AM net.filebot.License lambda$verifyLicense$1
WARNING: Activate License [P41846617] on [Sun Oct 16 09:41:06 EDT 2022]
Oct 16, 2022 10:25:06 AM net.filebot.DiskStore acquireDiskStore
WARNING: Initialize new disk cache: C:\Users\jlwServerWin\AppData\Roaming\FileBot\cache\1
Oct 16, 2022 10:26:01 AM net.filebot.License lambda$verifyLicense$1
WARNING: Activate License [P41846617] on [Sun Oct 16 10:26:01 EDT 2022]
Oct 16, 2022 10:49:42 AM net.filebot.License lambda$verifyLicense$1
WARNING: Activate License [P41846617] on [Sun Oct 16 10:49:42 EDT 2022]
Oct 16, 2022 2:26:59 PM net.filebot.DiskStore acquireDiskStore
WARNING: Initialize new disk cache: C:\Users\jlwServerWin\AppData\Roaming\FileBot\cache\0
Oct 16, 2022 2:27:22 PM net.filebot.License lambda$verifyLicense$1
WARNING: Activate License [P41846617] on [Sun Oct 16 14:27:22 EDT 2022]

This does not happen with FileBot when renaming files on local drives. But I can replicate this on multiple machines with multiple shares via FileBot.

For what ever reason it's not releasing the directory. I don't see any background tasks running to kill either.

Originally I thought this was just an UnRaid problem. However, while It might be related, Bulk Rename Utility can easily rename the files and not cause this same problem.

[rednoah]

I can replicate it locally by CD into the directory with CMD.exe. Change the file name with "rename". If I keep command prompt open or stay in that directory, I cannot change it. As soon as I close command prompt/leave that directory I can do what ever I want with the file that's over the network/locally.

Are you using the CLI? If you call filebot -rename then the filebot process will do just that end then exit. If there is no running filebot process then filebot cannot possibly lock the folder. Is there a running filebot process?


If the folder is locked while the CMD process is running, and unlocked when the CMD process is killed, then the CMD process is likely locking the folder. Presumably, local file locking works differently from SMB remote shared file system locking, and so you might get different behaviours. This is likely a generic issue unrelated to FileBot specifically, and so you might find a solution via a generic Google search.


You can use Process Explorer to check which process is accessing which file or folder:
https://learn.microsoft.com/en-us/sysin ... s-explorer

[jlw4049]

Thanks for the response. You led me in the right direction regardless!

This is definitely a SMB/UnRaid issue when it comes to VM's. Thanks man!

[rednoah]

Thanks for the response. You led me in the right direction regardless!

Cheers, if you end up with some kind of solution or workaround. Please keep us posted here as well, because someone will stumble on this thread sooner or later.

[jlw4049]

This is an UnRaid edge case error.

The error was caused by two systems trying to read/index the metadata/or do something with the media file after the other system changed it. The solution was to stop the array, reboot the VM, main system, restart the array. I've not had an issue since then.

[frumble]

I don't think the above conclusion is wholly accurate, or there are multiple paths here. I am not using UnRaid but typical NAS samba configs will have similar configuration as I have.

I can reproduce this issue like this:

1. Start with samba server and a Windows client with a share mounted. Make sure "vfs objects = streams_xattr" is set (or you have streams_xattr in your vfs objects stack somewhere).
2. Create a folder in the share, "test".
3. Touch a file inside the folder, "test.mkv"
4. Open Filebot, drag test.mkv into the file pane, drag some random episode into the right pane, and rename.
5. After renaming, back out your Windows Explorer to the parent of "test", and try renaming folder "test".
6. You'll get a file locking error in Windows. If you turn on debug logging in samba (logging = 10) you'll see that can_rename ends up returning NT_STATUS_ACCESS_DENIED.
7. Now disable streams_xattr in Samba and restart it. Delete the test folder and repeat steps 2-5. You will be able to rename fine.

Renaming files by hand in the file explorer in Windows (or Mac OS) works fine for me; it's specifically something to do with the way FileBot is doing it. I don't mean I'm *blaming* FileBot, but there is some interaction here that is unusual and may need to be fixed somewhere, whether it be FB, Samba, our configurations, or even ZFS. Samba has tremendous piles of tech debt and there are a large number of potential configuration conflicts that do not log or complain, so it's entirely possible something is misconfigured for me. But it only comes up when I'm using FileBot.

Restarting samba will fix the problem until it happens again. Restarting samba every time you rename something with FileBot sucks though.

I have noticed FileBot does successfully write some xattrs (user.DosStream.net.filebot.filename and user.DosStream.net.filebot.metadata). There may be something unsafe going on internally with Samba where it does not release locks on files where xattrs were manipulated.

Additional repro data if the above isn't enough:
1. Server is Debian testing with samba 4.17.5+dfsg-2
2. Server FS is on zfs 2.1.9; dataset has xattr=sa
3. Windows machine is Windows 11 22H2
4. Filebot is 4.9.6
5. Samba looks like it negotiated SMB2.

[rednoah]

Thanks you for looking into things in details.




Do you get a different behaviour is you disable xattr or configure FileBot to use .xattr folders instead of native xattr?

Sounds like the issue is possibly related to xattr metadata in your specific case at the very least. You could try to disable xattr metadata and see what happens. FileBot will rename files and then set xattr. This is different from other rename programs which only rename files.




Do you get different behaviours when you switch between different internal move/rename implementations?

FileBot (default configuration launcher) uses IFileOperation to move/rename files, i.e. Windows Shell API.

FileBot (platform) (alternate configuration launcher) uses Files::move to move/rename files, i.e. Java API, which then internally uses some Windows API.

This may or may not make a difference though, since most of the work is presumably performed by the file system driver so higher-level application code has no idea what Windows actually ends up doing internally when the application asks the OS to "move A to B".

[frumble]

Thanks for the reply.

I did a deeper dive on ADS's (Alternate Data Streams) which are the "xattr" at work here. NTFS uses ADS's for storing named streams like what FileBot is using; usually on a Unix based host this would be done with extended attributes. The SMB protocol is using NTFS as a virtual filesystem, so Samba is emulating this behavior. If you have streams_xattr set in your vfs objects, Samba will emulate ADS support by mapping them to xattr's on the underlying filesystem. This is as opposed to a Windows system where your typical hard drive is NTFS and ADS's are stored natively. Of course, the Java library that FileBot is using has no idea about any of this and is using a high level function for editing a "user defined file attribute" (UserDefinedFileAttributeView) whose implementation is platform-and-filesystem-specific, but in this case would be using the NTFS ADS capabilities whether local or on a remote share.

So then I set out to see if I can reproduce the issue without using FileBot at all.

You can use PowerShell to set and retrieve ADS's like this:

PS Z:\>mkdir test
PS Z:\>cd test
PS Z:\test> echo 1 > test.mkv
PS Z:\test> Set-Content .\test.mkv:big -Value "chungus"
PS Z:\test> Get-Item .\test.mkv -Stream *

After running the above series of commands the parent folder (test) is now locked in the windows explorer and cannot be renamed. So yeah, very much a bug in Samba.

The net of it is, FileBot users who are renaming files on a Samba share may run into this issue if their Samba configuration has enabled streams_xattr (very useful most of the time).

The workarounds, until Samba fixes the issue, are:
1. Disable xattr writing in FileBot, which will lose historic filename information which may be useful to you; or
2. Disable streams_xattr, which may have unintended side effects depending on your usage patterns and clients, *as well as the effects in (1)*; or
3. Restart Samba every time you rename files in a folder you intend to rename afterwards.

I'll report this bug to Samba.

[rednoah]

I'll report this bug to Samba.

Cheers! Thank you for running tests and systematically narrowing down the issue, and reporting it.


As a workaround, I would recommend configuring FileBot to store metadata to .xattr folders and plain/text files to keep metadata but without using xattr / Alternate Data Streams:

Code: Select all

filebot -script fn:properties --def net.filebot.xattr.store=.xattr

https://www.filebot.net/help/xattr.html

[frumble]

Yeah, that's a good workaround.

I was just re-reading the OP one more time and noticed:

I can replicate it locally by CD into the directory with CMD.exe. Change the file name with "rename". If I keep command prompt open or stay in that directory, I cannot change it. As soon as I close command prompt/leave that directory I can do what ever I want with the file that's over the network/locally.

This is the same behavior once I rename a file with FileBot over the network. Once the the file has been renamed, I cannot move/rename the directory the file was in at all what so ever until I reboot the computer.

The first sentence is not replicating what's happening with FileBot, OP; the reason it was locked is because you had cmd.exe open inside that folder which locked it. The behavior with FileBot is almost certainly due to the Samba bug I discussed above. Both are locks, but a normal command line rename will work just fine, you just needed to exit the directory or cmd.exe before attempting to rename it.

[frumble]

The bug was introduced in Samba 4.17.0. Unraid switched to the 4.17 series in late September 2022, so OP probably ran into this error for the first time after updating Unraid.