Filebot CLI hangs for days with no output

Support for Windows users
Post Reply
randyest
Posts: 16
Joined: 17 Jan 2018, 05:19

Filebot CLI hangs for days with no output

Post by randyest »

I've been using filebot for years and purchased it from the windows store as soon as it was there. It's been working up until a year or so ago where the script I have used so many times would run ok sometimes, but occasionally hang forever (I let it sit 3 days once) with no output. The hanging issue became more frequent over time and now it hangs always. I can't get it to run at all. The GUI works but that's not practical for my purposes.

Sysinfo:

Code: Select all

U:\Video3>filebot -script fn:sysinfo
FileBot 4.9.6 (r9125)
JNA Native: 6.1.2
MediaInfo: 21.09
7-Zip-JBinding: 16.02
Tools: fpcalc/1.5.0
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2022-08-15 (r836)
Groovy: 3.0.9
JRE: OpenJDK Runtime Environment 17.0.2
JVM: OpenJDK 64-Bit Server VM
CPU/MEM: 16 Core / 17 GB Max Memory / 31 MB Used Memory
OS: Windows 10 (amd64)
STORAGE: NTFS [(C:)] @ 526 GB | NTFS [Cygwin] @ 27 GB | NTFS [Incoming 2T] @ 744 GB | NTFS [games] @ 7 GB | NTFS [2T BigApps] @ 804 GB | NTFS [SSD2 256] @ 71 GB | NTFS [incoming] @ 24 TB | NTFS [Steam] @ 80 GB | NTFS [video] @ 16 TB | NTFS [pub] @ 16 TB | NTFS [Music] @ 24 TB | NTFS [pub] @ 24 TB | NTFS [video] @ 7 TB | NTFS [NEW] @ 5 TB | NTFS [photo] @ 16 TB | NTFS [homes] @ 16 TB | NTFS [photo] @ 16 TB
Package: APPX
License: Microsoft Store License
Done ヾ(@⌒ー⌒@)ノ
The .bat file I use includes:

Code: Select all

cmd /c filebot . -script fn:amc --action move --def excludeList=amc.txt --def --output "n:\video" --def minFileSize=0 --def minLengthMS=0 --def subtitles=en --def artwork=y --def deleteAfterExtract=y  --conflict auto -non-strict --log-file ./filebot.log
Here's what I get:

Code: Select all

U:\Video2>cmd /c filebot . -script fn:amc --action move --def excludeList=amc.txt --def --output "n:\video" --def minFileSize=0 --def minLengthMS=0 --def subtitles=en --def artwork=y --def deleteAfterExtract=y  --conflict auto -non-strict --log-file ./filebot.log

... and a blinking cursor underneath. Nothing happens.

I've tried different source and destination drives/folders but it seems to make no difference if there are 5 files to process or 500.

I saw suggestions to (rarely, never scripted) clear the cache, so I tried it and got this error:

Code: Select all

U:\Video2>filebot -clear-cache
Clear cache
* Delete C:\Users\randy\AppData\Roaming\FileBot\cache\0 (6 MB)[/color]
An unexpected error occurred during startup: C:\Users\randy\AppData\Roaming\FileBot\cache\0\.ehcache-diskstore.lock: The process cannot access the file because it is being used by another process
java.nio.file.FileSystemException: C:\Users\randy\AppData\Roaming\FileBot\cache\0\.ehcache-diskstore.lock: The process cannot access the file because it is being used by another process
        at net.filebot.util.FileUtilities$2.visitFile(Unknown Source)
        at net.filebot.util.FileUtilities$2.visitFile(Unknown Source)
        at net.filebot.util.FileUtilities.delete(Unknown Source)        at net.filebot.Main.main(Unknown Source)
Not only is nothing else using that file, it doesn't exist. In fact there is no C:\Users\randy\AppData\Roaming\FileBot directory at all (nor on local or localnow). Could it be somewhere else or do I somehow need run get filebot to make it? This is a default install from the windows store so I didn't specify any special file or data directories.

Finally, I have 4.9.6.(r2195). I see that 4.9.8 is available from https://get.filebot.net/filebot/BETA/ as a x64 PKG or portable. I can't figure out how to install the PKG, and tried the stand-alone GUI. It works but asks me for a license key if I want to rename from the GUI. Is it worth it for me to try to figure out how to install the latest PKG beta (and will it work with my license?)

Thank you in advance for any suggestions.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot CLI hangs for days with no output

Post by rednoah »

:idea: The process cannot access the file because it is being used by another process means that the file path is in use, typically by a FileBot instance. Process Explorer (3rd party tool) can be used to find processes that lock certain files. A Restart will typically resolve any such kind of issue.


:idea: Your filebot -clear-cache output indicates that a FileBot instance is currently running. Perhaps the GUI. Perhaps a background process. On Windows, one process can't delete files that are in use by another.


:idea: The APPX runtime (i.e. all Windows Store applications) will transparently (i.e. the app itself is not aware of this) redirect I/O operations to system protected folders (e.g. user home) to a package-specific folder:

Code: Select all

%USERPROFILE%\AppData\Local\Packages\PointPlanck.FileBot_*\LocalCache\Roaming\FileBot

:idea: *.pkg files are macOS installer packages. If you want to try the latest beta, then you'll want to use the *.zip package, and you will have to request a complimentary license:
viewtopic.php?t=1609


:?: Can you reliably reproduce a test case where it locks right on startup? Really no single line of output? It stops before printing anything? That would be highly unusual. Each FileBot instance will typically print the file path it's trying to lock, before waiting for any lock.



EDIT:

:?: If you have an open CMD window, and if filebot is running, then you can use the PAUSE / BREAK key to see what FileBot is currently doing, and that might give us a clue as to where it's waiting for something:
In Windows operating systems, we can capture a thread dump using the CTRL and Break key combination. To take a thread dump, navigate to the console used to launch the Java application, and press the CTRL and Break keys together.

It's worth noting that, on some keyboards, the Break key isn't available. Therefore, in such cases, a thread dump can be captured using the CTRL, SHIFT, and Pause keys together.

Both of these commands print the thread dump to the console.
:idea: Please read the FAQ and How to Request Help.
randyest
Posts: 16
Joined: 17 Jan 2018, 05:19

Re: Filebot CLI hangs for days with no output

Post by randyest »

Thank you very much for the quick and thorough reply. I did have a filebot instance running when I tried and failed to clean the cache. I killed it, cleaned the cache, and now it is working. Instead of repeating the command line and hanging, it quickly went through ignored file list and is in the input file list stage within seconds. I'm assuming the cache was corrupted by too many unclean shutdowns, which was happening increasingly frequently until I replaced my power supply. I promise never to script-in cache clearing or even use it unless I have exhausted everything else and posted a thread for help. I don't want a ban!

I can't and never could get filebot to lock right on startup before printing anything. It always summarized the command line and went to flashing cursor. Now I wish I'd have CTRL-BREAK'ed to see what was up before I cleared the cache. I'll know for next time if it ever happens again. Man, it has been a rough several weeks with no filebot -- I took for granted having everything organized and plex-friendly.

Thank you for the pointer to the proper appdata location. The history xml file there will be useful for some other plans I have and as a key to re-obtaining some files I lost in a NAS massacre :twisted: The CTRL-BREAK thing will be handy for debugging too; I can't believe I missed that in the docs and the hundreds of threads I've read here.

:D I have a couple of usage questions about clutter if you have time:

--def unsorted=y moves all leftover "clutter" to ./unsorted/*, correct? Is there any way to move the clutter from each source dir into the corresponding destination dir instead of lumping them all into one folder? I want to keep Featurettes and Extras and such with the filebot-cleaned folders and

I know how to use and the effect of filebot -script fn:cleaner "./" --def root=n minsize=0 but is there a way to set a max file size or terms/extensions to ignore (or maybe a maxLengthMS)? I've seen mention of number.clutter.maxfilesize but do not know how to set it. I'd like to use this to delete true clutter, like .nfo, readme.txt, etc. while keeping any extra audio or video files either by term/extension or filesize since my "true" clutter is always very small.

Thanks again for all of your help, and for helping me get filebot running as perfectly as I remember.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot CLI hangs for days with no output

Post by rednoah »

No worries. Especially don't worry about auto-ban. Those systems typically won't notice unless you repeatedly request the same resources hundreds of times per day for many days in a row. Except AniDB can be a bit strict with some ~200 request limit per day.

It could also be a strange issue with the Microsoft Store command launcher. Because it sounds a bit like FileBot code isn't executed in the first place. Let's see next time. The "dump stack traces" keyboard shortcut (not commonly known even to developers, applies to all java programs) might not work if the underlying java process isn't running yet. AFAIK, CMD itself will mirror commands before executing them when running *.bat scripts.



:idea: --def unsorted uses --def unsortedFormat so you can use a custom Plain File Format and rewrite file paths as needed.


:idea: The cleaner script does have a --def maxsize=bytes option. Does that work? Though if you just want to delete *.txt then I'd use PowerShell for that.
:idea: Please read the FAQ and How to Request Help.
Post Reply