Page 1 of 1

filebot-watch

Posted: 22 Nov 2017, 15:12
by thielj
The current filebot-watcher container has a few issues.

The major one is the input folder monitoring script based on inotifywait, which can trigger while folders are still being copied, in particular over a slow wireless link. Have a look at docker-inotify-command and docker-filebot here:
https://github.com/coppit

The other is the passing of arguments to the command line. While this doesn't matter for command line users, it's often not supported in embedded environments. In Unraid for example, it's only possible by exploiting an otherwise unchecked configuration value. Using environment variables would be better.

Re: filebot-watch

Posted: 22 Nov 2017, 15:45
by rednoah
My filebot-watcher favors simplicity. It's not necessarily the optimal solution for everyone. In fact, I'd highly discourage anyone from relying on file system events. It'll never work as well as you might hope, especially not from inside a docker container.

The ideal solution is to know when to call the amc script, such as qBT calling the amc script only on newly downloaded folders or files.

If you must use the filebot-watcher container the I'd recommend having a custom staging folder that is not watched, and move completed files/folders into the watched folder in a single atomic filesystem operation.

Re: filebot-watch

Posted: 22 Nov 2017, 16:22
by thielj
I Personally understand the implications (and how to work around them), but many people might just install your docker container just to find out it doesn't work as expected.

coppit's solution allows for a settlement period before kicking off AMC which would probably eliminate not all, but most of the problems.

And yes, having a filebot-watcher container on my headless media server seems like a good idea ;)

Re: filebot-watch

Posted: 22 Nov 2017, 17:11
by rednoah
Is 2 seconds ago not long enough as a settlement period? 2 seconds ago is pretty arbitrary. Feel free to suggest an alternative -newermt value.

Re: filebot-watch

Posted: 22 Nov 2017, 18:04
by thielj
Would you spin down your hard drives after 2 seconds of no activity?
20 minutes seems to be a safer guess, configurable from an environment variable.

Re: filebot-watch

Posted: 22 Nov 2017, 18:39
by rednoah
2 seconds is a long time for nothing to happen in a newly added folder during a copy operation. Anyway, I'll change it to 5 minutes via ENV then.

Re: filebot-watch

Posted: 24 Nov 2017, 01:38
by thielj
I'll have to check the shell code if it works like I would expect, i.e. to delay FileBot until there have been several minutes of no (significant) activity in the watched directory and any subfolder. It looks more like it is piecewise feeding any new stuff to AMC.

I have to understand what happens if the first file added is e.g. a .srt file, followed by 15GByte main movie over a slow connection.

What happens to aborted files, are they retried?

Re: filebot-watch

Posted: 24 Nov 2017, 01:47
by rednoah
There is no difference between an aborted file and a completed file. That's why most sync tools use temporary hidden files and move/rename them afterwords to make the completed file appear in a single atomic filesystem operation.

If you partially copy a file, and then not resume right away, then the "no recent changes" sanity check will be OK, and the amc script will be executed. If you copy some new files/folders every few seconds indefinitely, then the amc script will never be called.

In the case of "small file followed by big file" the small file will initially kick off the watcher, but the "no recent changes" sanity check will fail as the big file is constantly being changed while you're transferring the file. The result is likely that the amc script is never called, until you add some other unrelated files next time to kick off the watcher again.

Re: filebot-watch

Posted: 09 Feb 2023, 06:54
by rednoah