filebot-watch

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
thielj
Posts: 55
Joined: 05 Nov 2017, 22:15

filebot-watch

Post by thielj »

The current filebot-watcher container has a few issues.

The major one is the input folder monitoring script based on inotifywait, which can trigger while folders are still being copied, in particular over a slow wireless link. Have a look at docker-inotify-command and docker-filebot here:
https://github.com/coppit

The other is the passing of arguments to the command line. While this doesn't matter for command line users, it's often not supported in embedded environments. In Unraid for example, it's only possible by exploiting an otherwise unchecked configuration value. Using environment variables would be better.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot-watch

Post by rednoah »

My filebot-watcher favors simplicity. It's not necessarily the optimal solution for everyone. In fact, I'd highly discourage anyone from relying on file system events. It'll never work as well as you might hope, especially not from inside a docker container.

The ideal solution is to know when to call the amc script, such as qBT calling the amc script only on newly downloaded folders or files.

If you must use the filebot-watcher container the I'd recommend having a custom staging folder that is not watched, and move completed files/folders into the watched folder in a single atomic filesystem operation.
:idea: Please read the FAQ and How to Request Help.
thielj
Posts: 55
Joined: 05 Nov 2017, 22:15

Re: filebot-watch

Post by thielj »

I Personally understand the implications (and how to work around them), but many people might just install your docker container just to find out it doesn't work as expected.

coppit's solution allows for a settlement period before kicking off AMC which would probably eliminate not all, but most of the problems.

And yes, having a filebot-watcher container on my headless media server seems like a good idea ;)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot-watch

Post by rednoah »

Is 2 seconds ago not long enough as a settlement period? 2 seconds ago is pretty arbitrary. Feel free to suggest an alternative -newermt value.
:idea: Please read the FAQ and How to Request Help.
thielj
Posts: 55
Joined: 05 Nov 2017, 22:15

Re: filebot-watch

Post by thielj »

Would you spin down your hard drives after 2 seconds of no activity?
20 minutes seems to be a safer guess, configurable from an environment variable.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot-watch

Post by rednoah »

2 seconds is a long time for nothing to happen in a newly added folder during a copy operation. Anyway, I'll change it to 5 minutes via ENV then.
:idea: Please read the FAQ and How to Request Help.
thielj
Posts: 55
Joined: 05 Nov 2017, 22:15

Re: filebot-watch

Post by thielj »

I'll have to check the shell code if it works like I would expect, i.e. to delay FileBot until there have been several minutes of no (significant) activity in the watched directory and any subfolder. It looks more like it is piecewise feeding any new stuff to AMC.

I have to understand what happens if the first file added is e.g. a .srt file, followed by 15GByte main movie over a slow connection.

What happens to aborted files, are they retried?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot-watch

Post by rednoah »

There is no difference between an aborted file and a completed file. That's why most sync tools use temporary hidden files and move/rename them afterwords to make the completed file appear in a single atomic filesystem operation.

If you partially copy a file, and then not resume right away, then the "no recent changes" sanity check will be OK, and the amc script will be executed. If you copy some new files/folders every few seconds indefinitely, then the amc script will never be called.

In the case of "small file followed by big file" the small file will initially kick off the watcher, but the "no recent changes" sanity check will fail as the big file is constantly being changed while you're transferring the file. The result is likely that the amc script is never called, until you add some other unrelated files next time to kick off the watcher again.
:idea: Please read the FAQ and How to Request Help.
Post Reply