watcher script not respecting ignored folders

[cstrong]

Hello! I'm using filebot-watcher on my downloads folder, /srv/media/downloads. Incomplete downloads are kept in a subdirectory, /srv/media/downloads/incomplete.

I am watching the downloads folder recursively, so I have excluded the incomplete folder in two places:

• As an amc option: --def ignore="/srv/media/downloads/incomplete/.*"
  • This prevents the amc script from considering anything in the incomplete directory.

• As an inotifywait option, passed via environment variable: INOTIFYWAIT_OPTS: "--recursive @/srv/media/downloads/incomplete".
  • This prevents in-progress downloads from triggering the amc script in the first place.

My configuration works if only one file is being downloaded at a time. As the file downloads, inotifywait ignores it. When it is moved (atomically) from the incomplete folder to the downloads folder, inotify triggers the amc script, which processes the file. Hurray!

This does not work if a download completes while another download is still in progress. The amc script is triggered, but since the still-downloading file was modified <5 seconds ago, processing is deferred.

Logs

Code: Select all

2025-02-06 12:08:16.696	
Setting up watches.  Beware: since -r was given, this may take a while!
	
2025-02-06 12:08:17.782	
Watches established.
	
2025-02-06 12:15:32.566	
[INOTIFY] /srv/media/downloads/movies/Completed Movie
	
2025-02-06 12:15:32.566	
[2025/02/06 12:15:32 PST] Waiting 6 seconds for changes to settle down...
	
2025-02-06 12:15:38.575	
[2025/02/06 12:15:32 PST] /srv/media/downloads/incomplete/movies/Incomplete Movie/Incomplete Movie.mkv.!qB was modified less than 5 seconds ago
	
2025-02-06 12:15:38.575	
Processing deferred until next change...

We see that inotify was triggered on "Completed Movie", but "Incomplete Movie" prevents it from being processed.

Expected behavior
I think the watcher script should respect the configured ignore path. Could the ignore path could be added to this line of the watcher script: (https://github.com/filebot/filebot-dock ... atcher#L15)

Code: Select all

RECENTLY_MODIFIED_FILES="$(find "$1" -type f -newermt "$SETTLE_DOWN_CHECK" -not -path '*/[.@]*' -print -quit)"

Workarounds

• Don't put the incomplete folder underneath the watch folder. I'd rather not do this because this path is shared by several containers, and I'd like to keep my bind mounts clean (with each container only accessing the minimum of what it needs).

• Only download one thing at a time. Not great, but not the end of the world

• Accept that amc will only run after the last download has completed. Not great.

Output of fn:sysinfo

Console Output: Select all

FileBot 5.1.6 (r10435)
JNA Native: 7.0.0
MediaInfo: 24.06
Tools: fpcalc/1.5.1 7z/24.08 unrar/7.01 mkvpropedit/86.0 AtomicParsley/8
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2024-10-16 (r986)
Groovy: 4.0.21
JRE: OpenJDK Runtime Environment 21.0.5
JVM: OpenJDK 64-Bit Server VM
CPU/MEM: 2 Core / 4.2 GB Max Memory / 41 MB Used Memory
OS: Linux (amd64)
HW: Linux fd65449d3f20 6.1.0-30-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.124-1 (2025-01-12) x86_64 x86_64 x86_64 GNU/Linux
CPU/MEM: : Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz / MemTotal: 16 GB / MemFree: 355 MB / MemAvailable: 13 GB / SwapCached: 311 KB / SwapTotal: 1.0 GB / SwapFree: 1.0 GB
STORAGE: overlay [/] @ 63 GB | ext4 [/data] @ 63 GB | ext4 [/srv/media] @ 771 GB
UID/GID: uid=1000(ubuntu) gid=8000(filebot) groups=8000(filebot),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),1000(ubuntu)
DATA: /data/filebot
Package: DOCKER
License: FileBot License P69436474 (Valid-Until: 2026-02-13)
Done ヾ(＠⌒ー⌒＠)ノ

[rednoah]

The official answer is going to be clearly separating input / output folders, as well as clearly separating incomplete / complete download folders.

e.g.

Code: Select all

/srv/media/Downloads/incomplete
/srv/media/Downloads/complete
/srv/media/Plex/Movies
/srv/media/Plex/TV Shows

You can then use -v /srv/media:/srv/media to consistently mount your media folder into all your container folders:

Shell: Select all

-v /srv/media:/srv/media

This will also ensure that move / hardlink operations always work as expected. Using different volume mounts for input / output folders is a typical mistake for users new to docker users as that often leads to unnecessary copy+delete move operations.


Additionally, you won't need --recursive because any newly completed download will add a file or a folder, either of which triggers the file system event.






EDIT:

Parsing the argument array for amc script options such as --def ignore is not feasible within the context of the filebot-watcher shell script. I'd probably build my own container if I needed my own watcher logic.

That said, maybe you can use -newermt "$SETTLE_DOWN_CHECK" to trick find into not finding any recently modified files? Maybe a last modified date in the future will ensure that RECENTLY_MODIFIED_FILES remains empty?

Shell: Select all

-e SETTLE_DOWN_CHECK="-1 seconds ago"

NOTE: Untested. I do not recommend this approach. Use at your own discretion. There may be unexpected side-effects.

[cstrong]

Thanks for the reply. I like your suggested folder structure: it's a lot more elegant than fiddling with the watcher script.