conflict replace gets skipped

[hama4reall]

I have two folders files & Media. I'm sorting everything in files to output it to Media but more than half of the videos is already sorted using Hardlink from files to Media and now i want to redo sorting everything in files to make sure everything is sorted alongside the new videos that aren't yet sorted.
Before that I'm testing before i actually start renaming ~2k files but Now i have come to a problem;
My filebot command is like this

Shell: Select all

~/bin/filebot -rename The.Escape.of.the.Seven.S02E04.1080p.Shafilm.mp4 --output output  -script fn:amc --action hardlink --log INFO --conflict replace -non-strict

(this is for testing before the actual process of sorting ~2k files)
so i run it twice to see if the --conflict replace does override the file in the second attempt. First attempt it's fine it sorted it

Console Output: Select all

Input: /mnt/mpathc/hamashareef/Lab/The.Escape.of.the.Seven.S02E04.1080p.Shafilm.mp4
Group files by movie or series
Group: {Series=The Escape of the Seven (2023)} => [The.Escape.of.the.Seven.S02E04.1080p.Shafilm.mp4]
Rename episodes using [TheMovieDB] with [Airdate Order]
Lookup via [The Escape of the Seven, Runaway] [hamashareef, mpathc, The Escape of the Seven]
Fetching episode data for [The Escape of the Seven]
* The Escape of the Seven [200707] | 33 episodes | 1x01-17 .. 2x01-16
Fetching episode data for [Runaway]
* The Escape of the Seven [200707] | 33 episodes | 1x01-17 .. 2x01-16
* Runaway [198189] | 8 episodes | 1x01-08
[HARDLINK] from [/mnt/mpathc/hamashareef/Lab/The.Escape.of.the.Seven.S02E04.1080p.Shafilm.mp4] to [/mnt/mpathc/hamashareef/Lab/output/TV Shows/The Escape of the Seven {tmdb-200707}/Season 02/The Escape of the Seven - S02E04 - Episode 4.mp4]
Processed 1 file
Done ヾ(＠⌒ー⌒＠)ノ

But in the second attempt:

Console Output: Select all

[HARDLINK] Skipped [/mnt/mpathc/hamashareef/Lab/The.Escape.of.the.Seven.S02E04.1080p.Shafilm.mp4] because [/mnt/mpathc/hamashareef/Lab/output/TV Shows/The Escape of the Seven {tmdb-200707}/Season 02/The Escape of the Seven - S02E04 - Episode 4.mp4] already exists
Finished without processing any files

It ignored the "--conflict replace" option even i tried with "--conflict override" or "--conflict OVERRIDE" as i saw the typos in some of the forum but still same output.

My Environment:

Console Output: Select all

FileBot 5.1.3 (r10185) / Groovy 4.0.15 / OpenJDK Runtime Environment 21.0.2 / Linux 6.1.67-gentoo-whatbox (amd64)

I'm on a shared seedbox with a non-root user i compiled openjdk and compiled filebot and give the absolute path of java to filebot.

[rednoah]

You'll want to process only files that have not yet been processed.


Since you're using hardlinks, the link count will tell you if the file has been processed or not. See How do I process only specific kinds of files? › Option 4: Start FileBot with --file-filter from Terminal for details and examples.


Alternatively, the existence / non-existence of xattr metadata. See Re-process previously organized files using local xattr metadata for details.

[hama4reall]

Ps: the reason i want it to replace the videos is because initially i tried with the files folder not knowing the issue but then after it sorted the files i saw in the log that every duplicate/hardlinked was skipped and right after it was skipped filebot sorted them to the Unsorted folder which is not what i want because 1,681 files where put into the Unsorted folder because they already existed in the Media Folder.
In That initial process i run filebot with the filebot-watcher bash script that i modified like so:
https://pastebin.com/uAM9dsKk

With the command

Format: Select all

screen -S filebot ~/filebot-watcher.sh ~/files --output ~ --def jellyfin=xxx --conflict replace --action hardlink --def minLengthMS=3000 --def minFileSize=1000 --def excludeList=~/.config/filebot/amc-exclude-list.txt --def storeReport=~/.config/filebot/report --log-file ~/.config/filebot/amc.log --def movieFormat="Media/{f.path.match(/Badini/).upperInitial()}/{hd==/UHD/?'4KMovies':'Movies'}/{any{f.path.match(/Short/).upperInitial()}{f.path.match(/Anime/).upperInitial()}{anime?'Anime':''}{f.path.match(/Documentary/).upperInitial()}{genre=~/Documentary/?'Documentary':''}{genres=~/Animation/?'Animation':''}{info.OriginalLanguage.toLocale().displayLanguage}}/{~jellyfin.id}" seriesFormat="Media/{f.path.match(/Badini/).upperInitial()}/{hd==/UHD/?'4KShows':'Shows'}/{any{file.path.match(/Pesrian/).upperInitial()}{f.path.match(/Short/).upperInitial()}{f.path.match(/Anime/).upperInitial()}{anime?'Anime':''}{f.path.match(/Documentary/).upperInitial()}{genre=~/Documentary/?'Documentary':''}{genres=~/Animation/?'Animation':''}{info.OriginalLanguage.toLocale().displayLanguage}}/{~jellyfin.id}" animeFormat="Media/{f.path.match(/Badini/).upperInitial()}/{hd==/UHD/?'4KShows':'Shows'}/{file.path.match(/Perisan/).upperInitial()}/{any{file.path.match(/Anime/).upperInitial()}{anime?'Anime':''}}/{~jellyfin.id}" musicFormat="Media/Music/{~plex}" unsortedFormat="Media/Unsorted/{fn}.{ext}" --def movieDB=TheMovieDB seriesDB=TheMovieDB::TV animeDB=TheMovieDB::TV musicDB=ID3

The output was (couldn't upload it to pastebin)
https://drive.google.com/file/d/1qE0oeh ... sp=sharing

[hama4reall]

You'll want to process only files that have not yet been processed.


Since you're using hardlinks, the link count will tell you if the file has been processed or not. See How do I process only specific kinds of files? › Option 4: Start FileBot with --file-filter from Terminal for details and examples.

I just saw your post before i post the last post, with

Format: Select all

--file-filter "f.linkCount == 1"

I'll be able to only process the files that haven't been hardlinked right? The rest will be ignored.

[rednoah]

If you use --action hardlink and --conflict replace and if source / destination are already the same file then FileBot will still skip the operation since deleting the hardlink and then creating the same hardlink again is effectively no operation:

Code: Select all

[HARDLINK] Skipped [A] because [B] already exists

--def unsorted is a higher level behaviour specific to the amc script and I guess it just doesn't mesh well with your use case. The amc script does not support fine-grained "unsorted for some reasons but not other reasons" behaviours. You'll want to disable --def unsorted or just process not-yet-processed files that don't fall into --def unsorted for the reason above.

[hama4reall]

I took your advise and skipped the already sorted files with the

Shell: Select all

--def excludeLink=y

from viewtopic.php?f=4&t=215 which skips sombolic and hard links. This avoided the --def unsorted issue and sorted all my not sorted files with both

Shell: Select all

--conflict auto --action hardlink

options.

[rednoah]

--def excludeLink=y is a good solution. I forgot the amc script had its own built-in option for skipping already-hardlinked files as well.

[hama4reall]

you already been a big help explaining the issue and referencing to solutions. thanks a lot