Possible bug - series matching and --conflict + KEEPLINK action

[devster]

I recently acquired and processed episodes from an amazing Italian series, Boris.
I found 2 things which confused me a bit: the first one, very minor, is that despite processing the entire season at once (passing the main folder as argument) and FileBot detecting it as a single group, it still matched each episode on its own, in some cases causing mismatches; the second one was that despite --conflict skip being passed as option, one of the episodes got relinked in a wrong way, effectively losing the original file.

First issue:

Code: Select all

Parameter: ut_title = Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST
...
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x01.La.mia.Africa.Parte.1.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x02.La.mia.Africa.Parte 2.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x03.Chi.si.salvera.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x04.Il.travestimento.e.saltato.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x05.L.affaire.Martellone.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x06.No.logo.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x07.A.morte.il.conte.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x08.Il.cielo.sopra.Stanis.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x09.La.figlia.di.Mazinga.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x10.Un.tuzzo.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x11.L.Italia.che.lavora.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x12.Usa.la.forza.Ferretti.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x13.Il.sordomuto.il.senatore.e.gli.equilibri.del.Paese.Parte.1.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Input: /srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x14.Il.sordomuto.il.senatore.e.gli.equilibri.del.Paese.Parte.2.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv
Group: [Series:boris] => [Boris.S02x01.La.mia.Africa.Parte.1.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x02.La.mia.Africa.Parte 2.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x03.Chi.si.salvera.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x04.Il.travestimento.e.saltato.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x05.L.affaire.Martellone.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x06.No.logo.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x07.A.morte.il.conte.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x08.Il.cielo.sopra.Stanis.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x09.La.figlia.di.Mazinga.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x10.Un.tuzzo.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x11.L.Italia.che.lavora.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x12.Usa.la.forza.Ferretti.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x13.Il.sordomuto.il.senatore.e.gli.equilibri.del.Paese.Parte.1.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv, Boris.S02x14.Il.sordomuto.il.senatore.e.gli.equilibri.del.Paese.Parte.2.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv]
...
[KEEPLINK] from [/srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x05.L.affaire.Martellone.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv] to [/cephfs/Media/TV Shows/Boris (2013)/Season 2/Boris - S02E20 - [720p AVC - 2.0 AC-3 - WEBDL].mkv]
Skipped [/srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x11.L.Italia.che.lavora.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv] because [/cephfs/Media/TV Shows/Boris (2013)/Season 2/Boris - S02E20 - [720p AVC - 2.0 AC-3 - WEBDL].mkv] already exists

While the rest of the episodes were correctly matched to Boris (2007) as seen below.

The second issue is the following:

Code: Select all

already existing, processed in the previous round
[KEEPLINK] from [/srv/pv/torrent/downloads/complete/tv_shows/Boris.S02x01-14.iTALiAN.NF.HD.WEBDL.720.H264-BST/Boris.S02x14.Il.sordomuto.il.senatore.e.gli.equilibri.del.Paese.Parte.2.iTALiAN.NF.HD.WEBDL.720.H264-BST.mkv] to [/cephfs/Media/TV Shows/Boris (2007)/Season 2/Boris - S02E14 - Il sordomuto, il senatore e gli equilibri del Paese (seconda parte) [720p AVC - 2.0 AC-3 - WEBDL].mkv]

new file, shouldn't have been linked again
[KEEPLINK] from [/srv/pv/torrent/downloads/complete/tv_shows/Boris.S03x01-14.iTALiAN.NF.HD.720p.WEBDL.H264-BST/Boris.S03x02.Un.altra.televisione.e.possibile.Parte.2.iTALiAN.NF.HD.720p.WEBDL.H264-BST.mkv] to [/cephfs/Media/TV Shows/Boris (2007)/Season 2/Boris - S02E14 - Il sordomuto, il senatore e gli equilibri del Paese (seconda parte) [720p AVC - 2.0 AC-3 - WEBDL].mkv]

And this is how I launch FileBot (part of a post-processing script of qBittorrent)

Code: Select all

FB=("$FILEBOT" -script fn:amc --action keeplink --output "$MEDIA_OUTPUT" --conflict skip \
--filter \"'!readLines('\'"$SCRIPTS"'/excludes.txt'\'').contains(n)'\" \
-non-strict --log-file amc.log --def excludeList=".excludes" \
--def ut_dir="$_arg_qb_root" ut_kind="$_qb_multi" ut_title="$_arg_qb_name" ut_label="$_arg_qb_category" \
--def @"$SCRIPTS"/notify.txt \
--def movieFormat=@"$SCRIPTS"/movieFormat.groovy \
--def seriesFormat=@"$SCRIPTS"/seriesFormat.groovy \
--def animeFormat=@"$SCRIPTS"/animeFormat.groovy)

log "${FB[@]}"

"${FB[@]}"

[rednoah]

1.

is that despite processing the entire season at once (passing the main folder as argument) and FileBot detecting it as a single group, it still matched each episode on its own, in some cases causing mismatches

The S02x01-14 pattern quite possibly means FileBot identifies each file as (potentially) being multi-episode S02E01-E14 and since matching only allows one episode to match one file, FileBot will process each file within it's own group (kinda like how it would process 2x01.1080p and 2x01.720p in separate groups even if you pass them in together).


2.

despite --conflict skip being passed as option, one of the episodes got relinked in a wrong way, effectively losing the original file.

I don't see how the original file got lost. The first one was moved. The second one was skipped. None were deleted.


3.
Does your filter not completely exclude Boris (2013)? How can FileBot match episodes that your filter would have excluded in the first place? Is --filter not working?

[devster]

1. My bad, didn't think about that.

2. Episode 2x14 got moved to /cephfs/Media/TV Shows/Boris (2007)/Season 2/Boris - S02E14 - Il sordomuto, il senatore e gli equilibri del Paese (seconda parte) [720p AVC - 2.0 AC-3 - WEBDL].mkv while processing season 2.

When processing season 3, episode 3x02 got mismatched as 2x14, it was subsequently moved to /cephfs/Media/TV Shows/Boris (2007)/Season 2/Boris - S02E14 - Il sordomuto, il senatore e gli equilibri del Paese (seconda parte) [720p AVC - 2.0 AC-3 - WEBDL].mkv which was already existing and linked to the original location.

I'm surmising the "move" by the difference in size of the files, the 2x14 episode was about 600Mb while the 3x02 was over a Gb. The original 2x14 file was nowhere to be found.

3. I originally didn't have "Boris (2013)" in the filter, I didn't know it existed, it does exclude it now, but I already processed all three seasons.

[rednoah]

2.
I can't reproduce this issue:

Code: Select all

[KEEPLINK] from [Firefly - S01E01 - The Train Job (A).avi] to [TV Shows/Firefly/Season 01/Firefly - S01E01 - The Train Job.avi]
Skipped [Firefly - S01E01 - The Train Job (B).avi] because [TV Shows/Firefly/Season 01/Firefly - S01E01 - The Train Job.avi] already exists

There isn't really anything that can go wrong in the code, unless the OS incorrectly reports the file as not existing when it actually already does:

Code: Select all

if (!destination.equals(source) && !destination.exists()) {
	log.info(format("[%s] from [%s] to [%s]", renameAction, source, destination));
	destination = renameAction.rename(source, destination);

	// remember successfully renamed matches for history entry and possible revert
	renameLog.put(source, destination);
} else {
	log.info(format("Skipped [%s] because [%s] already exists", source, destination));
}

[devster]

It's the first time it's happening for me as well. Never happened before, I'll keep an eye out if it happens again.
It may very well be related to the filesystem I'm using: Ceph.

[rednoah]

If you can reproduce the issue in a test case, then maybe we could get more information about the filesystem calls via strace. See what FileBot gets back from the OS fs driver.