[SOLVED] Filebot 4.8.5 switching up anime on Synology

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

[SOLVED] Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 28 Apr 2019, 15:16

Hi Rednoah,

I've come to ask for your help once again. This time for something very strange:
The new anime season has started, and this season Filebot seems to switch up some series, and I have no clue why.

You can see it happen in the following excerpt from my log:

Code: Select all

Script run started at: Sun Apr 28 12:00:03 CEST 2019.
Run script [fn:amc] at [Sun Apr 28 12:00:19 CEST 2019]
Input: /volume1/video/Completed/[HorribleSubs] Boruto - Naruto Next Generations - 104 [1080p].mkv
Input: /volume1/video/Completed/[HorribleSubs] Fairy Tail Final Season - 306 [1080p].mkv
Input: /volume1/video/Completed/[HorribleSubs] Kimetsu no Yaiba - 04 [1080p].mkv
Input: /volume1/video/Completed/[HorribleSubs] One Piece - 882 [1080p].mkv
Rename episodes using [TheTVDB]
Auto-detected query: [Boruto Naruto Next Generations, Fairy Tail, boruto, fairy tail final season, kimetsu no yaiba, one piece]
Fetching episode data for [Boruto: Naruto Next Generations]
Fetching episode data for [boruto]
Fetching episode data for [Fairy Tail]
Fetching episode data for [Demon Slayer: Kimetsu no Yaiba]
Fetching episode data for [One Piece]
Fetching episode data for [None Piece]
Fetching episode data for [One Pace]
Fetching episode data for [One Piece Cut To Manga]
Apply filter [age <= 30] on [1771] items
[MOVE] from [/volume1/video/Completed/[HorribleSubs] Boruto - Naruto Next Generations - 104 [1080p].mkv] to [/volume1/video/TV Shows/Demon Slayer - Kimetsu no Yaiba/Season 01/Demon Slayer - Kimetsu no Yaiba - S01E04 - Final Selection.mkv]
[MOVE] from [/volume1/video/Completed/[HorribleSubs] Fairy Tail Final Season - 306 [1080p].mkv] to [/volume1/video/TV Shows/Fairy Tail/Season 08/Fairy Tail - S08E29 - The Winter Wizard.mkv]
Skipped [/volume1/video/Completed/[HorribleSubs] Kimetsu no Yaiba - 04 [1080p].mkv] because [/volume1/video/TV Shows/One Piece/Season 20/One Piece - S20E04 - The Next Move - New Obsessive Fleet Admiral Sakazuki.mkv] already exists
[MOVE] from [/volume1/video/Completed/[HorribleSubs] One Piece - 882 [1080p].mkv] to [/volume1/video/TV Shows/One Piece/Season 20/One Piece - S20E05 - The Summit War - Pirate King's Inherited Will.mkv]
Processed 3 files
The detections for Fairy Tail and One Piece work, but those for Kimetsu no Yaiba and Boruto are complete nonsense.

My script is as follows:

Code: Select all

#!/bin/sh
# Log current date and time
echo -e "\nScript run started at: $(date)." >> "/volume1/torrents/amc_log.txt"
# Set RAM to 512MB - otherwise process will run out of memory
export JAVA_OPTS="-Xmx512m"
# Show-specific queries to fix incorrect detection
/usr/bin/find /volume1/video/Completed -iname '*Tate no Yuusha*.mkv' -not -path "*@eaDir*" -exec /volume1/@appstore/filebot/filebot.sh -rename --action move --output /volume1/video/Completed --db thetvdb --format "{n} - {order.airdate.s00e00} - {t.replaceAll("/:/", ' -').replaceAll("/!/", '')}" -non-strict --q 'The Rising of the Shield Hero' {} + | tee -a "/volume1/torrents/amc_log.txt"
# Automated queries
filebot -script 'fn:amc' /volume1/video/Completed --output "/volume1/video" --action move --db thetvdb -non-strict --conflict skip --lang en --filter 'age <= 30' --def 'ut_label=tv' 'skipExtract=y' --log fine | tee -a "/volume1/torrents/amc_log.txt"
Running FileBot Node manually gives the same result. FileBot for PC does work correctly, but even if I rename the files first on PC, and then try to move them using the Synology version, it will still mix them up.

It is also not a one-time error. The same two series have been detected incorrectly for the past three weeks now.

Do you have any clue what could be happening here?
I've tried reinstalling all FileBot-related packages, but that did not help.

Thanks in advance!
Last edited by xIntenso on 12 May 2019, 23:36, edited 2 times in total.

User avatar
rednoah
The Source
Posts: 16578
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot 4.8.5 switching up anime on Synology

Post by rednoah » 28 Apr 2019, 15:40

1.
--log fine might prevent you to see what's going on, i.e. you're not seeing which episodes your --filter explicitly includes.

:idea: Best guess, the filter excluded all episodes of Boruto, and the "next best" possible match was some other episode data that was under consideration for the current batch.


2.
Here's what I get:

Code: Select all

Input: [HorribleSubs] Boruto - Naruto Next Generations - 104 [1080p].mkv
Input: [HorribleSubs] Kimetsu no Yaiba - 04 [1080p].mkv
Group: [Series:true] => [[HorribleSubs] Boruto - Naruto Next Generations - 104 [1080p].mkv, [HorribleSubs] Kimetsu no Yaiba - 04 [1080p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Boruto Naruto Next Generations, boruto, kimetsu no yaiba]
Fetching episode data for [Boruto: Naruto Next Generations]
Fetching episode data for [boruto]
Fetching episode data for [Demon Slayer: Kimetsu no Yaiba]
[TEST] from [[HorribleSubs] Boruto - Naruto Next Generations - 104 [1080p].mkv] to [TV Shows/Boruto - Naruto Next Generations/Season 01/Boruto - Naruto Next Generations - S01E04 - A Ninjutsu Battle of the Sexes.mkv]
[TEST] from [[HorribleSubs] Kimetsu no Yaiba - 04 [1080p].mkv] to [TV Shows/Demon Slayer - Kimetsu no Yaiba/Season 01/Demon Slayer - Kimetsu no Yaiba - S01E04 - Final Selection.mkv]

3.
In this specific use case, it might make sense to call the amc script for each individual file, so you can't get spill-over effects, though spill-over effects are the result of there not being a correct match, so the best case scenario here might just be exiting with an error.
:idea: Please read the FAQ and How to Request Help.

xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

Re: Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 28 Apr 2019, 15:51

Thanks for the quick response.
I'll check if I can find anything with --log all set.

The strange part is that I just tried with Kimetsu no Yaiba on its own, and it worked perfectly. Only if I combine it with different files it will act strangely. Does FileBot assume that files in the same folder belong to the same series?
Another strange thing is that Boruto has been indexed using this script for multiple seasons. Only this season it started being detected incorrectly.

I can't SSH into the NAS remotely, and I have to reset the xattr of the files to try again, so I'll get back to you in a few hours when I'm home.

Edit: Whoops, I didn't read 3) before replying. So basically what this means is that if FileBot cannot detect any matching show for a certain file, it will take a rejected solution from the previous file instead of failing?

User avatar
rednoah
The Source
Posts: 16578
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot 4.8.5 switching up anime on Synology

Post by rednoah » 28 Apr 2019, 16:15

2.
FileBot does not assume that files in the same folder belong to the same series. It does however process files in groups, which may or may not belong to the same series, it might help if they do though. However, if TV mode is forced, then it seems that all files are processed in a single TV group.

Code: Select all

Group: [Series:true] => [...]

3.
Not really, FileBot does not know which file belongs to which series and makes no assumption. It just so happens that it almost always works. It takes all files, matches all series names, queries for all series names and collects all episodes, and then finds the best combination between files and episodes.

TL;DR the same logic that usually gives you the correct match, will give you the "next best" but completely wrong match, if there is no correct match in the first place (among the matches being considered)
:idea: Please read the FAQ and How to Request Help.

xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

Re: Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 29 Apr 2019, 11:36

2 & 3)
Okay, I see. Thanks for explaining it clearly.

For some reason it seems to work again for now. I'll report back next week when the same problem will either occur again or not.

Once again thanks for the quick help!

User avatar
rednoah
The Source
Posts: 16578
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot 4.8.5 switching up anime on Synology

Post by rednoah » 29 Apr 2019, 11:52

That happens. FileBot takes into account data that changes over time.

Most likely, the airdate was undefined at the time, and thus the filter excluded the correct episode.
:idea: Please read the FAQ and How to Request Help.

xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

Re: Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 06 May 2019, 11:30

New week, same problem, but now with detailed logging...

The new log:

Code: Select all

Script run started at: Sun May  5 12:00:03 CEST 2019.
Run script [fn:amc] at [Sun May 05 12:00:18 CEST 2019]
Parameter: ut_label = tv
Parameter: skipExtract = y
Argument[0]: /volume1/video/Completed
Ignore system path: /volume1/video/Completed/@eaDir
Input: /volume1/video/Completed/[HorribleSubs] Boruto - Naruto Next Generations - 105 [1080p].mkv
Input: /volume1/video/Completed/[HorribleSubs] Fairy Tail Final Season - 307 [1080p].mkv
Input: /volume1/video/Completed/[HorribleSubs] Kimetsu no Yaiba - 05 [1080p].mkv
Input: /volume1/video/Completed/[HorribleSubs] One Piece - 883 [1080p].mkv
Group: [Series:true] => [[HorribleSubs] Boruto - Naruto Next Generations - 105 [1080p].mkv, [HorribleSubs] Fairy Tail Final Season - 307 [1080p].mkv, [HorribleSubs] Kimetsu no Yaiba - 05 [1080p].mkv, [HorribleSubs] One Piece - 883 [1080p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Boruto Naruto Next Generations, Fairy Tail, boruto, fairy tail final season, kimetsu no yaiba, one piece]
Fetching episode data for [Boruto: Naruto Next Generations]
Fetching episode data for [Boruto: Naruto Next Generations]
Fetching episode data for [Fairy Tail]
Fetching episode data for [Demon Slayer: Kimetsu no Yaiba]
Fetching episode data for [One Piece]
Fetching episode data for [None Piece]
Fetching episode data for [One Pace]
Fetching episode data for [One Piece Cut To Manga]
Apply filter [age <= 14] on [1773] items
Include [Boruto: Naruto Next Generations - 1x103 - Migration Season]
Include [Boruto: Naruto Next Generations - 1x104 - The Little Roommate]
Include [Boruto: Naruto Next Generations - 1x105 - A Wound on The Heart]
Include [Fairy Tail - 8x28 - White Dragneel]
Include [Fairy Tail - 8x29 - The Winter Wizard]
Include [Fairy Tail - 8x30 - Gray and Juvia]
Include [Demon Slayer: Kimetsu no Yaiba - 1x04 - Final Selection]
Include [Demon Slayer: Kimetsu no Yaiba - 1x05 - My Own Steel]
Include [One Piece - 20x04 - The Next Move - New Obsessive Fleet Admiral Sakazuki]
Include [One Piece - 20x05 - The Summit War - Pirate King's Inherited Will]
Include [One Piece - 20x06 - One Step Ahead of the Dream - Shirahoshi's Path to the Sun!]
Include [One Pace - 19x06 - Enies Lobby 6]
Include [One Pace - 26x16 - Marineford 16]
[MOVE] from [/volume1/video/Completed/[HorribleSubs] Boruto - Naruto Next Generations - 105 [1080p].mkv] to [/volume1/video/TV Shows/Demon Slayer - Kimetsu no Yaiba/Season 01/Demon Slayer - Kimetsu no Yaiba - S01E05 - My Own Steel.mkv]
[MOVE] from [/volume1/video/Completed/[HorribleSubs] Fairy Tail Final Season - 307 [1080p].mkv] to [/volume1/video/TV Shows/Fairy Tail/Season 08/Fairy Tail - S08E30 - Gray and Juvia.mkv]
Skipped [/volume1/video/Completed/[HorribleSubs] Kimetsu no Yaiba - 05 [1080p].mkv] because [/volume1/video/TV Shows/One Piece/Season 20/One Piece - S20E05 - The Summit War - Pirate King's Inherited Will.mkv] already exists
[MOVE] from [/volume1/video/Completed/[HorribleSubs] One Piece - 883 [1080p].mkv] to [/volume1/video/TV Shows/One Piece/Season 20/One Piece - S20E06 - One Step Ahead of the Dream - Shirahoshi's Path to the Sun.mkv]
Processed 3 files
Done ヾ(@⌒ー⌒@)ノ

Script run started at: Sun May  5 18:00:02 CEST 2019.
Run script [fn:amc] at [Sun May 05 18:00:14 CEST 2019]
Parameter: ut_label = tv
Parameter: skipExtract = y
Argument[0]: /volume1/video/Completed
Ignore system path: /volume1/video/Completed/@eaDir
Input: /volume1/video/Completed/[HorribleSubs] Kimetsu no Yaiba - 05 [1080p].mkv
Group: [Series:true] => [[HorribleSubs] Kimetsu no Yaiba - 05 [1080p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [kimetsu no yaiba]
Fetching episode data for [Demon Slayer: Kimetsu no Yaiba]
Apply filter [age <= 14] on [13] items
Include [Demon Slayer: Kimetsu no Yaiba - 1x04 - Final Selection]
Include [Demon Slayer: Kimetsu no Yaiba - 1x05 - My Own Steel]
Skipped [/volume1/video/Completed/[HorribleSubs] Kimetsu no Yaiba - 05 [1080p].mkv] because [/volume1/video/TV Shows/Demon Slayer - Kimetsu no Yaiba/Season 01/Demon Slayer - Kimetsu no Yaiba - S01E05 - My Own Steel.mkv] already exists
Processed 0 files
Abort (×_×)
As you can see, all correct episodes are listed, but two of them are assigned to the wrong files somehow.
My theory is that the first assignment is the one that messes the 3rd one up, since it consumes the correct detection for Kimetsu no Yaiba, leaving FileBot no choice but to select the next best thing, which is some random other title. This is also evidenced by the second log, where KnY gets detected correctly on its own.

I also have a 2nd theory: I think this behavior started when Boruto passed episode 100, and I've seen the FileBot standalone application have troubles with this too. It will somehow detect "Series X - 104" as "Series X - S01E04" instead of "Series X - S01E104".
This is the main reason why I have included the "age <= 14" filter, and this has usually worked for me, but it seems episode numbers of the shape X0X will sometimes cause trouble.
:!: In fact, you can see this happen in your own logfile a few posts back, I just noticed It detects S01E04 instead of S01E104.

What do you think, Rednoah? And if my theory is correct, how would I fix this?

Edit:
Renaming the last part of the Boruto file to " - S1E105" from " - 105" seems to indeed fix the problem. However, doing this manually every time for the coming 5 weeks does not exactly sound like a very practical solution...

User avatar
rednoah
The Source
Posts: 16578
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot 4.8.5 switching up anime on Synology

Post by rednoah » 06 May 2019, 12:18

1.
Looks like --def ut_label=TV leads to unforutnate grouping if you don't process one completed file / folder a time.

Try putting your files into a TV Shows folder, instead of using ut_label, that'll also force TV mode, but possibly lead to more fortunate grouping:

Code: Select all

/volume1/video/Completed/TV Shows

2.
If you're Linux savvy, then you can use also find -type -exec or filebot -mediainfo -exec to call the amc script once per file, and not just once for all files. That'll also lead to better grouping.


3.
If you know in advance that you're processing absolute numbered episodes, then you can use --order absolute to make FileBot work primarily with absolute numbers. You can always map to SxE numbers after matching via the format. See the following thread for details.
viewtopic.php?f=3&t=2769

:idea: Note that by default, the amc script would be using AniDB for these files, so it would work correctly out of the box since AniDB only has absolute numbers.
:idea: Please read the FAQ and How to Request Help.

xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

Re: Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 07 May 2019, 16:29

Thanks for the suggestions!

I've gone with option 3. I'll be using --order absolute and converting my filename to airdate using the format from the suggested link, albeit in a slightly changed form.
For anyone else who might stumble upon this thread, this is what I've used:

Code: Select all

EDIT: Code removed, as it is now outdated. See next post.
It should basically replicate the TheTVDB layout FileBot AMC uses by default, except it takes absolute order input instead of airdate order input.

I'll let you know if it works next week!
Last edited by xIntenso on 08 May 2019, 11:47, edited 1 time in total.

User avatar
rednoah
The Source
Posts: 16578
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot 4.8.5 switching up anime on Synology

Post by rednoah » 07 May 2019, 20:38

I think your custom format is fairly equivalent to the {plex} format, so this one might be easier and get the job done just as well:

Code: Select all

{order.airdate.plex}
:idea: Please read the FAQ and How to Request Help.

xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

Re: Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 08 May 2019, 11:49

As always, you are right. Also, using --format seems to no longer work when order is set to Absolute, so I used --def seriesFormat instead.

This results in the following code:

Code: Select all

filebot -script 'fn:amc' /volume1/video/Completed --output "/volume1/video" --action move --db thetvdb -non-strict --order Absolute --conflict skip --lang en --filter 'age < 7' --def 'ut_label=tv' 'seriesFormat=TV Shows/{order.airdate.plex.tail}' --log all

User avatar
rednoah
The Source
Posts: 16578
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot 4.8.5 switching up anime on Synology

Post by rednoah » 08 May 2019, 12:08

The amc script completely ignores the --format option in favor of --def movieFormat / seriesFormat / etc for practical reasons, i.e. it needs more than one.
:idea: Please read the FAQ and How to Request Help.

xIntenso
Posts: 22
Joined: 14 Jan 2018, 23:39

Re: Filebot 4.8.5 switching up anime on Synology

Post by xIntenso » 12 May 2019, 23:35

Good news, everything works as I expect it.
Thanks once again for the great, speedy help!

Post Reply