Page 1 of 1
[AMC] Failed to fetch episode data
Posted: 22 Jun 2018, 17:51
by Kuchiru
Hi,
AMC works fine for literally everything except this one show which wont fetch the data with whatever i try.
Version: FileBot 4.7.9 (r4984) / Java(TM) SE Runtime Environment 1.8.0_171 (headless) (can't find the update)
Show:
Quanzhi Gaoshou
Input:
Code: Select all
filebot -script fn:amc --output "$HOME/downloads" --log-file "$HOME/amc.log" --action hardlink --conflict override -non-strict --def "clean=y" "ut_dir=/home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou" "ut_kind=multi" "ut_title=Quanzhi Gaoshou"
Output:
Code: Select all
Locking /home/sbuser/amc.log
Run script [fn:amc] at [Fri Jun 22 19:30:47 CEST 2018]
Parameter: clean = y
Parameter: ut_dir = /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou
Parameter: ut_kind = multi
Parameter: ut_title = Quanzhi Gaoshou
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 01 [1080p][C3AA0F05].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 02 [1080p][26E86DCC].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 03 [1080p][9FAFB01C].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 04 [1080p][E737BA3F].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 05 [1080p][85ACE5F3].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 06 [1080p][B951BAF9].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 07 [1080p][5863C616].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 08 [1080p][4B014B0B].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 09 [1080p][BEE542CB].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 10 [1080p][5C3869D5].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 11 [1080p][BC2C5707].mkv
Input: /home/sbuser/rtorrent/download/Anime/Quanzhi Gaoshou/[Aira] Quanzhi Gaoshou - 12 [1080p][0A818576].mkv
Group: [anime:quanzhi gaoshou] => [[Aira] Quanzhi Gaoshou - 01 [1080p][C3AA0F05].mkv, [Aira] Quanzhi Gaoshou - 02 [1080p][26E86DCC].mkv, [Aira] Quanzhi Gaoshou - 03 [1080p][9FAFB01C].mkv, [Aira] Quanzhi Gaoshou - 04 [1080p][E737BA3F].mkv, [Aira] Quanzhi Gaoshou - 05 [1080p][85ACE5F3].mkv, [Aira] Quanzhi Gaoshou - 06 [1080p][B951BAF9].mkv, [Aira] Quanzhi Gaoshou - 07 [1080p][5863C616].mkv, [Aira] Quanzhi Gaoshou - 08 [1080p][4B014B0B].mkv, [Aira] Quanzhi Gaoshou - 09 [1080p][BEE542CB].mkv, [Aira] Quanzhi Gaoshou - 10 [1080p][5C3869D5].mkv, [Aira] Quanzhi Gaoshou - 11 [1080p][BC2C5707].mkv, [Aira] Quanzhi Gaoshou - 12 [1080p][0A818576].mkv]
Rename episodes using [AniDB]
Auto-detected query: [quanzhi gaoshou]
Failed to fetch episode data: [quanzhi gaoshou]
CmdlineException: Failed to match files to episode data
Finished without processing any files
Failure (°_°)
I tried setting the info.id to the AID value as well which results in the same.
Re: [AMC] Failed to fetch episode data
Posted: 22 Jun 2018, 21:28
by rednoah
Search for
Quanzhi Gaoshou indeed does not work:
Code: Select all
$ filebot -list --db AniDB --q "Quanzhi Gaoshou"
AniDB: no results
No idea, and I don't have time to debug now. Maybe it'll magically start working in the future for some reason.
You can use
filebot -rename and
--q "12560" in the meanwhile to process these files via the shell:
Code: Select all
$ filebot -list --db anidb --q "12560"
The King`s Avatar - 01 - Ye Qiu? Ye Xiu?
The King`s Avatar - 02 - The Skilled Supervisor
...
Re: [AMC] Failed to fetch episode data
Posted: 22 Jun 2018, 22:03
by Kuchiru
No idea, and I don't have time to debug now. Maybe it'll magically start working in the future for some reason.
Hah yeah, that wouldn't be the first time i've seen that happen.
Thanks for the workaround, worked like a charm.
Re: [AMC] Failed to fetch episode data
Posted: 09 Jul 2018, 18:08
by Kuchiru
Okay so, in addition to this, it seems to be happening elsewhere as well.
Show:
Phantom in the Twilight
Input:
Code: Select all
filebot -script fn:amc --output "$HOME/downloads" --log-file "$HOME/amc.log" --action hardlink --conflict override -non-strict --def "clean=y" "ut_dir=/home/sbuser/download/Anime/Phantom in the Twilight" "ut_kind=multi" "ut_title=Phantom in the Twilight" --filter "info.id=13978"
Output:
Code: Select all
Locking /home/sbuser/amc.log
Run script [fn:amc] at [Mon Jul 09 19:45:28 CEST 2018]
Parameter: clean = y
Parameter: ut_dir = /home/sbuser/download/Anime/Phantom in the Twilight
Parameter: ut_kind = multi
Parameter: ut_title = Phantom in the Twilight
Input: /home/sbuser/download/Anime/Phantom in the Twilight/[HorribleSubs] Phantom in the Twilight - 01 [1080p].mkv
xattr: [[HorribleSubs] Phantom in the Twilight - 01 [1080p].mkv] => [Phantom The Animation - 01 - Assassin]
Group: [anime:phantom the animation] => [[HorribleSubs] Phantom in the Twilight - 01 [1080p].mkv]
Rename episodes using [AniDB]
Auto-detected query: [Phantom The Animation]
Fetching episode data for [Phantom The Animation]
Fetching episode data for [Ray The Animation]
Fetching episode data for [Aria the Animation]
Fetching episode data for [Kara The Animation]
Fetching episode data for [Neon The Animation]
Apply filter [info.id=13978] on [52] items
No matching episode: [HorribleSubs] Phantom in the Twilight - 01 [1080p].mkv
CmdlineException: Failed to match files to episode data
Finished without processing any files
Failure (°_°)
Code: Select all
filebot --db AniDB --q 13978 -list
Phantom in the Twilight - 01 - Episode 1
Phantom in the Twilight - 02 - Episode 2
Phantom in the Twilight - 03 - Episode 3
Phantom in the Twilight - 04 - Episode 4
Doing this without AMC does work.
Show:
Planet With
Input:
Code: Select all
filebot -script fn:amc --output "$HOME/downloads" --log-file "$HOME/amc.log" --action hardlink --conflict override -non-strict --def "clean=y" "ut_dir=/home/sbuser/rtorrent/download/Anime/Planet With" "ut_kind=multi" "ut_title=Planet With" --filter "info.id=13965"
Output:
Code: Select all
Locking /home/sbuser/amc.log
Run script [fn:amc] at [Mon Jul 09 20:02:29 CEST 2018]
Parameter: clean = y
Parameter: ut_dir = /home/sbuser/download/Anime/Planet With
Parameter: ut_kind = multi
Parameter: ut_title = Planet With
Input: /home/sbuser/download/Anime/Planet With/[HorribleSubs] Planet With - 01v2 [1080p].mkv
Group: [anime:planet with] => [[HorribleSubs] Planet With - 01v2 [1080p].mkv]
Rename episodes using [AniDB]
Auto-detected query: [planet with]
Fetching episode data for [Planetes]
Apply filter [info.id=13965] on [39] items
No matching episode: [HorribleSubs] Planet With - 01v2 [1080p].mkv
CmdlineException: Failed to match files to episode data
Finished without processing any files
Failure (°_°)
Code: Select all
filebot --db AniDB --q 13965 -list
Planet With - 01 - Light, Seven Flashes
Planet With - 02 - Nebula Soldier
Planet With - 03 - Episode 3
Planet With - 04 - Episode 4
I can honestly say that this is the first time i've had so many "Failed to match files to episode data" since i started using the
option.
Not really sure whats going on.
Re: [AMC] Failed to fetch episode data
Posted: 09 Jul 2018, 18:39
by rednoah
You're using
= (assignment operator) instead of
== (comparison operator) so it doesn't work and can't ever work.
e.g.
Code: Select all
$ filebot -list --db anidb --q 13965 --format '{series.id} => {episode}' --filter 'series.id == 13965'
Apply filter [series.id == 13965] on [4] items
Include [Planet With - 01 - Light, Seven Flashes]
Include [Planet With - 02 - Nebula Soldier]
Include [Planet With - 03 - Episode 3]
Include [Planet With - 04 - Episode 4]
13965 => Planet With - 01 - Light, Seven Flashes
13965 => Planet With - 02 - Nebula Soldier
13965 => Planet With - 03 - Episode 3
13965 => Planet With - 04 - Episode 4
Re: [AMC] Failed to fetch episode data
Posted: 09 Jul 2018, 18:57
by Kuchiru
rednoah wrote: ↑09 Jul 2018, 18:39
You're using
= (assignment operator) instead of
== (comparison operator) so it doesn't work and can't ever work.
e.g.
Code: Select all
$ filebot -list --db anidb --q 13965 --format '{series.id} => {episode}' --filter 'series.id == 13965'
Apply filter [series.id == 13965] on [4] items
Include [Planet With - 01 - Light, Seven Flashes]
Include [Planet With - 02 - Nebula Soldier]
Include [Planet With - 03 - Episode 3]
Include [Planet With - 04 - Episode 4]
13965 => Planet With - 01 - Light, Seven Flashes
13965 => Planet With - 02 - Nebula Soldier
13965 => Planet With - 03 - Episode 3
13965 => Planet With - 04 - Episode 4
You're right, this is because i cleaned up my actual AMC command quite a bit because i didn't want to put unnecessary information here as it seemed a clear-cut issue, but, it appears i made an error in the process, below is my actual command, though i did try several times with info.id=123456 as i interpreted what it says below wrong.
This is what i use to process Anime with AMC
Code: Select all
filebot -script fn:amc --output "$HOME/downloads" --log-file "$HOME/amc.log" --action hardlink --conflict override -non-strict --def "ignore=(?i)Programs" "minLengthMS=0" "minFileSize=0" "clean=y" "ut_dir=$TORRENT_PATH" "ut_kind=multi" "ut_title=$TORRENT_NAME" "ut_label=$TORRENT_LABEL" --filter "label =~ info.id"
If i change it to series.id == 123456 nothing changes on my end, nor does info.id == 123456 and i made sure to clear my cache.
Re: [AMC] Failed to fetch episode data
Posted: 09 Jul 2018, 19:28
by rednoah
1.
What value does
label have? Are you passing in the exact numeric id as label?
2.
Back to the original problem, this filter excludes all items, and hence there can be no matching episodes, since the filter excludes each and every one of them:
Code: Select all
Apply filter [info.id=13965] on [39] items
No matching episode: [HorribleSubs] Planet With - 01v2 [1080p].mkv
If the same happens in your production filebot call, then we know that the
--filter return false for each and every episode object.
e.g.
--filter and
label fundamentally work:
Code: Select all
$ filebot -list --db anidb --q 13965 --filter 'label =~ series.id' --def ut_label=13965
Apply filter [label =~ series.id] on [4] items
Include [Planet With - 01 - Light, Seven Flashes]
Include [Planet With - 02 - Nebula Soldier]
Include [Planet With - 03 - Episode 3]
Include [Planet With - 04 - Episode 4]
Planet With - 01 - Light, Seven Flashes
Planet With - 02 - Nebula Soldier
Planet With - 03 - Episode 3
Planet With - 04 - Episode 4
Re: [AMC] Failed to fetch episode data
Posted: 09 Jul 2018, 19:44
by Kuchiru
rednoah wrote: ↑09 Jul 2018, 19:28
1.
What value does
label have? Are you passing in the exact numeric id as label?
2.
Back to the original problem, this filter excludes all items, and hence there can be no matching episodes, since the filter excludes each and every one of them:
Code: Select all
Apply filter [info.id=13965] on [39] items
No matching episode: [HorribleSubs] Planet With - 01v2 [1080p].mkv
If the same happens in your production filebot call, then we know that the
--filter return false for each and every episode object.
e.g.
--filter and
label fundamentally work:
Code: Select all
$ filebot -list --db anidb --q 13965 --filter 'label =~ series.id' --def ut_label=13965
Apply filter [label =~ series.id] on [4] items
Include [Planet With - 01 - Light, Seven Flashes]
Include [Planet With - 02 - Nebula Soldier]
Include [Planet With - 03 - Episode 3]
Include [Planet With - 04 - Episode 4]
Planet With - 01 - Light, Seven Flashes
Planet With - 02 - Nebula Soldier
Planet With - 03 - Episode 3
Planet With - 04 - Episode 4
1.
Labels are set like "Anime 123456" this currently still works in production at a very high successrate with only exception, currently, being the above.
2.
As i said, i made a mistake during troubleshooting here, production does not have this error.
Honestly, thank you for your time and i apologize for making a silly error, was certainly not intended.
Re: [AMC] Failed to fetch episode data
Posted: 09 Jul 2018, 20:28
by rednoah
1.
Yep, this will work.
2.
Never mind then. I thought you meant you encountered a similar looking issue in your production system.
Re: [AMC] Failed to fetch episode data
Posted: 10 Jul 2018, 10:19
by Kuchiru
rednoah wrote: ↑09 Jul 2018, 20:28
1.
Yep, this will work.
2.
Never mind then. I thought you meant you encountered a similar looking issue in your production system.
Well, it's still failing which is why i was troubleshooting to begin with.
Running filebot without the AMC script will work without issue here just like when you try.
With AMC however, it does not.
I matched the input perfectly this time, i don't really understand why AMC is still failing here.
Input:
Code: Select all
filebot -script fn:amc --output "$HOME/downloads" --log-file "$HOME/amc.log" --action hardlink --conflict override -non-strict --def "ut_dir=/home/sbuser/download/Anime/Planet With" "ut_kind=multi" "ut_title=Planet With" "ut_label=Anime 13965" --filter "label =~ series.id"
Output:
Code: Select all
Parameter: ut_dir = /home/sbuser/download/Anime/Planet With
Parameter: ut_kind = multi
Parameter: ut_title = Planet With
Parameter: ut_label = Anime 13965
Input: /home/sbuser/download/Anime/Planet With/[HorribleSubs] Planet With - 01v2 [1080p].mkv
Group: [anime:planet with] => [[HorribleSubs] Planet With - 01v2 [1080p].mkv]
Rename episodes using [AniDB]
Auto-detected query: [planet with]
Fetching episode data for [Planetes]
Apply filter [label =~ series.id] on [39] items
No matching episode: [HorribleSubs] Planet With - 01v2 [1080p].mkv
CmdlineException: Failed to match files to episode data
Finished without processing any files
Failure (°_°)
Re: [AMC] Failed to fetch episode data
Posted: 10 Jul 2018, 10:25
by rednoah
1.
Easily explained.
Planetes (a895) is the only option that FileBot has found:
Code: Select all
Fetching episode data for [Planetes]
Hence all 39 episodes of Planetes will be excluded since
/Anime 13965/ =~ /895/ yields
false:
Code: Select all
Apply filter [label =~ series.id] on [39] items
This means that the
--filter is working correctly in that it filters out all incorrect matches makes makes sure that you don't get any mismatches, which is usually what happens when you process a series that doesn't exist according to FileBot.
2.
Why does FileBot not know this brand new Anime exists? Easily explained. AniDB does not have a search API, so FileBot search is based on data dumps, which need to be updated constantly as new shows get added. If you're using an outdated version of FileBot, then it's most likely using outdated data dumps that don't include newer shows.
Have you tried the latest beta yet?
viewtopic.php?f=6&t=6006
3.
If you tag everything so meticulously, then you might want to consider not using the amc script at all, and just running your own
filebot -rename --q script. Ask yourself if there's any particular reason you're using the amc script, and not simple filebot -rename commands?
Re: [AMC] Failed to fetch episode data
Posted: 10 Jul 2018, 11:11
by Kuchiru
rednoah wrote: ↑10 Jul 2018, 10:25
1.
Easily explained.
Planetes (a895) is the only option that FileBot has found:
Code: Select all
Fetching episode data for [Planetes]
Hence all 39 episodes of Planetes will be excluded since
/Anime 13965/ =~ /895/ yields
false:
Code: Select all
Apply filter [label =~ series.id] on [39] items
This means that the
--filter is working correctly in that it filters out all incorrect matches makes makes sure that you don't get any mismatches, which is usually what happens when you process a series that doesn't exist according to FileBot.
2.
Why does FileBot not know this brand new Anime exists? Easily explained. AniDB does not have a search API, so FileBot search is based on data dumps, which need to be updated constantly as new shows get added. If you're using an outdated version of FileBot, then it's most likely using outdated data dumps that don't include newer shows.
Have you tried the latest beta yet?
viewtopic.php?f=6&t=6006
This makes a lot of sense, thanks for elaborating.
That means that the issue i had with the first Anime i made this topic for is unrelated to the other two, as they are new, my apologies for mixing them up.
rednoah wrote: ↑10 Jul 2018, 10:25
3.
If you tag everything so meticulously, then you might want to consider not using the amc script at all, and just running your own
filebot -rename --q script. Ask yourself if there's any particular reason you're using the amc script, and not simple filebot -rename commands?
I've never really thought about this, i set this up years ago together with you back when there was no publicly known way for rtorrent to communicate with filebot.
Though you've certainly got me thinking here, this does seem like a better option as i am very meticulous with the tagging, things are great when they work perfectly after all.
Beta + License works like a charm by the way.
Re: [AMC] Failed to fetch episode data
Posted: 10 Aug 2018, 09:11
by Kuchiru
rednoah wrote: ↑10 Jul 2018, 10:25
3.
If you tag everything so meticulously, then you might want to consider not using the amc script at all, and just running your own
filebot -rename --q script. Ask yourself if there's any particular reason you're using the amc script, and not simple filebot -rename commands?
To loop back on this, i've got this working.
However, i want the AMC naming scheme.
As per your examples this is
Code: Select all
/{primaryTitle}/{primaryTitle} - {sxe} - {t.replaceAll(/[!?.]+$/).replaceAll(/[`´‘’ʻ']/, /'/).replacePart(', Part $1')}
However, when i put that in like so:
Code: Select all
filebot --db AniDB --output "$HOME/filebot_test/processed" --log-file "$HOME/filebot_test/amc.log" --action hardlink --conflict override -non-strict -rename "$FULLPATH" --filter "series.id == $AID" --format "{primaryTitle}/{primaryTitle} - {sxe} - {t.replaceAll(/[!?.]+$/).replaceAll(/[`´‘’ʻ']/, /'/).replacePart(', Part $1')}"
Something goes wrong with this part
which ends up in a syntax error: unexpected end of file.
I've already done some experimenting with this, but i'm honestly not sure that when it does go through that it's working as expected and im no star with regular expression ..or with any of this.
I do wonder if it's needed outside of the AMC script?
I could just roll with this instead but i'm not entirely sure if i'll get away with that on the long term without issues due to the special characters you're filtering out with that bit of regex.
Code: Select all
{primaryTitle}/{primaryTitle} - {sxe} - {t}
Re: [AMC] Failed to fetch episode data
Posted: 10 Aug 2018, 10:21
by rednoah
1.
` has special meaning on the command-line:
viewtopic.php?f=4&t=1899
$1 doesn't mean what you think it means in the command-line context either.
2.
I recommend reading and understanding the problem by reading the command-line tutorial linked above. And then sidestepping the problem all together by passing complex argument values via text files instead of the command-line.
Use the
@file syntax for reading command-line arguments from external text files.
e.g. @/path/to/format.txt
Code: Select all
--format
{primaryTitle}/{primaryTitle} - {sxe} - {t.replaceAll(/[!?.]+$/).replaceAll(/[`´‘’ʻ']/, /'/).replacePart(', Part $1')}
Alternatively, avoiding special character and using '...' instead of "..."
(because $ expansion) will work too:
Code: Select all
--format '{primaryTitle}/{primaryTitle} - {sxe} - {t.asciiQuotes().replaceAll(/[!?.]+$/).replacePart(/, Part $1/)}'
Re: [AMC] Failed to fetch episode data
Posted: 10 Aug 2018, 11:13
by Kuchiru
rednoah wrote: ↑10 Aug 2018, 10:21
1.
` has special meaning on the command-line:
viewtopic.php?f=4&t=1899
$1 doesn't mean what you think it means in the command-line context either.
2.
I recommend reading and understanding the problem by reading the command-line tutorial linked above. And then sidestepping the problem all together by passing complex argument values via text files instead of the command-line.
Use the
@file syntax for reading command-line arguments from external text files.
e.g. @/path/to/format.txt
Code: Select all
--format
{primaryTitle}/{primaryTitle} - {sxe} - {t.replaceAll(/[!?.]+$/).replaceAll(/[`´‘’ʻ']/, /'/).replacePart(', Part $1')}
Alternatively, avoiding special character and using '...' instead of "..."
(because $ expansion) will work too:
Code: Select all
--format '{primaryTitle}/{primaryTitle} - {sxe} - {t.asciiQuotes().replaceAll(/[!?.]+$/).replacePart(/, Part $1/)}'
Part of the issue here is me blindly assuming that what was put here (Automated Media Center Default Formats)
viewtopic.php?f=5&t=2 would just 'work'.
I do know what $1 means in command line context, however, part of that also requires me to understand what it is that you're actually doing here, which i don't.
So your advice on reading and understanding the problem is the main issue for me, as i don't.
I'll spend some more time to try and figure this out, thank you for the pointers, i appreciate it
!
Re: [AMC] Failed to fetch episode data
Posted: 10 Aug 2018, 11:57
by rednoah
If you're using the GUI, or if you're using @files, then you can paste formats verbatim. But if you use another language, like bash, then you'll need to escape the format so bash doesn't interpret bits and pieces, before passing the result to FileBot.
In the context of
t.replacePart(/, Part $1/),
$1 is a regex replacement pattern referring to the first match.
However, if you write
--format "t.replacePart(/, Part $1/)" then bash will interpret
$1 first, which is probably empty, and then the argument value effectively passed on to FileBot will be
t.replacePart(/, Part /) and that will just replace the part number with ", Part " instead of actually adding the part number as well.
So while you don't need to understand formats per se, understanding command-line argument parsing and how to pass complex arguments verbatim is vital for using the command-line.