Having issue with older episodes of still-running series

[mihawk90]

Hello there,

I've been trying to get AMC to work for a while now but for lack of time I've been just logging the results for a few months now. Today I looked through the log a little and I found this little issue:

Code: Select all

Run script [fn:amc] at [Tue Nov 29 01:12:42 CET 2016]
Parameter: music = n
Parameter: artwork = n
Parameter: ut_label = tv
Parameter: ut_state = 5
Parameter: ut_title = [yibis]_One_Piece_762_[720p][F59500BA].mkv
Parameter: ut_kind = single
Parameter: ut_file = [yibis]_One_Piece_762_[720p][F59500BA].mkv
Parameter: ut_dir = C:\Volumes\NAS 1-2 1TB\Videos\Anime\Subbed\One Piece
Parameter: seriesFormat = {es.collect{ s.pad(2)+'x'+e.pad(2)}.join('-')} [{group}] {episodes*.absolute*.pad(3).join('-')} [{crc32}]
Input: C:\Volumes\NAS 1-2 1TB\Videos\Anime\subbed\One Piece\[yibis]_One_Piece_762_[720p][F59500BA].mkv
Group: [tvs:one piece] => [[yibis]_One_Piece_762_[720p][F59500BA].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [One Piece]
Fetching episode data for [One Piece]
Fetching episode data for [None Piece]
Apply Filter: {age < 30 || !model.any{ it.age < 30 }}
Include [One Piece - 18x17 - The Truth Behind His Disappearance - Sanji's Shocking Invitation]
Include [One Piece - 18x18 - To My Buds! Sanji's Farewell Note!]
Include [One Piece - 18x19 - Let's Go See Master Nekomamushi!]
Include [One Piece - 18x20 - Luffy's Decision - The Sanji Abdication Crisis!]
[TEST] Rename [C:\Volumes\NAS 1-2 1TB\Videos\Anime\subbed\One Piece\[yibis]_One_Piece_762_[720p][F59500BA].mkv] to [C:\Users\tarulia\Downloads\BitTorrent\_organized\18x18 [Yibis] 764 [F59500BA].mkv]
Processed 1 files
Done ヾ(＠⌒ー⌒＠)ノ

Apart from the done emote looking a bit off you'll notice it's trying to rename Episode 762 to 764. Now I see that 762 is not fetched because it would be 18x16 and I could raise the age limit, but I can't change the age limit for every single download, right? Well, I could, but that's not the goal, is it

I'm wondering why it matches 764 of all the available episodes, I would understand if it would use the oldest or newest episode, but why that... Well, not the point... how can I get it to not rename at all if it doesn't match the absolute number 100%?

In general this filter is a little problematic when downloading older seasons of currently running series, is there anything I could do about that? :/ I could remove the filter alltogether but fetching all episodes everytime is a little problematic as I understand right?

[rednoah]

If you use a filter, then FileBot then you limit FileBot to these options. If the correct option isn't one of these options in the first place, the FileBot is set up to fail miserably.

Not being able to process older episodes of running shows is the price you pay for perfect accuracy (within the limited contract of the filter). You could not use the filter, or come up with your own filter that somehow magically doesn't have any cons.

PS: FileBot does need to fetch the complete episode list so that i can then apply the filter. You won't put any more or less load on any database by using a filter.

[mihawk90]

OK, seems I misunderstood something somewhere then Will just remove the filter for now then and see how it goes.

Theoretically though, as I understand the filter takes any arbitrary code, correct? So it should be possible to process the label for example and apply different filters depending on the label? Say for example I could use a label called tv_old or similar and drop the filter for just that? Unfortunately I'm not really familiar with the syntax, but reading a little into groovy this should work, right?

Code: Select all

--filter "F:{(ut_label != 'tv_old') ? (age < 7 || !model.any{ it.age < 7 })}"

That "should" return nothing if the statement is true and else return the filter string I think? Apparently I have to use the F: or else I get a weird error that I have no idea what it means, but that's what google suggests

Unfortunately that fails for me too. It takes forever to even fetch the episode data, and then it just dumps me the whole XML into the console and log file...
/edit Doesn't even work with my old command now is TVDB having issues? :/

Code: Select all

[2016-12-08 03:58:56]  Executing: filebot -script fn:amc --output "C:\Users\tarulia\Downloads\BitTorrent\_organized" --log-file "C:\Users\tarulia\Downloads\BitTorrent\_organized\amc.log" --action test --conflict override -non-strict --filter "age < 30 || !model.any{ it.age < 30 }" --def music=n artwork=n "ut_label=tv" "ut_state=11" "ut_title=18x16 [yibis] 762 [F59500BA].mkv" "ut_kind=single" "ut_file=18x16 [yibis] 762 [F59500BA].mkv" "ut_dir=C:\Volumes\NAS 1-2 1TB\Videos\Anime\Subbed\One Piece" "seriesFormat={es.collect{ s.pad(2)+'x'+e.pad(2)}.join('-')} [{group}] {episodes*.absolute*.pad(3).join('-')} [{crc32}]"

Results in

Code: Select all

Run script [fn:amc] at [Thu Dec 08 03:59:00 CET 2016]
Parameter: music = n
Parameter: artwork = n
Parameter: ut_label = tv
Parameter: ut_state = 11
Parameter: ut_title = 18x16 [yibis] 762 [F59500BA].mkv
Parameter: ut_kind = single
Parameter: ut_file = 18x16 [yibis] 762 [F59500BA].mkv
Parameter: ut_dir = C:\Volumes\NAS 1-2 1TB\Videos\Anime\Subbed\One Piece
Parameter: seriesFormat = {es.collect{ s.pad(2)+'x'+e.pad(2)}.join('-')} [{group}] {episodes*.absolute*.pad(3).join('-')} [{crc32}]
Input: C:\Volumes\NAS 1-2 1TB\Videos\Anime\subbed\One Piece\18x16 [yibis] 762 [F59500BA].mkv
Group: [tvs:one piece] => [18x16 [yibis] 762 [F59500BA].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [One Piece]
Fetching episode data for [One Piece]
<?xml version="1.0" encoding="UTF-8" ?>


//Data here, it just breaks up at some point


org.xml.sax.SAXParseException; lineNumber: 55; columnNumber: 13; XML-Dokumentstrukturen müssen innerhalb derselben Entity beginnen und enden.
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.endEntity(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.endEntity(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.endEntity(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipSpaces(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.seekCloseOfStartTag(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
	at net.filebot.web.WebRequest.validateXml(WebRequest.java:324)
	at net.filebot.CachedResource.lambda$validateXml$5(CachedResource.java:169)
	at net.filebot.CachedResource.lambda$get$1(CachedResource.java:99)
	at net.filebot.Cache.computeIf(Cache.java:90)
	at net.filebot.CachedResource.get(CachedResource.java:82)
	at net.filebot.web.TheTVDBClientV1.getXmlResource(TheTVDBClientV1.java:272)
	at net.filebot.web.TheTVDBClientV1.fetchSeriesData(TheTVDBClientV1.java:119)
	at net.filebot.web.AbstractEpisodeListProvider.lambda$getSeriesData$1(AbstractEpisodeListProvider.java:60)
	at net.filebot.Cache.computeIf(Cache.java:90)
	at net.filebot.Cache$TypedCache.computeIf(Cache.java:176)
	at net.filebot.Cache.computeIfAbsent(Cache.java:96)
	at net.filebot.Cache$TypedCache.computeIfAbsent(Cache.java:181)
	at net.filebot.web.AbstractEpisodeListProvider.getSeriesData(AbstractEpisodeListProvider.java:59)
	at net.filebot.web.AbstractEpisodeListProvider.getEpisodeList(AbstractEpisodeListProvider.java:36)
	at net.filebot.cli.CmdlineOperations.fetchEpisodeSet(CmdlineOperations.java:299)
	at net.filebot.cli.CmdlineOperations.renameSeries(CmdlineOperations.java:207)
	at net.filebot.cli.CmdlineOperations.rename(CmdlineOperations.java:110)
	at net.filebot.cli.ScriptShellBaseClass.rename(ScriptShellBaseClass.java:332)
	at Script1$_run_closure55.doCall(Script1.groovy:418)
	at Script1.run(Script1.groovy:404)
	at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:62)
	at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:72)
	at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:113)
	at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:28)
	at net.filebot.Main.main(Main.java:122)

Fetch failed: org.xml.sax.SAXParseException; lineNumber: 55; columnNumber: 13; XML-Dokumentstrukturen müssen innerhalb derselben Entity beginnen und enden. [http://thetvdb.com/api/694FAD89942D3827/series/81797/all/en.xml]
XML-Dokumentstrukturen müssen innerhalb derselben Entity beginnen und enden.
Finished without processing any files
Failure (°_°)

What have I done :/ Have I even done this

[rednoah]

1.
The -list command makes testing formats and filters easy:

Code: Select all

filebot -list --q Firefly --filter "e < 5"

Code: Select all

filebot -list --q Firefly --filter "label == /test/" --format "{label}; {json}" --def ut_label=test

2.
TheTVDB is known to send back malformed or invalid XML:

Code: Select all

org.xml.sax.SAXParseException; lineNumber: 55; columnNumber: 13

They usually fix this after a few days, and the document seems valid now:
http://thetvdb.com/api/694FAD89942D3827 ... all/en.xml

If it's not fixed after a while, then you can report the issue in the TheTVDB forums.

[mihawk90]

Sigh... an API shouldn't be returning invalid data but.. it is what it is...

Didn't know about the list command, filebot has so many tricks and they are sometimes hard to find

So, now I could finally play around a little more. After groovy gave me some syntax errors and I fixed them I still can't get any data out however

Code: Select all

PS C:\Users\tarulia> filebot -list --q 'One Piece' --filter "F:{(ut_label != 'tv_old') ? 'age < 7 || !model.any{ it.age < 7 }':''}" --def ut_label=tv_old
Apply Filter: {F:{(ut_label != 'tv_old') ? 'age < 7 || !model.any{ it.age < 7 }':''}}

I also tried a longer version without an "else" expressoin like this (since using the shorthand without the "else" seems to always give me a syntax error)

Code: Select all

filebot -list --q 'One Piece' --def ut_label=tv_old --filter "F:{if (ut_label != 'tv_old') then {'age < 7 || !model.any{ it.age < 7 }'}}"

Unfortunatly the result is the same, and it doesn't tell me the actual filter either :/ It just gives me everything in the filter string, so I don't even know if that condition is even run

Code: Select all

PS C:\Users\tarulia> filebot -list --q 'One Piece' --def ut_label=tv_old --filter "F:{if (ut_label != 'tv_old') then {'age < 7 || !model.any{ it.age < 7 }'}}"
Apply Filter: {F:{if (ut_label != 'tv_old') then {'age < 7 || !model.any{ it.age < 7 }'}}}

Am I even correct in assuming that whatever string I put in the "then" branch is returned to the filter parameter? Or do I need to do something else to actually return it?

[rednoah]

The --filter parameter expects a Groovy expression. The Groovy Truth of the result of that expression decides if an episode gets included or excluded.

Compare & Contrast:

Code: Select all

filebot -list --q Firefly --filter "if (label == /all/) return true; e < 5" --def ut_label=all

Code: Select all

filebot -list --q Firefly --filter "if (label == /all/) return true; e < 5" --def ut_label=other

[mihawk90]

OK... well... seems I have completely misunderstood the functionality then I thought whatever the condition returned would then be used as the actual filter

So just for my understanding of those Snippets...
If the provided label is the one we're looking for the condition returns true, thus including the episode and not returning any filter expression at all (Snippet 1). But if it isn't what we're looking for the condition is false, letting the rest of the filter code run, thus applying the filter string (Snippet 2)?

I should probably have used a shorter series for testing this, takes ages to process One Piece with the filter applied, but it works! In my case this is the (currently) final query:

Code: Select all

filebot -list --q 'One Piece' --def ut_label=tv_old --filter "if (label == /tv_old/) return true; age < 30 || !model.any{ it.age < 30 }"

Works really well, as long as I remember setting the label

Thanks for the awesome support again!

[rednoah]

return just returns the value so anything after the return is never executed.

You never return an expression, but every expression has a value, such as (e < 5) can have the values true or false. You could write return e < 5 but the return keyword is not necessary in this case since it's the last statement.