FileBot

Posted: **02 Oct 2020, 12:00**

I've recently discovered when using AniDB.getSeriesInfo in a groovy script, that the aliasNames field is frequently unreliable.
On the primary system I have been developing my groovy script on, I've recently been able to pinpoint some irregularities in my match results to INVALID (as in Aliases for DIFFERENT AniDB series) aliasNames entries. None of those invalid aliasNames can be seen in the AniDB web ui or anime-titles.xml.

As AniDB has an "I ban you if you look at me funny" API, I would like to understand if there is some kind of hinting system, or alternative data sources that Filebot uses to expand on the aliasNames field when using the AniDB.getSeriesInfo method that perhaps is getting confused, or maybe building 'hints" that work for TVDB, but not so much for AniDB matching ..

Or if perhaps, using that specific invocation/method is depreciated and I should be doing it a different way.
For example:
I recently (last day or so) identified two AniDB series where the aliasNames contains references to DIFFERENT series, there may be more, but I stopped looking and started writing this post immediately after noticing it wasn't just an isolated case.

14488 has Enen no Shouboutai S2, Enen no Shouboutai: Ni no Shou, both of which are the 2nd Series with the ID #15335
9310 .. ugh. It has the aliasNames for the 2nd AND 3rd Series in it (Yahari Ore no Seishun Love Come wa Machigatteiru Zoku, Yahari Ore no Seishun Love Comedy wa Machigatteiru Zoku, My Teen Romantic Comedy SNAFU Zoku, Yahari Ore no Seishun Love Comedy wa Machigatteiru. Zoku - My Teenage RomCom SNAFU TOO!, Yahari Ore no Seishun Love Come wa Machigatteiru Kan, Yahari Ore no Seishun Love Comedy wa Machigatteiru Kan, My Teen Romantic Comedy SNAFU Climax, Yahari Ore no Seishun Love Comedy wa Machigatteiru. Kan û My Teen Romantic Comedy SNAFU Climax, Yahari Ore no Seishun LoveCome wa Machigatte Iru. Kan).

Code: Select all

//--------------------------------//
Search by ID: 14488
Get Series Info: 14488
Results: [runtime:null, startDate:2019-07-06, genres:[], certification:null, rating:6.15, id:14488, name:En`en no Shouboutai, network:null, ratingCount:null, type:Anime, class:class net.filebot.web.SeriesInfo, spokenLanguages:[], order:Absolute, status:null, language:en, aliasNames:[Fire Force, ??????, Enn Enn no Shouboutai, enen, Enen no Shouboutai S2, Enen no Shouboutai: Ni no Shou], database:AniDB]
//--------------------------------//
Search by ID: 9310
Get Series Info: 9310
Results: [runtime:null, startDate:2013-04-05, genres:[], certification:null, rating:7.1, id:9310, name:Yahari Ore no Seishun LoveCome wa Machigatte Iru., network:null, ratingCount:null, type:Anime, class:class net.filebot.web.SeriesInfo, spokenLanguages:[], order:Absolute, status:null, language:en, aliasNames:[My Teen Romantic Comedy: SNAFU, ???????????????????., Yahari Ore no Seishun Love Comedy wa Machigatteiru, My Teenage RomCom SNAFU, My Youth Romantic Comedy Is Wrong As I Expected., Oregairu, Yahari Ore no Seishun Love Come wa Machigatteiru., Yahari Ore no Seishun Love Comedy wa Machigatteiru - My Teenage RomCom SNAFU, Yahari Ore no Seishun Love Come wa Machigatteiru Zoku, Yahari Ore no Seishun Love Comedy wa Machigatteiru Zoku, My Teen Romantic Comedy SNAFU Zoku, Yahari Ore no Seishun Love Comedy wa Machigatteiru. Zoku - My Teenage RomCom SNAFU TOO!, Yahari Ore no Seishun Love Come wa Machigatteiru Kan, Yahari Ore no Seishun Love Comedy wa Machigatteiru Kan, My Teen Romantic Comedy SNAFU Climax, Yahari Ore no Seishun Love Comedy wa Machigatteiru. Kan û My Teen Romantic Comedy SNAFU Climax, Yahari Ore no Seishun LoveCome wa Machigatte Iru. Kan], database:AniDB]

the Groovy behind this is fairly simple

Code: Select all

def locale = any { _args.language.locale } { Locale.ENGLISH }
println "//--------------------------------//"
println "Search by ID: 14488"
myQueryAniDB = 14488
println "Get Series Info: ${myQueryAniDB}"
myAniDBseriesInfo = AniDB.getSeriesInfo(myQueryAniDB, locale)
println "Results: ${myAniDBseriesInfo.properties}"
println "//--------------------------------//"
println "Search by ID: 9310"
myQueryAniDB = 9310
println "Get Series Info: ${myQueryAniDB}"
myAniDBseriesInfo = AniDB.getSeriesInfo(myQueryAniDB, locale)
println "Results: ${myAniDBseriesInfo.properties}"

I noticed awhile ago that the aliasNames field would sometimes not have an alias that I can see in the AniDB WebUI and anime-titles.xml, however it wasn't until recently I started looking at the complete list of aliasNames and their values to see both how many are returned, but also if they are for that Series.

This is when I noticed that A) The number of aliasNames return is frequently more then what I can see in the AniDB WebUI/anime-titles.xml and B) I am identifying series with aliasNames that are flat out wrong.

The aliasNames that are flat out wrong are a problem for me, and concerning as I just can't figure out HOW.

Primary system I test with:

Code: Select all

PS C:\Users\vitki\Desktop\scripts> filebot -script fn:sysinfo
FileBot 4.9.2 (r8004)
JNA Native: 6.1.0
MediaInfo: 20.08
7-Zip-JBinding: 16.02
Chromaprint: 1.5.0
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-09-10 (r673)
Groovy: 3.0.5
JRE: OpenJDK Runtime Environment 14.0.2
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 2.1 GB Max Memory / 37 MB Used Memory
OS: Windows Server 2019 (amd64)
STORAGE: NTFS [(C:)] @ 85 GB | NTFS [music] @ 19 TB | NTFS [pictures] @ 19 TB | NTFS [video] @ 19 TB | NTFS [multimedia] @ 19 TB | NTFS [animebt] @ 17 TB
DATA: C:\Users\vitki\Desktop\Scripts\FileBot_4.9.2-portable-r8004\data
Package: ZIP

Secondary System I test with:

Code: Select all

PS C:\Users\vitki\OneDrive\GitHub\runecalico\filebot\developing> filebot -script fn:sysinfo
FileBot 4.9.1 (r7372)
JNA Native: 6.1.0
MediaInfo: 19.09
7-Zip-JBinding: 9.20
Chromaprint: 1.4.3
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-09-10 (r673)
Groovy: 3.0.3
JRE: OpenJDK Runtime Environment 14
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 24 Core / 17 GB Max Memory / 133 MB Used Memory
OS: Windows 10 (amd64)
STORAGE: NTFS [SYS] @ 337 GB | NTFS [SYSGAMES2] @ 671 GB | NTFS [SYSGAMES1] @ 526 GB | NTFS [SysSlow] @ 960 GB | NTFS [Backup01] @ 2.3 TB | NTFS [HandBrake1] @ 2.5 TB | NTFS [DATA01] @ 7 TB | NTFS [music] @ 19 TB | NTFS [animebt] @ 19 TB | NTFS [backups] @ 19 TB | NTFS [dataroot] @ 19 TB | NTFS [pictures] @ 19 TB | NTFS [animebt] @ 17 TB | NTFS [video] @ 19 TB | NTFS [vitki] @ 19 TB | NTFS [multimedia] @ 19 TB
DATA: C:\Users\vitki\AppData\Roaming\FileBot
Package: MSI

This is further complicated as on the Secondary System, filebot r7372 will return the same aliasNames for 14488, but NO aliasNames for 9310 (even if I run it multiple times)
and filebot r8004 on the Secondary System doesn't return any aliasNames for 14488 or 9310.

I am fairly confused by the variability of results between systems, as well as how there are wrong aliasNames for a series.

Posted: **02 Oct 2020, 13:23**

The index is compiled from various sources. I'll look into it. Getting different results on different machines is strange though. Upgrade everything to the latest and greatest, clear cache, double-check results.

Posted: **02 Oct 2020, 16:15**

Thanks for looking into it.

Posted: **04 Oct 2020, 13:37**

hm... perhaps there is something behind the scenes wrong? perhaps a windows update/defender that is causing issues?
I downloaded r8046, and tried my groovy script to get the series info and it does find the name of the series, but no aliases?

I just don't get it.

Code: Select all

//--------------------------------//
Search by ID: 14488
Get Series Info: 14488
Results: [runtime:null, startDate:2019-07-06, genres:[], certification:null, rating:6.16, id:14488, name:En`en no Shouboutai, network:null, ratingCount:null, type:Anime, class:class net.filebot.web.SeriesInfo, spokenLanguages:[], order:Absolute, status:null, language:en, aliasNames:[], database:AniDB]
//--------------------------------//
Search by ID: 9310
Get Series Info: 9310
Results: [runtime:null, startDate:2013-04-05, genres:[], certification:null, rating:7.1, id:9310, name:Yahari Ore no Seishun LoveCome wa Machigatte Iru., network:null, ratingCount:null, type:Anime, class:class net.filebot.web.SeriesInfo, spokenLanguages:[], order:Absolute, status:null, language:en, aliasNames:[], database:AniDB]

I tried enabling debug logging ... all it did was tell me it's using disk cache, but the results remain as above ..

Code: Select all

[2020-10-04 08:34:08.462] Using persistent disk cache: C:\Users\xxxxx\Desktop\Scripts\FileBot_4.9.2-portable-r8046\data\cache\0

Code: Select all

filebot -script fn:properties --def net.filebot.logging.debug=ALL
filebot -script fn:properties --def net.filebot.logging.time=true

I also manually added it to the Windows Firewall (no change), and added the entire filebot beta directory to the Defender exclusions (no change).

Just to verify, what is the method you suggest to lookup AniDB seriesinfo in a groovy script? am I doing something wrong?

Posted: **04 Oct 2020, 15:45**

rv23 wrote: ↑04 Oct 2020, 13:37 Just to verify, what is the method you suggest to lookup AniDB seriesinfo in a groovy script? am I doing something wrong?

Nobody is doing what you're doing, so it's pretty much uncharted territory.

Here's what I get:

Code: Select all

def i = AniDB.getSeriesInfo(14488, Locale.ENGLISH)
println i.toJsonString()

Code: Select all

{"@type":"net.filebot.web.SeriesInfo","database":"AniDB","order":"Absolute","language":"en","type":"Anime","id":14488,"name":"En`en no Shouboutai","aliasNames":[],"startDate":{"year":2019,"month":7,"day":6},"genres":[],"rating":6.16}

Looks like aliasNames just isn't populated if you do lookup-by-id, but may be populated if you do lookup-by-name. However, since we do cache results by id for a day, whatever you do first sticks for a day, and so any subsequent lookup of any kind will get a copy of whatever the first result was for that id.

The filebot command unexpectedly exhibits highly stateful behaviors for your particular purpose.

Posted: **06 Oct 2020, 12:25**

Does AniDB not return that information if you query by ID then? Would this also explain the invalid aliases returned (somehow incorrect cached results)? or is it just a behavior of the current filebot query code if you query by ID?

<sigh>

I switched to lookups by ID as lookups by name *usually* worked, but for some names wouldn't return any results despite the fact that I was using the actual series name and knew it existed in AniDB (because I was using the title from the AniDB Titles xml file, so it's 100% accurate as of the date/time the xml was generated by AniDB).

The "some names do not return a search result" would also then spill over into when I was renaming using a query.

Switching to ID looked to help with that behavior, but then the aliases didn't show up.
I have since switched to looking up the aliases in the Title XML, which bypasses the issue with aliases and lookup by ID. It also has the bonus of probably reducing the calls to the AniDB http API

Thanks for looking into it.

FileBot

Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?

Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?

Re: Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?

Re: Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?

Re: Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?

Re: Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?

Re: Groovy Script: Anidb SeriesInfo Aliases Bug or Feature?