db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Any questions? Need some help?
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

I'm using the official AMC script, and while I'm using theTVDB for lookups, I'd like the anidb ID (when in the mapper) to be part of the directory name.

However, I've noticed that when I often do not have an db.anidb.id binding, that animeList.anidb.id binding seems to generally be there, the number it returns is not from AniDB, Nor TheTVDB.

For some of these shows, I have verified that the ScudLee xml (anime-list.xml) has these entries, and has had them for over 20+ days.

I'm a bit baffled on not only why there is no db.anidb.id (when it should), but even if there wasn't why animelist.anidb.id is a seemingly nonsensical number ..

These are in ScudLee's xml (and have been for over 20+ days)

Code: Select all

Rename episodes using [TheTVDB] with [Absolute Order]
Auto-detected query: [The God of High School, god of]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Fetching episode data for [God of]
Fetching episode data for [Soldier]
Fetching episode data for [The God of Risk]
Fetching episode data for [The God of Music]
Fetching episode data for [Master of Study]
Apply mapper [AnimeList.AniDB] on [133] episodes
[TEST] from [W:\FB-1\anime\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556] [animelist-7827572]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
Rename episodes using [TheTVDB] with [Absolute Order]
Auto-detected query: [Super Dragon Ball Heroes]
Fetching episode data for [Super Dragon Ball Heroes]
Apply mapper [AnimeList.AniDB] on [26] episodes
[TEST] from [W:\FB-1\anime\Super Dragon Ball Heroes [tvdb-347734]\[Desu is dead] Super Dragon Ball Heroes Big Bang Mission Episode 5 (SDBH Episode 25) [720p].mkv] to [Z:\1-InitialSort\2020\summer\Super Dragon Ball Heroes [tvdb-347734] [animelist-7819927]\[Desu is dead] Super Dragon Ball Heroes Big Bang Mission Episode 5 (SDBH Episode 25) [720p].mkv]
Here is the full format I'm using (I added code to show both tvdb/animelist or anidb/tvdb id's)

Code: Select all

  {
    if ("${self.media.Encoded_Date}" == "null" ) {
      if ("${self.media.File_Created_Date_Local}" == "null" ) {
        if ("${self.media.File_Modified_Date_Local}" == "null" ) {
          myReleaseYear = "${today}".split('-')[0]
        } else {
          myReleaseYear = "${self.media.File_Modified_Date_Local('-')[0]}"
        }
      } else {
        myReleaseYear = "${self.media.File_Created_Date_Local.split('-')[0]}"       
      }
    } else {
      myReleaseYear = "${self.media.Encoded_Date.replaceFirst(/UTC\s/, '').split('-')[0]}"
    }
    // This check is for when the binding is defined, but the UTC date data is screwed up.
    // Failed to read media characteristics: DateTimeParseException: Text 'UTC ' could not be parsed at index 4
    // Which gives UTC  as the entire binding value (useless)
    if ( myReleaseYear == "" ) {
      myReleaseYear = "${today}".split('-')[0]
    }
    // myShowYear/myShowMonth seems to frequently not populate
    myShowYear = any {"${airdate.year}"} {"${d.year}"} {"${startdate.year}"} {"${today}".split('-')[0]} {"Year"}
    myShowMonth = any {"${airdate.month}"} {"${d.month}"} {"${startdate.month}"} {"${today}".split('-')[1]} {"Month"}
    myShowSeason = "${switch (myShowMonth) {
      case {myShowMonth.matches("0?[1-3]")}:
          return 'winter'
      case {myShowMonth.matches("0?[4-6]")}:
          return 'spring'
      case {myShowMonth.matches("0?[7-9]")}:
          return 'summer'
      case {myShowMonth.matches("10|11|12")}:
          return 'fall'
      default:
          return 'unknown'
      }
    }"
    // Sometimes the metadata for media.Encoded_date is bogus. Or if you are HorribleSubs, seems to be stuck at 2010
    // So if myShowYear is less then or Equal to myReleaseYear, then assume the encoded date field is bogus.
    any
      {
        if ( "${myShowYear}" >=  "${myReleaseYear}" ) {
          myShowYear+"/"+myShowSeason
        } else {
          myReleaseYear+"_rerelease"
        }
      }
      {
        myReleaseYear+"_creation"
      }
      { "Failure"}
  }/
    { any { db.AniDB.n } { db.AniDB.primaryTitle } { db.TheTVDB.n } {n} }
    { any 
      { if (db.AniDB.id) '[anidb-'+{db.AniDB.id}+']' } 
      {'[tvdb-'+db.TheTVDB.id+']'} 
      {'[tmdb-'+tmdbid+']'} 
    }
    { any 
      { if (Animelist.AniDB.id) '[animelist-'+{Animelist.AniDB.id}+']' } 
      {'[tvdb-'+db.TheTVDB.id+']'} 
      {'[tmdb-'+tmdbid+']'} 
    }
/{fn}
And the CLI command I'm using ..

Code: Select all

$ACTION = "test"
$FileBotSourceDirectory = "W:\FB-1\anime\"
$FileBotDestDirectory = "Z:\1-InitialSort\"
$MYScriptPath = (Get-Location).Path
$MyScript="fn:amc"
$MyFormat="$MYScriptPath\initialsort_testing.groovy"
  filebot -script $MyScript --action $ACTION "$($FileBotSourceDirectory)" --output "$($FileBotDestDirectory)" `
    --def movieFormat=@$($MyFormat)  `
    animeFormat=@$($MyFormat)  `
    seriesFormat=@$($MyFormat)  `
    animeDB=TheTVDB `
    minFileSize=10 `
    minLengthMS=5 `
    -no-xattr `
    -non-strict `
    --mapper AnimeList.AniDB `
    -r `
    --lang=en `
    --log all 
and output from fn:sysinfo

Code: Select all

FileBot 4.9.1 (r7372)
JNA Native: 6.1.0
MediaInfo: 19.09
7-Zip-JBinding: 9.20
Chromaprint: 1.4.3
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-08-02 (r665)
Groovy: 3.0.3
JRE: OpenJDK Runtime Environment 14
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 24 Core / 17 GB Max Memory / 129 MB Used Memory
OS: Windows 10 (amd64)
DATA: C:\Users\xxxxx\AppData\Roaming\FileBot
Package: MSI
License: FileBot License PX10195720 (Valid-Until: 2069-11-10)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

Here's what I get:

Code: Select all

filebot -list --q "The God of High School" --db TheTVDB --format "{id} | {db.AniDB.id} | {episode}"
378556 | 15408 | The God of High School - 1x01 - set up/stand up
378556 | 15408 | The God of High School - 1x02 - renewal/soul
378556 | 15408 | The God of High School - 1x03 - wisdom/kingdom
...
:arrow: https://www.thetvdb.com/dereferrer/series/378556
:arrow: https://anidb.net/anime/15408
:idea: Please read the FAQ and How to Request Help.
kim
Power User
Posts: 1251
Joined: 15 May 2014, 16:17

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by kim »

Code: Select all

<anime anidbid="14086" tvdbid="347734" defaulttvdbseason="1"><name>Super Dragon Ball Heroes</name><tvdbname>Super Dragon Ball Heroes</tvdbname></anime>
I think it's something with the season
season 1 = OK
season 3 =
Mapping not found
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Re: Seasons
hmm.. Seasons..
I didn't consider that perhaps the mapping has the series, but may not include the tvdb season, which invalidates the mapping.

Okay, that's reasonable (annoying, but reasonable), tho that still doesn't explain why I can't seem to get db.anidb.id from The God of High School ..

Code: Select all

  
  <anime anidbid="15408" tvdbid="378556" defaulttvdbseason="1">
    <name>The God of High School</name>
  </anime>


Re: filebot query
hmm.. Perhaps there is an underlying issue that is being masked when running amc (vs non-script filebot query).. I don't see this error when I run amc ...

Code: Select all

 filebot -list --q "The God of High School" --db TheTVDB --format "{id} | {db.AniDB.id} | {episode}"
java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError
        at net.filebot.CachedResource.lambda$fetchIfModified$10(Unknown Source)
        at net.filebot.CachedResource.lambda$withPermit$18(Unknown Source)
        at net.filebot.CachedResource.lambda$get$1(Unknown Source)
        at net.filebot.CachedResource.retry(Unknown Source)
        at net.filebot.CachedResource.lambda$get$2(Unknown Source)
        at net.filebot.Cache.computeIf(Unknown Source)
        at net.filebot.CachedResource.get(Unknown Source)
        at net.filebot.web.AnidbClient.getXmlResource(Unknown Source)
        at net.filebot.web.AnidbClient.fetchSeriesData(Unknown Source)
        at net.filebot.web.AbstractEpisodeListProvider.lambda$getSeriesData$1(Unknown Source)
        at net.filebot.Cache.computeIf(Unknown Source)
        at net.filebot.Cache$TypedCache.computeIf(Unknown Source)
        at net.filebot.Cache.computeIfAbsent(Unknown Source)
        at net.filebot.Cache$TypedCache.computeIfAbsent(Unknown Source)
        at net.filebot.web.AbstractEpisodeListProvider.getSeriesData(Unknown Source)
        at net.filebot.web.AbstractEpisodeListProvider.getEpisodeList(Unknown Source)
        at net.filebot.web.AbstractEpisodeListProvider.getEpisodeList(Unknown Source)
        at net.filebot.web.EpisodeUtilities.fetchEpisodeList(Unknown Source)
        at net.filebot.web.EpisodeUtilities.hydrateEpisode(Unknown Source)
        at net.filebot.format.MediaBindingBean.lambda$getDatabaseMapper$23(Unknown Source)
        at net.filebot.format.DynamicBindings.getProperty(Unknown Source)
        at Script1.run(Script1.groovy:1)
        at net.filebot.format.SecureCompiledScript$1.run(Unknown Source)
        at net.filebot.format.SecureCompiledScript.eval(Unknown Source)
        at net.filebot.format.ExpressionFormat.format(Unknown Source)
        at net.filebot.format.ExpressionFormat.format(Unknown Source)
        at net.filebot.cli.CmdlineOperations.lambda$fetchEpisodeList$16(Unknown Source)
        at net.filebot.cli.ArgumentProcessor.print(Unknown Source)
        at net.filebot.cli.ArgumentProcessor.runCommand(Unknown Source)
        at net.filebot.cli.ArgumentProcessor.run(Unknown Source)
        at net.filebot.Main.main(Unknown Source)
Caused by: java.security.AccessControlException: access denied ("java.net.NetPermission" "setCookieHandler")
        at net.filebot.web.WebRequest.<clinit>(Unknown Source)
        ... 31 more

Error (o_O)

Re: --mapper
The filebot query directly to TheTVDB doesn't have --mapper AnimeList.AniDB, is that only needed to map the episodes (ordering/names?), and not the ID etc?
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

okay, found a machine where the filebot query works (no java error) ..

and .. two out of three have db.anidb.id, tho all of them are correctly enumerated to be The God of High School.

AMC did however group the working two as "movies", with the non-working one as "Anime" ..

Code: Select all

Argument[0]: Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4
Argument[1]: Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[EMBER] God of Highschool - 04.mkv
Argument[2]: Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[Glue] The God of High School - 04 [5AE8F082].mkv
Input: Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4
Input: Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[EMBER] God of Highschool - 04.mkv
Input: Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[Glue] The God of High School - 04 [5AE8F082].mkv
Group: {Movie=null, Series=the god of high school} => [[DeadFish] The God of High School - 04 [720p][AAC].mp4, [EMBER] God of Highschool - 04.mkv]
Group: {Anime=the god of high school} => [[Glue] The God of High School - 04 [5AE8F082].mkv]
Rename episodes using [TheTVDB] with [Airdate Order]
Auto-detected query: [The God of High School, Summer, 2020, god of]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Fetching episode data for [_Summer]
Fetching episode data for [Summer Fever]
Fetching episode data for [Super Summer]
Fetching episode data for [Summer Murder]
Fetching episode data for [Sommer]
Fetching episode data for [20/20]
Fetching episode data for [2020 Space Wonder Kiddy]
Fetching episode data for [2020: The Year of Disasters]
Fetching episode data for [God of]
Fetching episode data for [Soldier]
Fetching episode data for [The God of Risk]
Fetching episode data for [The God of Music]
Fetching episode data for [Master of Study]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[EMBER] God of Highschool - 04.mkv] to [Z:\1-InitialSort\2020\summer\The God of High School [anidb-15408] [tvdb-378556]\[EMBER] God of Highschool - 04.mkv]
Processed 2 files
Rename episodes using [TheTVDB] with [Absolute Order]
Auto-detected query: [The God of High School, Summer, 2020]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Fetching episode data for [_Summer]
Fetching episode data for [Summer Fever]
Fetching episode data for [Super Summer]
Fetching episode data for [Summer Murder]
Fetching episode data for [Sommer]
Fetching episode data for [20/20]
Fetching episode data for [2020 Space Wonder Kiddy]
Fetching episode data for [2020: The Year of Disasters]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[Glue] The God of High School - 04 [5AE8F082].mkv] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556] [animelist-7827572]\[Glue] The God of High School - 04 [5AE8F082].mkv]
Processed 1 files
I tried passing ut_label=Anime to amc .. And now none of them have db.anidb.id

Code: Select all

[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556] [animelist-7827572]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[EMBER] God of Highschool - 04.mkv] to [Z:\1-InitialSort\2020_rerelease\20\20 [tvdb-72289] [animelist-7400327]\[EMBER] God of Highschool - 04.mkv]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[Glue] The God of High School - 04 [5AE8F082].mkv] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556] [animelist-7827572]\[Glue] The God of High School - 04 [5AE8F082].mkv]
and then ut_label=series .. and once again two work as expected, one does not.

Code: Select all

[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[EMBER] God of Highschool - 04.mkv] to [Z:\1-InitialSort\2020_rerelease\20\20 [tvdb-72289] [animelist-7400327]\[EMBER] God of Highschool - 04.mkv]
[TEST] from [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[Glue] The God of High School - 04 [5AE8F082].mkv] to [Z:\1-InitialSort\2020\summer\The God of High School [anidb-15408] [tvdb-378556]\[Glue] The God of High School - 04 [5AE8F082].mkv]
I'm just confused on *why*, I've been manually fixing them every few days but I just can't grasp the rules on when it will work vs not work..
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

This is caused by a strange race condition which makes things randomly work or not work depending on the code flow and cache state prior to this call:

Code: Select all

java.lang.ExceptionInInitializerError
        at net.filebot.CachedResource.lambda$fetchIfModified$10(Unknown Source)
...
Caused by: java.security.AccessControlException: access denied ("java.net.NetPermission" "setCookieHandler")
        at net.filebot.web.WebRequest.<clinit>(Unknown Source)
...

FileBot 4.9.2 (r7800) and higher should hopefully have fixed this issue already:
viewtopic.php?t=1609


rv23 wrote: 02 Aug 2020, 21:44 Re: --mapper
The filebot query directly to TheTVDB doesn't have --mapper AnimeList.AniDB, is that only needed to map the episodes (ordering/names?), and not the ID etc?
The --mapper allows you to replace the episode object that is used for matching, but the final match is going to be the original episode object and the so the --format will work with the original information, and not the information provided by your --mapper expression:
viewtopic.php?t=10996
:idea: Please read the FAQ and How to Request Help.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Thanks.

I'll try updating to 4.9.2 (r7800) and see if that helps.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Hmm.. Still not getting db.AniDB.id using r7829 with the amc script ..

Code: Select all

[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 01 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 02 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 03 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4] to [Z:\1-InitialSort\2020\summer\The God of High School [tvdb-378556]\[DeadFish] The God of High School - 05 [720p][AAC].mp4]
Okay, well perhaps if I simplify the format (but still use amc) .. So set the format for amc to

Code: Select all

"{ any { if (db.AniDB.id) '[anidb-'+{db.AniDB.id}+']' } {'[tvdb-'+db.TheTVDB.id+']'} {'[tmdb-'+tmdbid+']'} }\{fn}"
Nope..

Code: Select all

[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4] to [Z:\1-InitialSort\[tvdb-378556]\[DeadFish] The God of High School - 01 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4] to [Z:\1-InitialSort\[tvdb-378556]\[DeadFish] The God of High School - 02 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4] to [Z:\1-InitialSort\[tvdb-378556]\[DeadFish] The God of High School - 03 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\[tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4] to [Z:\1-InitialSort\[tvdb-378556]\[DeadFish] The God of High School - 05 [720p][AAC].mp4]
well, that's no fun. Okay, how about straight up filebot and no script ..

Code: Select all

.\filebot.exe --action $ACTION -rename "$($FileBotSourceDirectory)" --output "$($FileBotDestDirectory)" --db TheTVDB --log all --format $($MyFormat) -non-strict --mapper AnimeList.AniDB -r
Rename episodes using [TheTVDB] with [Airdate Order]
Auto-detected query: [The God of High School]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Apply mapper [AnimeList.AniDB] on [31] episodes

...

[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4] to [Z:\1-InitialSort\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 01 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4] to [Z:\1-InitialSort\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 02 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4] to [Z:\1-InitialSort\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 03 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 04 [720p][AAC].mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4] to [Z:\1-InitialSort\The God of High School [anidb-15408] [tvdb-378556]\[DeadFish] The God of High School - 05 [720p][AAC].mp4]
Processed 5 files
and works .. Sigh, so is it *something* to do with running amc or running *any* script ..

Any thoughts on the difference between using filebot with a script and without? considering the db should be the same, as well as the format?

Code: Select all

FileBot 4.9.2 (r7829)
JNA Native: 6.1.0
MediaInfo: 20.03
7-Zip-JBinding: 9.20
Chromaprint: 1.5.0
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-08-04 (r667)
Groovy: 3.0.5
JRE: OpenJDK Runtime Environment 14.0.2
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 2.1 GB Max Memory / 24 MB Used Memory
OS: Windows Server 2019 (amd64)
STORAGE: NTFS [(C:)] @ 87 GB | NTFS [music] @ 24 TB | NTFS [pictures] @ 24 TB | NTFS [video] @ 24 TB | NTFS [multimedia] @ 24 TB | NTFS [animebt] @ 23 TB
DATA: C:\Users\xxxx\Desktop\Scripts\FileBot_4.9.2-portable\data
Package: ZIP
License: FileBot License PX10195720 (Valid-Until: 2069-11-10)
Done ?(?????)?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

1.
I'd reduce the entire format to db.AniDB.id and then run with --action TEST.

That should confirm that filebot -rename --format works exactly the same as filebot -script fn:amc --def seriesFormat. Once you have confirmed that db.AniDB.id works within the context of a simple format where nothing can go wrong, the next step is to narrow down where things go wrong in your complex format.


2.
rv23 wrote: 07 Aug 2020, 18:30 Any thoughts on the difference between using filebot with a script and without? considering the db should be the same, as well as the format?
By design, there are no differences, and it does work as far as I can tell:

Code: Select all

$ filebot . -script "g:rename(folder:args, db:'TheTVDB', format:'{db.AniDB.id} | {db.TheTVDB.id} | {episode}')" --action TEST -non-strict --log INFO
[TEST] from [The God of High School.1x01.mp4] to [15408 | 378556 | The God of High School - 1x01 - set up/stand up.mp4]

3.
any(Closure...) will eat up any errors. But you can do your own try / catch / debug print-to-stdout logging:

Code: Select all

{
	any
	{
		try {
			return db.AniDB.id
		} catch(e) {
			println json
			println e
		}
	}
	{
		'Undefined'
	}
}
{json} might help as well so we can see exactly what we're dealing with.
:idea: Please read the FAQ and How to Request Help.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Thanks, I'll try this over the next few days and see what happens.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Re: #1. Entire format db.AniDB.id
Did not work (using r7829).

Code: Select all

PS C:\Users\xxxxx\Desktop\scripts\FileBot_4.9.2-portable>   .\filebot.exe -script $MyScript --action $ACTION "$($FileBotSourceDirectory)" --output "$($FileBotDestDirectory)" `
>>     --def movieFormat="{db.AniDB.id}"  `
>>     animeFormat="{db.AniDB.id}" `
>>     seriesFormat="{db.AniDB.id}"  `
>>     animeDB=TheTVDB `
>>     minFileSize=10 `
>>     minLengthMS=5 `
>>     -no-xattr `
>>     -non-strict `
>>     --mapper AnimeList.AniDB `
>>     -r `
>>     --lang=en `
>>     --log all
Run script [fn:amc] at [Sun Aug 09 09:20:33 CDT 2020]
Parameter: movieFormat = {db.AniDB.id}
Parameter: animeFormat = {db.AniDB.id}
Parameter: seriesFormat = {db.AniDB.id}
Parameter: animeDB = TheTVDB
Parameter: minFileSize = 10
Parameter: minLengthMS = 5
Argument[0]: Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4
Argument[1]: Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4
Argument[2]: Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4
Argument[3]: Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4
Argument[4]: Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4
Group: {Anime=the god of high school} => [[DeadFish] The God of High School - 01 [720p][AAC].mp4, [DeadFish] The God of High School - 02 [720p][AAC].mp4, [DeadFish] The God of High School - 03 [720p][AAC].mp4, [DeadFish] The God of High School - 04 [720p][AAC].mp4, [DeadFish] The God of High School - 05 [720p][AAC].mp4]
Rename episodes using [TheTVDB] with [Absolute Order]
Auto-detected query: [The God of High School]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Apply mapper [AnimeList.AniDB] on [31] episodes
Map [The God of High School - 1x09] to [The God of High School - 09]
Map [The God of High School - 1x10] to [The God of High School - 10]
Map [The God of High School - 1x11] to [The God of High School - 11]
Map [The God of High School - 1x12] to [The God of High School - 12]
Map [The God of High School - 1x13] to [The God of High School - 13]
Mapping not found
net.filebot.format.SuppressedThrowables: Expression yields empty value: Binding "AniDB": Mapping not found
        at net.filebot.format.ExpressionFormat.format(Unknown Source)
        at net.filebot.format.ExpressionFormat.format(Unknown Source)
        at net.filebot.cli.CmdlineOperations.formatMatch(Unknown Source)
        at net.filebot.cli.CmdlineOperations.getRenameMap(Unknown Source)
        at net.filebot.cli.CmdlineOperations.renameSeries(Unknown Source)
        at net.filebot.cli.CmdlineOperations.rename(Unknown Source)
        at net.filebot.cli.ScriptShellBaseClass.rename(Unknown Source)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at Script1$_run_closure65.doCall(Script1.groovy:357)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at Script1.run(Script1.groovy:343)
        at net.filebot.cli.ScriptShell.evaluate(Unknown Source)
        at net.filebot.cli.ScriptShell.runScript(Unknown Source)
        at net.filebot.cli.ArgumentProcessor.runScript(Unknown Source)
        at net.filebot.cli.ArgumentProcessor.run(Unknown Source)
        at net.filebot.Main.main(Unknown Source)
Caused by: net.filebot.format.BindingException: Binding "AniDB": Mapping not found
        at net.filebot.format.DynamicBindings.getProperty(Unknown Source)
        at Script3.run(Script3.groovy:1)
        at net.filebot.format.SecureCompiledScript.lambda$eval$1(Unknown Source)
        at net.filebot.format.SecureCompiledScript.eval(Unknown Source)
        ... 20 more
Caused by: java.lang.Exception: Mapping not found
        at net.filebot.format.MediaBindingBean.lambda$getDatabaseMapper$25(Unknown Source)
        ... 24 more

Finished without processing any files
Abort (?_?)
2. Difference between -script and no-script
Well, it seems on my system there *is* a different behavior .. I do not expect any difference by design, but since I seem to be encountering one how can I figure out why?

Is there a low level debug or something that could be enabled to determine why this doesn't seem to work via script?


3. Here is the output from filebot (using script) and the try/catch/json ..

Code: Select all

...
Argument[0]: Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4
Argument[1]: Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4
Argument[2]: Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4
Argument[3]: Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4
Argument[4]: Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4
Input: Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4
Group: {Anime=the god of high school} => [[DeadFish] The God of High School - 01 [720p][AAC].mp4, [DeadFish] The God of High School - 02 [720p][AAC].mp4, [DeadFish] The God of High School - 03 [720p][AAC].mp4, [DeadFish] The God of High School - 04 [720p][AAC].mp4, [DeadFish] The God of High School - 05 [720p][AAC].mp4]
Rename episodes using [TheTVDB] with [Absolute Order]
Auto-detected query: [The God of High School]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Apply mapper [AnimeList.AniDB] on [31] episodes
Map [The God of High School - 1x09] to [The God of High School - 09]
Map [The God of High School - 1x10] to [The God of High School - 10]
Map [The God of High School - 1x11] to [The God of High School - 11]
Map [The God of High School - 1x12] to [The God of High School - 12]
Map [The God of High School - 1x13] to [The God of High School - 13]
{"@type":"Episode","seriesName":"The God of High School","episode":1,"title":"set up stand up","absolute":1,"airdate":{"year":2020,"month":7,"day":6},"id":7768847,"seriesInfo":{"database":"TheTVDB","order":"Absolute","language":"en","type":"TV Series","id":378556,"name":"The God of High School","aliasNames":["Koko no Kami","GOHS","The God of Highschool","Deus do Ensino MΘdio","God of High School","The God of High School (2020)","God of Highschool","THE GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ???????????","? ?? ????","???? ????","????"],"certification":"TV-14","startDate":{"year":2020,"month":7,"day":6},"genres":["Action","Adventure","Animation","Anime","Comedy","Fantasy","Martial Arts","Science Fiction"],"network":"AT-X","rating":0.0,"ratingCount":0,"runtime":25,"status":"Continuing"}}
net.filebot.format.BindingException: Binding "AniDB": Mapping not found
{"@type":"Episode","seriesName":"The God of High School","episode":2,"title":"renewal soul","absolute":2,"airdate":{"year":2020,"month":7,"day":13},"id":7817266,"seriesInfo":{"database":"TheTVDB","order":"Absolute","language":"en","type":"TV Series","id":378556,"name":"The God of High School","aliasNames":["Koko no Kami","GOHS","The God of Highschool","Deus do Ensino MΘdio","God of High School","The God of High School (2020)","God of Highschool","THE GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ???????????","? ?? ????","???? ????","????"],"certification":"TV-14","startDate":{"year":2020,"month":7,"day":6},"genres":["Action","Adventure","Animation","Anime","Comedy","Fantasy","Martial Arts","Science Fiction"],"network":"AT-X","rating":0.0,"ratingCount":0,"runtime":25,"status":"Continuing"}}
net.filebot.format.BindingException: Binding "AniDB": Mapping not found
{"@type":"Episode","seriesName":"The God of High School","episode":3,"title":"wisdom kingdom","absolute":3,"airdate":{"year":2020,"month":7,"day":20},"id":7827571,"seriesInfo":{"database":"TheTVDB","order":"Absolute","language":"en","type":"TV Series","id":378556,"name":"The God of High School","aliasNames":["Koko no Kami","GOHS","The God of Highschool","Deus do Ensino MΘdio","God of High School","The God of High School (2020)","God of Highschool","THE GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ???????????","? ?? ????","???? ????","????"],"certification":"TV-14","startDate":{"year":2020,"month":7,"day":6},"genres":["Action","Adventure","Animation","Anime","Comedy","Fantasy","Martial Arts","Science Fiction"],"network":"AT-X","rating":0.0,"ratingCount":0,"runtime":25,"status":"Continuing"}}
net.filebot.format.BindingException: Binding "AniDB": Mapping not found
{"@type":"Episode","seriesName":"The God of High School","episode":4,"title":"marriage bonds","absolute":4,"airdate":{"year":2020,"month":7,"day":27},"id":7827572,"seriesInfo":{"database":"TheTVDB","order":"Absolute","language":"en","type":"TV Series","id":378556,"name":"The God of High School","aliasNames":["Koko no Kami","GOHS","The God of Highschool","Deus do Ensino MΘdio","God of High School","The God of High School (2020)","God of Highschool","THE GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ???????????","? ?? ????","???? ????","????"],"certification":"TV-14","startDate":{"year":2020,"month":7,"day":6},"genres":["Action","Adventure","Animation","Anime","Comedy","Fantasy","Martial Arts","Science Fiction"],"network":"AT-X","rating":0.0,"ratingCount":0,"runtime":25,"status":"Continuing"}}
net.filebot.format.BindingException: Binding "AniDB": Mapping not found
{"@type":"Episode","seriesName":"The God of High School","episode":5,"title":"ronde hound","absolute":5,"airdate":{"year":2020,"month":8,"day":3},"id":7827620,"seriesInfo":{"database":"TheTVDB","order":"Absolute","language":"en","type":"TV Series","id":378556,"name":"The God of High School","aliasNames":["Koko no Kami","GOHS","The God of Highschool","Deus do Ensino MΘdio","God of High School","The God of High School (2020)","God of Highschool","THE GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ?????????????","GOD OF HIGH SCHOOL ???????????","? ?? ????","???? ????","????"],"certification":"TV-14","startDate":{"year":2020,"month":7,"day":6},"genres":["Action","Adventure","Animation","Anime","Comedy","Fantasy","Martial Arts","Science Fiction"],"network":"AT-X","rating":0.0,"ratingCount":0,"runtime":25,"status":"Continuing"}}
net.filebot.format.BindingException: Binding "AniDB": Mapping not found
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 01 [720p][AAC].mp4] to [Z:\1-InitialSort\Undefined.mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 02 [720p][AAC].mp4] to [Z:\1-InitialSort\Undefined.mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 03 [720p][AAC].mp4] to [Z:\1-InitialSort\Undefined.mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 04 [720p][AAC].mp4] to [Z:\1-InitialSort\Undefined.mp4]
[TEST] from [Z:\FB-1\anime\[DeadFish] The God of High School - 05 [720p][AAC].mp4] to [Z:\1-InitialSort\Undefined.mp4]
The primary system I run this on is a Windows Server 2019 Standard machine (Build 17763)
I've tried the command using PowerShell 5.x, PowerShell 7.x and Command prompt with all the same results.

My development system is a Windows 10 Professional machine (build 18363)
Same results as the Windows Server 2019 Standard machine

Please advise.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Code: Select all

Group: {Anime=the god of high school} => [[DeadFish] The God of High School - 01 [720p][AAC].mp4, [DeadFish] The God of High School - 02 [720p][AAC].mp4, [DeadFish] The God of High School - 03 [720p][AAC].mp4, [DeadFish] The God of High School - 04 [720p][AAC].mp4, [DeadFish] The God of High School - 05 [720p][AAC].mp4]
Rename episodes using [TheTVDB] with [Absolute Order]
Auto-detected query: [The God of High School]
Fetching episode data for [The God of High School]
Fetching episode data for [King of High School]
Apply mapper [AnimeList.AniDB] on [31] episodes
Map [The God of High School - 1x09] to [The God of High School - 09]
Map [The God of High School - 1x10] to [The God of High School - 10]
Map [The God of High School - 1x11] to [The God of High School - 11]
Map [The God of High School - 1x12] to [The God of High School - 12]
Map [The God of High School - 1x13] to [The God of High School - 13]
Exception: Mapping not found
Finished without processing any files
Abort (×_×)
Just to check if perhaps it was a "windows" thing, I installed filebot on my Qnap NAS and encounter the same error with db.anidb.id when trying to use a script (vs no script). (version r7372)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

The amc script example works by default, so I played around with your customizations. Turns out --def animeDB=TheTVDB breaks things.

By default --def animeDB=AniDB happens to work no mapping required, but switching to TheTVDB (in combination with --order Absolute as per Anime Mode) might not work if Absolute numbers are in play since mapping assumes SxE numbers.



e.g. works by default because --order Airdate happens to be default:

Code: Select all

$ filebot -list --q "The God of High School" --db TheTVDB --format "{id} | {db.AniDB.id} | {episode}" --order Airdate
378556 | 15408 | The God of High School - 1x01 - set up/stand up
...
378556 |  | The God of High School - 1x09
..


e.g. adding --order Absolute makes things not work:

Code: Select all

$ filebot -list --q "The God of High School" --db TheTVDB --format "{id} | {db.AniDB.id} | {episode}" --order Absolute
378556 |  | The God of High School - 1x09
...
378556 |  | The God of High School - 01 - set up/stand up
...


e.g. the {order.airdate} dynamic episode order can be used to fix up the episode numbers to make {db.AniDB} work as expected:

Code: Select all

$ filebot -list --q "The God of High School" --db TheTVDB --format "{id} | {order.airdate.db.AniDB.id} | {episode}" --order Absolute
378556 |  | The God of High School - 1x09
...
378556 | 15408 | The God of High School - 01 - set up/stand up
...


EDIT:

FileBot r7885 will implicitly switch to Airdate order if necessary to make TheTVDB->AniDB mapping work. The other way around won't be an issue in this regard since AniDB only has one order anyway.
:idea: Please read the FAQ and How to Request Help.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

SUCCESS!

Yep, that works... I didn't know you could get the Series ID using {order.airdate.db.AniDB.id}, I was taking it for granted that db.anidb.id would always contain the AniDB ID if there was a match between the Series in theTVDB and AniDB using the AnimeList mapper (which is where I assumed filebot got the info on mapping to/from AniDB/TheTVDB).

But I'm finding lots of "fiddly bits" to that assumption of mine.

Thanks for your help. Even with my frustrations learning groovy/java/filebot, I think purchasing the license for Filebot is still 100% worth it.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

Well, it does come back to different TheTVDB seasons matching different AniDB series ids, so that's why {episode.season} is necessary for mapping things correctly, and {episode.season} happens to be undefined when the episode was generated with {order.absolute} in this case. Definitely a bug though, since {db} should be smart enough to make things work for you and hide these internal complexities.
:idea: Please read the FAQ and How to Request Help.
Wirly
Posts: 9
Joined: 10 Aug 2020, 01:08

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by Wirly »

I was reading through this thread and had a quick question.

Does FileBot use the ScudLee Anime list for matching AniDB titles to TVDB?

After reading through your conversation, I found out about db.TheTVDB and it does exactly what I want.

Code: Select all

# Input filename = [AnY-Conclave]_Mahou_Tsukai_ni_Taisetsu_na_Koto_Natsu_no_Sora_01_[H264_1280x720]_[33D8DD1B].mkv

Series/Anime/{db.TheTVDB.ny}/{'Season ' + db.TheTVDB.s.pad(2)}/{db.TheTVDB.s00e00 + ' - ' + db.TheTVDB.t}

# output filename = Series/Anime/Someday's Dreamers (2003)/Season 02/S02E01 - From Biei.mkv
I tried matching to TVDB directly like a normal show and it didn't work (It gave me S01 instead of S02). However, using this format and matching to AniDB did work. I'm just wondering how FileBot matched the AniDB title to TVDB and how reliable it is.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

1.
Wirly wrote: 13 Aug 2020, 06:31 Does FileBot use the ScudLee Anime list for matching AniDB titles to TVDB?
Yes, {db} and {AnimeList} are indeed based on the ScudLee/anime-lists project.

Wirly wrote: 13 Aug 2020, 06:31 I'm just wondering how FileBot matched the AniDB title to TVDB and how reliable it is.
I reckon that it's going to be fairly reliable for popular and established Anime, but less so for newer less popular Anime because there'll be less people adding mapping information and the numbering may be in flux for a while as people decide on how to best organize things into shows and seasons.

e.g. Gundam SEED and Gundam SEED Destiny used to be 2 separate series ids on both AniDB and TheTVDB, but TheTVDB recently merged those two separate series entries into a new series id that now has 2 seasons.



2.
Why use --db AniDB when you don't actually end up using any AniDB-based information in your format?

Code: Select all

{db.TheTVDB.ny}/{'Season ' + db.TheTVDB.s.pad(2)}/{db.TheTVDB.s00e00 + ' - ' + db.TheTVDB.t}
If you want to rename files with TheTVDB information, but match against AniDB numbers, then you'll want to use the --mapper option for that:
viewtopic.php?t=10996

Code: Select all

filebot -rename *.mkv --db TheTVDB --mapper AnimeList.AniDB --format {plex}
:idea: Please read the FAQ and How to Request Help.
Wirly
Posts: 9
Joined: 10 Aug 2020, 01:08

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by Wirly »

That --mapper option is very handy! I updated my AMC options and I believe I have it working optimally. I'm planning to run it on a folder that I download old anime series to, so the AnimeList will probably be accurate. It seems to work very well, thank you!

The only thing I'm not sure how to fix is changing the location of the excludeList so that it ends up relative to filebot.exe, not the output folder. Not a big deal, just wondering if there's a way to do it without using an absolute path.

Code: Select all

-script
fn:amc
S:/BitTorrent/Complete/z-Anime
--action
copy
--conflict
skip
-non-strict
--log-file
amc_anime.log
--output
R:/Media/Series/Anime
--mapper
AnimeList.AniDB
--def
excludeList=.filebot_amc_anime
unsorted=n
music=n
artwork=n
ut_label=TV
seriesFormat={ny}/{'Season ' + s.pad(2)}/{s00e00 + ' - ' + t}
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

Relative log locations such as --log-file anime.log are placed in the application data folder, but you can use absolute paths if you want to put it elsewhere.


:idea: If you're using the portable, then the log file will be under the program folder by default.


:idea: If you're using the installed version, then you cannot write to the program folder because on Windows the program folder will be write-protected.



EDIT:

Misread. Things apply to --def excludeList just the same more or less though, except that it's relative to --output by default, which you can override by specifying an absolute path. You can use environment variables.
:idea: Please read the FAQ and How to Request Help.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

I'm currently trying to find a solution to using TheTVDB, but wanting AniDB metadata and not being able to get it. Even for series that have mappings in Scud Lee's XML AND are single season. Which baffles me then on why the mappings I had been using don't work, namely {db.AniDB.id} and {order.airdate.db.AniDB.id}.
For all of these examples, I verified as of today (12/18/2020) all of them have mappings in Scud Lee's XML, and the TVDBID in the xml is correct, and it has a defaulttvdbseason ..
Several examples:
Is there something odd about the TVDB Episode info such that only the FIRST episode has ANIDB metadata?

Code: Select all

filebot -list --q "335088" --db TheTVDB --format "{id} | {order.airdate.db.AniDB.id} | {db.AniDB.id} | {episode}" --order Airdate
335088 | 12622 | 12622 | Nekopara - 1x01 - Welcome to La Soleil!
335088 |  |  | Nekopara - 1x02 - A Lost Kitten?
335088 |  |  | Nekopara - 1x03 - The One My Mind
335088 |  |  | Nekopara - 1x04 - The First Time...!
335088 |  |  | Nekopara - 1x05 - Cacao's Adventure
335088 |  |  | Nekopara - 1x06 - Battles Without Honor and Cat-Humanity!
335088 |  |  | Nekopara - 1x07 - House-Sitting Cats
335088 |  |  | Nekopara - 1x08 - The Melancholy of Shigure
335088 |  |  | Nekopara - 1x09 - True Feelings
335088 |  |  | Nekopara - 1x10 - Oh No! Bell Renewal Exams
335088 |  |  | Nekopara - 1x11 - Cacao Repays Her Debts
335088 |  |  | Nekopara - 1x12 - Summer Beach Cat Paradise!
335088 |  |  | Nekopara - Special 1 - NEKOPARA OVA
335088 |  |  | Nekopara - Special 2 - NEKOPARA OVA Extra

Code: Select all

filebot -list  --db TheTVDB --format "{id} | {order.airdate.db.AniDB.id} | {db.AniDB.id} | {episode}" --order Airdate --q "384424"
384424 |  |  | Ninja Collection - 1x01 - Smoke
384424 |  |  | Ninja Collection - 1x02 - Creepy Gaze
384424 |  |  | Ninja Collection - 1x03 - Night Pool
384424 |  |  | Ninja Collection - 1x04 - The Beautician
384424 |  |  | Ninja Collection - 1x05 - Stain
384424 |  |  | Ninja Collection - 1x06 - Shadey and Whoosher
384424 |  |  | Ninja Collection - 1x07 - Owl
384424 |  |  | Ninja Collection - 1x08 - The Waiting
384424 |  |  | Ninja Collection - 1x09 - Critical Condition
384424 |  |  | Ninja Collection - 1x10 - Forgetting
384424 |  |  | Ninja Collection - 1x11 - Water's Surface
384424 |  |  | Ninja Collection - 1x12 - Thirst
384424 |  |  | Ninja Collection - 1x13 - Cat's Cradle

Code: Select all

filebot -list  --db TheTVDB --format "{id} | {order.airdate.db.AniDB.id} | {db.AniDB.id} | {episode}" --order Airdate --q "384017"
384017 |  |  | Mr Love: Queen's Choice - 1x01 - The Beginning Bonds
384017 |  |  | Mr Love: Queen's Choice - 1x02 - When The Wind Blows
384017 |  |  | Mr Love: Queen's Choice - 1x03 - A Taste of Reminiscence
384017 |  |  | Mr Love: Queen's Choice - 1x04 - The Key in the Darkness
384017 |  |  | Mr Love: Queen's Choice - 1x05 - An Amber Wind
384017 |  |  | Mr Love: Queen's Choice - 1x06 - Beyond That Dream
384017 |  |  | Mr Love: Queen's Choice - 1x07 - Connected Memories
384017 |  |  | Mr Love: Queen's Choice - 1x08 - Room 404
384017 |  |  | Mr Love: Queen's Choice - 1x09 - Monochrome
384017 |  |  | Mr Love: Queen's Choice - 1x10 - Dawn of Farewell
384017 |  |  | Mr Love: Queen's Choice - 1x11 - At the Edge of the Coming Time
384017 |  |  | Mr Love: Queen's Choice - 1x12 - Bonds
and many more..
Along with at least one (might be able to locate others) that have mappings, are multiple seasons, and the only seasons that do not seem to map are those with offsets ..

Code: Select all

filebot -list  --db TheTVDB --format "{id} | {order.airdate.db.AniDB.id} | {db.AniDB.id} | {episode}" --order Airdate --q "259640"
259640 | 8692 | 8692 | Sword Art Online - 1x01 - The World of Swords
259640 | 8692 | 8692 | Sword Art Online - 1x02 - Beater
259640 | 8692 | 8692 | Sword Art Online - 1x03 - The Red-Nosed Reindeer
259640 | 8692 | 8692 | Sword Art Online - 1x04 - The Black Swordsman
259640 | 8692 | 8692 | Sword Art Online - 1x05 - Murder in the Safe Zone
259640 | 8692 | 8692 | Sword Art Online - 1x06 - Illusionary Avenger
259640 | 8692 | 8692 | Sword Art Online - 1x07 - The Temperature of the Heart
259640 | 8692 | 8692 | Sword Art Online - 1x08 - The Sword Dance of Black and White
259640 | 8692 | 8692 | Sword Art Online - 1x09 - The Blue-Eyed Demon
259640 | 8692 | 8692 | Sword Art Online - 1x10 - Crimson Killing Intent
259640 | 8692 | 8692 | Sword Art Online - 1x11 - The Girl of Morning Dew
259640 | 8692 | 8692 | Sword Art Online - 1x12 - Yui's Heart
259640 | 8692 | 8692 | Sword Art Online - 1x13 - Edge of Hell's Abyss
259640 | 8692 | 8692 | Sword Art Online - 1x14 - The End of the World
259640 | 8692 | 8692 | Sword Art Online - 1x15 - Return
259640 | 8692 | 8692 | Sword Art Online - 1x16 - Land of the Fairies
259640 | 8692 | 8692 | Sword Art Online - 1x17 - Captive Queen
259640 | 8692 | 8692 | Sword Art Online - 1x18 - To the World Tree
259640 | 8692 | 8692 | Sword Art Online - 1x19 - The Lugru Corridor
259640 | 8692 | 8692 | Sword Art Online - 1x20 - General of the Blazing Flame
259640 | 8692 | 8692 | Sword Art Online - 1x21 - The Truth About Alfheim
259640 | 8692 | 8692 | Sword Art Online - 1x22 - The Grand Quest
259640 | 8692 | 8692 | Sword Art Online - 1x23 - Bonds
259640 | 8692 | 8692 | Sword Art Online - 1x24 - Gilded Hero
259640 | 8692 | 8692 | Sword Art Online - 1x25 - The World Seed
259640 | 10376 | 10376 | Sword Art Online - 2x01 - The World of Guns
259640 | 10376 | 10376 | Sword Art Online - 2x02 - Cold-Hearted Sniper
259640 | 10376 | 10376 | Sword Art Online - 2x03 - Memories of Blood
259640 | 10376 | 10376 | Sword Art Online - 2x04 - GGO (Gun Gale Online)
259640 | 10376 | 10376 | Sword Art Online - 2x05 - Guns and Swords
259640 | 10376 | 10376 | Sword Art Online - 2x06 - Showdown in the Wilderness
259640 | 10376 | 10376 | Sword Art Online - 2x07 - Crimson Memories
259640 | 10376 | 10376 | Sword Art Online - 2x08 - Bullet of Bullets
259640 | 10376 | 10376 | Sword Art Online - 2x09 - Death Gun
259640 | 10376 | 10376 | Sword Art Online - 2x10 - Death Chaser
259640 | 10376 | 10376 | Sword Art Online - 2x11 - What It Means To Be Strong
259640 | 10376 | 10376 | Sword Art Online - 2x12 - Bullet of a Phantom
259640 | 10376 | 10376 | Sword Art Online - 2x13 - Phantom Bullet
259640 | 10376 | 10376 | Sword Art Online - 2x14 - One Little Step
259640 | 10376 | 10376 | Sword Art Online - 2x15 - The Queen of the Lake
259640 | 10376 | 10376 | Sword Art Online - 2x16 - The King of the Giants
259640 | 10376 | 10376 | Sword Art Online - 2x17 - Excalibur
259640 | 10376 | 10376 | Sword Art Online - 2x18 - Forest House
259640 | 10376 | 10376 | Sword Art Online - 2x19 - Zekken
259640 | 10376 | 10376 | Sword Art Online - 2x20 - The Sleeping Knights
259640 | 10376 | 10376 | Sword Art Online - 2x21 - The Monument of Swordsmen
259640 | 10376 | 10376 | Sword Art Online - 2x22 - The End of the Journey
259640 | 10376 | 10376 | Sword Art Online - 2x23 - The Dream Begins
259640 | 10376 | 10376 | Sword Art Online - 2x24 - Mother's Rosario
259640 | 13493 | 13493 | Sword Art Online - 3x01 - Underworld
259640 | 13493 | 13493 | Sword Art Online - 3x02 - The Demon Tree
259640 | 13493 | 13493 | Sword Art Online - 3x03 - The End Mountains
259640 | 13493 | 13493 | Sword Art Online - 3x04 - Departure
259640 | 13493 | 13493 | Sword Art Online - 3x05 - Ocean Turtle
259640 | 13493 | 13493 | Sword Art Online - 3x06 - Project Alicization
259640 | 13493 | 13493 | Sword Art Online - 3x07 - Swordcraft Academy
259640 | 13493 | 13493 | Sword Art Online - 3x08 - Swordsman's Pride
259640 | 13493 | 13493 | Sword Art Online - 3x09 - Nobleman's Responsibilities
259640 | 13493 | 13493 | Sword Art Online - 3x10 - Taboo Index
259640 | 13493 | 13493 | Sword Art Online - 3x11 - Central Cathedral
259640 | 13493 | 13493 | Sword Art Online - 3x12 - The Sage of the Library
259640 | 13493 | 13493 | Sword Art Online - 3x13 - Ruler and Mediator
259640 | 13493 | 13493 | Sword Art Online - 3x14 - The Crimson Knight
259640 | 13493 | 13493 | Sword Art Online - 3x15 - The Relentless Knight
259640 | 13493 | 13493 | Sword Art Online - 3x16 - The Osmanthus Knight
259640 | 13493 | 13493 | Sword Art Online - 3x17 - Truce
259640 | 13493 | 13493 | Sword Art Online - 3x18 - The Legendary Hero
259640 | 13493 | 13493 | Sword Art Online - 3x19 - The Seal of the Right Eye
259640 | 13493 | 13493 | Sword Art Online - 3x20 - Synthesis
259640 | 13493 | 13493 | Sword Art Online - 3x21 - The 32nd Knight
259640 | 13493 | 13493 | Sword Art Online - 3x22 - Titan of the Sword
259640 | 13493 | 13493 | Sword Art Online - 3x23 - Administrator
259640 | 13493 | 13493 | Sword Art Online - 3x24 - My Hero
259640 | 14796 | 14796 | Sword Art Online - 4x01 - In the Far North
259640 | 14796 | 14796 | Sword Art Online - 4x02 - Raids
259640 | 14796 | 14796 | Sword Art Online - 4x03 - The Final Load Test
259640 | 14796 | 14796 | Sword Art Online - 4x04 - Dark Territory
259640 | 14796 | 14796 | Sword Art Online - 4x05 - The Night Before Battle
259640 | 14796 | 14796 | Sword Art Online - 4x06 - Battle of the Knights
259640 | 14796 | 14796 | Sword Art Online - 4x07 - Stigma of the Disqualified
259640 | 14796 | 14796 | Sword Art Online - 4x08 - Blood and Life
259640 | 14796 | 14796 | Sword Art Online - 4x09 - Sword and Fist
259640 | 14796 | 14796 | Sword Art Online - 4x10 - Stacia, the Goddess of Creation
259640 | 14796 | 14796 | Sword Art Online - 4x11 - Heartless Choice
259640 | 14796 | 14796 | Sword Art Online - 4x12 - Ray of Light
259640 |  |  | Sword Art Online - 4x13 - The War of Underworld
259640 |  |  | Sword Art Online - 4x14 - End to Eternity
259640 |  |  | Sword Art Online - 4x15 - Instigation
259640 |  |  | Sword Art Online - 4x16 - Code 871
259640 |  |  | Sword Art Online - 4x17 - Prince of Hell
259640 |  |  | Sword Art Online - 4x18 - Memories
259640 |  |  | Sword Art Online - 4x19 - Awakening
259640 |  |  | Sword Art Online - 4x20 - The Night-Sky Blade
259640 |  |  | Sword Art Online - 4x21 - Beyond Time
259640 |  |  | Sword Art Online - 4x22 - Alice
259640 |  |  | Sword Art Online - 4x23 - New World
With Season 4 after episode 12 having an offset to map to AniDB ..

Code: Select all

  <anime anidbid="15146" tvdbid="259640" defaulttvdbseason="4" episodeoffset="12">
    <name>Sword Art Online: Alicization - War of Underworld (2020)</name>
  </anime>
Here is the usual filebot sysinfo ..

Code: Select all

FileBot 4.9.2 (r8046)
JNA Native: 6.1.0
MediaInfo: 20.08
7-Zip-JBinding: 16.02
Tools: fpcalc/1.5.0
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-12-01 (r724)
Groovy: 3.0.6
JRE: OpenJDK Runtime Environment 15
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 2.1 GB Max Memory / 39 MB Used Memory
OS: Windows Server 2019 (amd64)
STORAGE: NTFS [(C:)] @ 79 GB | NTFS [music] @ 29 TB | NTFS [pictures] @ 29 TB | NTFS [video] @ 29 TB | NTFS [multimedia] @ 29 TB | NTFS [anime] @ 10 TB
DATA: C:\Users\xxxxx\AppData\Roaming\FileBot
Package: MSI
License: FileBot License xxxxxxx(Valid-Until: 2069-11-10)
Done ?(?????)?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

I'll look into it. The first episode working but the rest not working is very strange indeed.
:idea: Please read the FAQ and How to Request Help.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

If it makes any difference, a fresh download/run of r8296 returns the same behavior.
Using --q "ID#" or --q "Series Name" also returns the same behavior.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

1.
Looks like TheTVDB::335088 incorrectly maps to AniDB::12622 (which only has 1 episode) instead of AniDB::14570:

Code: Select all

335088 | Nekopara - 1x01 - Welcome to La Soleil! | 12622 | Nekopara - 01 - OVA

2.
FileBot will maintain its own copy of the repository, so recent changes may not immediately trickle down into your local cache. Might be that if the changes you need are recent. I'll look into it.


EDIT:

Further investigation reveals that our modified edition of anime-list.xml reveals that we're stripping entries where we don't have a corresponding TheTVDB series name, which eliminates less popular shows from our mapping table.
:idea: Please read the FAQ and How to Request Help.
rv23
Posts: 64
Joined: 19 Jul 2020, 15:05

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rv23 »

Re: #1 - Good call, I totally forgot to verify the mapping was correct, my apologies for wasting your time on that one.
Re: #2 -
Part 1 - Is it a set interval for FileBot to refresh the mappings? Is there any way to determine the last time it refreshed or initiate a refresh?

Part 2 (Stripping entries) - I'm confused on what this is explanation. Is FileBot stripping entries where FileBot doesn't have an entry for the corresponding TheTVDB Series Name (internally) that is listed in anime-list.xml?

As I'm primarily interested in AniDB names/ID for sorting and I can download anime-list.xml (So I can tell there is *a* mapping) .. I couldn't see any way to query FileBot if *it* has the mapping present in anime-list.xml, did I miss a method that I can use to do this?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: db.anidb.id, AnimeList.anidb.id inconsistency when using --mapper AnimeList.AniDB with theTVDB database

Post by rednoah »

rv23 wrote: 21 Dec 2020, 21:07 Part 2 (Stripping entries) - I'm confused on what this is explanation. Is FileBot stripping entries where FileBot doesn't have an entry for the corresponding TheTVDB Series Name (internally) that is listed in anime-list.xml?
There are lots of entries on AnimeLists where the corresponding TheTVDB ID hasn't been entered yet, or has been entered years ago, but then the corresponding entry has since been removed from TheTVDB making the mapping invalid. Additionally, FileBot requires both AniDB series name as well as TheTVDB series name to map back and forth seamlessly, rather than being limited to mapping from TheTVDB to AniDB one-way.

rv23 wrote: 21 Dec 2020, 21:07 As I'm primarily interested in AniDB names/ID for sorting and I can download anime-list.xml (So I can tell there is *a* mapping) .. I couldn't see any way to query FileBot if *it* has the mapping present in anime-list.xml, did I miss a method that I can use to do this?
You can use GroovyPad to run tests within FileBot like so:

Code: Select all

WebServices.AnimeList.model.anime.findAll{
	it.anidbid == 12622 || it.tvdbid == 335088
} each{
	println it
}
Turns out that there is a mapping, except there's more than one, and FileBot just so happens to pick the first one... Not really sure how to deal with this one then...

Code: Select all

<anime anidbid="12622" tvdbid="335088" defaulttvdbseason="1"><name>Nekopara</name><tvdbname>Nekopara</tvdbname></anime>
<anime anidbid="14206" tvdbid="335088" defaulttvdbseason="0"><name>Nekopara: Koneko no Hi no Yakusoku</name><tvdbname>Nekopara</tvdbname></anime>
<anime anidbid="14570" tvdbid="335088" defaulttvdbseason="1"><name>Nekopara (2020)</name><tvdbname>Nekopara</tvdbname></anime>

:idea: I've also forced a new build and cleared all caches. Please run -clear-cache once to see if you get better results afterwards.
:idea: Please read the FAQ and How to Request Help.
Post Reply