Page 1 of 1

anime-lists data sync

Posted: 17 Aug 2022, 07:20
by Placinta
Hi,

I saw your reply at viewtopic.php?p=54219#p54219 that Anime-Lists/anime-lists data is updated once per month.

In that context I have a few questions:
1) Is the updated data tied to the version of filebot?
2) Or is it done in some server-side process?
3) Is it possible to manually trigger the update?
4) Is it possible to point to filebot to a different fork of the data?
5) Is it possible to check what version of the database is currently used by filebot?
6) Is it possible to inspect if a specific show has a mapping from anidb to tvdb?

Hope you can answer at least some of the questions.

Thanks, and have a nice day!

Re: anime-lists data sync

Posted: 17 Aug 2022, 11:02
by rednoah
:?: What's not working?

:?: What specific problem did you encounter? Context is extremely useful.



1.
All recent versions of FileBot use the same data files and reload the latest data files periodically.

2.
The data files are re-generated on the server-side periodically and then served as static resources via CloudFlare Edge caches.

3.
You cannot manually trigger a server-side update. You can however clear the local application cache to maybe get the latest data files a little bit earlier. This is not recommended and generally not necessary though.

4.
No.

5. & 6.
I suppose you could use the {db} or {AnimeList} bindings to check the mappings that FileBot is currently working with:

Code: Select all

$ filebot -list --db AniDB --q 00026 --format "{AnimeList.TheTVDB}"
The Twelve Kingdoms - 1x01 - Shadow of the Moon, Sea of the Shadow - Chapter One
...
:?: Which specific series (i.e. AniDB ID) are you having trouble with?

Re: anime-lists data sync

Posted: 17 Aug 2022, 15:57
by Placinta
Hi,

Sorry for not posting more context, I figured the questions where generic enough that context was not needed, so thank you for providing the answers.
I also didn't have the time to try and troubleshoot the issue more deeply (although i did search quite a bit), so I didn't want to waste your time.

I use the amc script to auto-symlink downloaded anime shows into an anidb-based media folder for consumption in plex, and sometimes the wrong show is picked up, due to the usual discrepancy in files names. The same script with the same parameters is used to categorize regular tv shows and movies.

In particular the following episodes were mis-identified:
The paths were edited for privacy reasons.

Code: Select all

[SubsPlease] Dungeon ni Deai wo Motomeru no wa Machigatteiru Darou ka S4 - 01 (1080p) [1EFD929B].mkv -> 
Anime/Dungeon ni Deai o Motomeru no wa Machigatte Iru Darouka - Familia Myth III/Dungeon ni Deai o Motomeru no wa Machigatte Iru Darouka - Familia Myth III - 01 - Wiene - Dragon's Daughter.mkv

[SubsPlease] Kanojo, Okarishimasu - 13 (1080p) [733174A7].mkv -> 
Anime/Kanojo, Okarishimasu/Kanojo, Okarishimasu - 0x13 - Date Movie 13.mkv

[SubsPlease] Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e S2 - 07 (1080p) [CB6F0C63].mkv -> 

/Anime/Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e/Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e - 07 - Nothing Is As Dangerous as an Ignorant Friend; a Wise Enemy Is to Be Preferred.mkv
Here's a sample log

Code: Select all

Run script [fn:amc] at [Mon Aug 15 16:00:07 CEST 2022]
Parameter: music = y
Parameter: skipExtract = y
Parameter: ut_kind = multi
Parameter: ut_dir = ~/sonarr/[SubsPlease] Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e S2 - 07 (1080p) [CB6F0C63].mkv
Parameter: ut_title = [SubsPlease] Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e S2 - 07 (1080p) [CB6F0C63].mkv
Parameter: artwork = false
Parameter: extractfolder = /downloads/extracted
Input: ~/sonarr/[SubsPlease] Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e S2 - 07 (1080p) [CB6F0C63].mkv
Group: {Anime=youkoso jitsuryoku shijou shugi no kyoushitsu e} => [[SubsPlease] Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e S2 - 07 (1080p) [CB6F0C63].mkv]
Rename episodes using [AniDB] with [Absolute]
Lookup via [Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e, Youkoso, youkoso jitsuryoku shijou shugi no kyoushitsu e s2]
Fetching episode data for [Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e]
Fetching episode data for [Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e 2nd Season]
Fetching episode data for [Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e 3rd Season]
Fetching episode data for [Youkoso]
Fetching episode data for [NHK ni Youkoso!]
Fetching episode data for [Youkoso Japari Park]
Fetching episode data for [Youkoso Lodoss-tou e!]
Fetching episode data for [Youkoso! Sukebe Elf no Mori e]
[OVERRIDE] Delete [~/Anime/Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e/Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e - 07 - Nothing Is As Dangerous as an Ignorant Friend; a Wise Enemy Is to Be Preferred.mkv]
[SYMLINK] from [~/sonarr/[SubsPlease] Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e S2 - 07 (1080p) [CB6F0C63].mkv] to [~/Anime/Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e/Youkoso Jitsuryoku Shijou Shugi no Kyoushitsu e - 07 - Nothing Is As Dangerous as an Ignorant Friend; a Wise Enemy Is to Be Preferred.mkv]
Processed 1 file
My setup is quite unusual, so I didn't really want to request support because I know it's not kosher.

I'm using sonarr to download new episodes, which sends the requests to rtorrent, which ultimately invokes the amc script.
I can't really use the filebot age filter (at least i don't think i can), because i don't always download only recent episodes.

And unfortunately sonarr does not allow specifying separate labels based on the media type, so i could specify more specific options to the amc script based on the label.

For completeness sake:

Code: Select all


filebot -no-xattr -script fn:amc --def music=y --def skipExtract=n -non-strict --encoding utf8 --def ut_kind=multi --log all --log-file amc.log --def "ut_dir=$TORRENT_PATH" "ut_title=$TORRENT_NAME" --output "~/media" --action symlink --conflict override --def artwork=false --def extractfolder="~/extracted"

Code: Select all

filebot -script fn:sysinfo
FileBot 4.9.6 (r9125)
JNA Native: 5.2.2
MediaInfo: 21.09
Tools: unrar/5.61
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2022-08-15 (r836)
Groovy: 3.0.9
JRE: OpenJDK Runtime Environment 11.0.16
JVM: OpenJDK 64-Bit Server VM
CPU/MEM: 8 Core / 8 GB Max Memory / 53 MB Used Memory
OS: Linux (amd64)
HW: Linux host 5.15.6-xanmod2 #0~git20211202.9a80d50 SMP Thu Dec 2 17:33:26 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
CPU/MEM: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz [MemTotal: 33 GB | MemFree: 492 MB | MemAvailable: 27 GB | SwapCached: 30 MB | SwapTotal: 17 GB | SwapFree: 16 GB]
STORAGE: btrfs [/] @ 1.7 TB 
USER: myuser
DATA: ~/myuser/.filebot
Package: DEB
License: FileBot License PX33732442 (Valid-Until: 2072-02-12)
Done ヾ(@⌒ー⌒@)ノ

Re: anime-lists data sync

Posted: 17 Aug 2022, 21:02
by rednoah

Code: Select all

Rename episodes using [AniDB] with [Absolute]
Note that if you simply match and rename against AniDB then anime-lists data is never used.


:idea: The anime-lists mapping information is used for matching against AniDB and then mapping that to the corresponding TheTVDB episode information (with SxE numbers) requires the anime-lists mapping information. But if you only match against AniDB and rename accordingly (naming and numbering) then only AniDB episode information is used.


Here's my educated guess as to why you got mismatches:
:arrow: You'll definitely want to somehow set up a date filter for your recently aired episodes so that you can just rename your files to whatever episode was aired recently, especially if the naming / numbering doesn't at all match what's in your selected database.

Re: anime-lists data sync

Posted: 18 Aug 2022, 06:27
by Placinta
The anime-lists mapping information is used for matching against AniDB and then mapping that to the corresponding TheTVDB episode information (with SxE numbers) requires the anime-lists mapping information. But if you only match against AniDB and rename accordingly (naming and numbering) then only AniDB episode information is used.
Kanojo, Okarishimasu has only 12 episodes. Maybe your Episode 13 is actually Episode 1 of Kanojo, Okarishimasu (2022)
It is indeed episode 1 of the the 2022 season.

Looking at both databases

https://thetvdb.com/series/rental-girlf ... official/2
https://anidb.net/anime/15743

none of them mention 13, but Sonarr was able to match the file name containing 13 to season 2 episode 1 of tvdb db, and I think 13 means tvdb absolute numbering, according to viewtopic.php?p=44274#p44274.

I suppose I expected for filebot to be able to use the anime-lists data, to map from tvdb absolute numbering, to anidb style numbering. Is that not possible?

I don't think that an age / date based approach would work in this particular case, because in the end i want anidb style naming, but for dealing with filenames containing tvdb absolute numbers i'd be forced to use the tvdb database for renaming.

Re: anime-lists data sync

Posted: 18 Aug 2022, 12:13
by rednoah
The anime-lists data can be used to map AniDB / Absolute numbers to TheTVDB SxE numbers and vice versa. But that's not relevant unless you explicitly use --mapper to match files differently or --format to name files differently, i.e. FileBot does not consider mapping information unless you explicitly configure FileBot to do so.



:idea: If the files are named according to TheTVDB information then you must use --db TheTVDB for things to just work:

Code: Select all

$ filebot -rename *.mp4 --db TheTVDB -non-strict --action TEST --log INFO
[TEST] from [Kanojo, Okarishimasu - 13.mp4] to [Rent-a-Girlfriend - 2x01 - Dream and Girlfriend.mp4]

Placinta wrote: 18 Aug 2022, 06:27 I suppose I expected for filebot to be able to use the anime-lists data, to map from tvdb absolute numbering, to anidb style numbering. Is that not possible?
The anime-lists mapping information comes into play if you use --mapper to work with AniDB as database but match episodes as if you're working with TheTVDB episode numbers:

Code: Select all

$ filebot -rename *.mp4 --db AniDB --mapper db.TheTVDB.episode -non-strict --action TEST --log INFO
[TEST] from [Kanojo, Okarishimasu - 13.mp4] to [Rent-a-Girlfriend (2022) - 01 - Dream and Girlfriend.mp4]

Placinta wrote: 18 Aug 2022, 06:27 I don't think that an age / date based approach would work in this particular case, because in the end i want anidb style naming, but for dealing with filenames containing tvdb absolute numbers i'd be forced to use the tvdb database for renaming.
The age / date approach is exactly what you need if you want to make magically make the leap from Series A - Episode 14 to Series B - Episode 2 without external mapping information.

Code: Select all

$ filebot -rename *.mp4 --db AniDB --filter "age < 7" -non-strict --action TEST --log INFO
[TEST] from [Kanojo, Okarishimasu.mp4] to [Rent-a-Girlfriend (2022) - 07 - Ex and Girlfriend.mp4]
(in this example, I've omitted the episode number entirely to illustrate how it doesn't matter, because for this test case, we have decided that our test file is the most recently released episode)

:idea: This of course works only if you are able to process the newly released episode within the allotted time frame, but that's pretty much what the *arrs tools do as far as I know.