[DEPRECATED] support for XEM?
-
- Posts: 35
- Joined: 14 Jul 2018, 17:22
[DEPRECATED] support for XEM?
Hi,
I am wondering if support for XEM has been considered (to augment tvdb for example). I have run across several shows that have discrepancies between naming/numbering from the scene and what thetvdb lists. These are always fixable in a manual way, but having it automated would be even better.
many thanks.
-hm
I am wondering if support for XEM has been considered (to augment tvdb for example). I have run across several shows that have discrepancies between naming/numbering from the scene and what thetvdb lists. These are always fixable in a manual way, but having it automated would be even better.
many thanks.
-hm
Re: support for XEM?
1.
What is XEM?
2.
Are you using the amc script? The Advanced File-Tuning section will explain a good solution for making things pretty much error-proof for newly aired episodes.
What is XEM?
2.
Are you using the amc script? The Advanced File-Tuning section will explain a good solution for making things pretty much error-proof for newly aired episodes.
-
- Posts: 35
- Joined: 14 Jul 2018, 17:22
Re: support for XEM?
XEM is the cross entity map for tv shows.
http://thexem.de/
Basically, discrepancies in tvdb, anidb and the scene are managed here. The episode mismatches for shows like Mythbusters and SNL have already been re-mapped in XEM. Season 43 of SNL is a great example of the issue, where the scene episode numbering is off by 2 compared to tvdb. This is because of how tvdb handled 2 specials that aired this season.
Fixing this is not that difficult, but i have to manually edit the files and change the filename. if we could use XEM, this wouldn't be necessary as someone has already "fixed" this mismatch in the XEM data. Just for clarity, the mismatch is in the scene numbering and what tvdb thinks that episode is. For example, SNL S43E23 is Tina Fey/Nicki Minaj per the scene numbering. SNL S43E23 does not exist in tvdb, S43E21 is the Tina Fey/Nicki Minaj episode in their database.
Lastly, I am using your AMC script (it is fantastic) and am using the advanced file tuning suggestions. This doesn't remedy this particular situation however.
-hm
http://thexem.de/
Basically, discrepancies in tvdb, anidb and the scene are managed here. The episode mismatches for shows like Mythbusters and SNL have already been re-mapped in XEM. Season 43 of SNL is a great example of the issue, where the scene episode numbering is off by 2 compared to tvdb. This is because of how tvdb handled 2 specials that aired this season.
Fixing this is not that difficult, but i have to manually edit the files and change the filename. if we could use XEM, this wouldn't be necessary as someone has already "fixed" this mismatch in the XEM data. Just for clarity, the mismatch is in the scene numbering and what tvdb thinks that episode is. For example, SNL S43E23 is Tina Fey/Nicki Minaj per the scene numbering. SNL S43E23 does not exist in tvdb, S43E21 is the Tina Fey/Nicki Minaj episode in their database.
Lastly, I am using your AMC script (it is fantastic) and am using the advanced file tuning suggestions. This doesn't remedy this particular situation however.
-hm
Re: support for XEM?
A few examples of original file names that get matched incorrectly by the amc script would be helpful.
-
- Posts: 35
- Joined: 14 Jul 2018, 17:22
Re: support for XEM?
filebot output is not all that useful (in this example) as the file is seemingly properly identified as Saturday Night live S43E21 and filebot using tvdb applies S43E21 metadata.
However, the scene numbering for S43E21 is really the Donald Glover/Childish Gambino episode. the TVDB has this as the Tina Fey/Nicki Minage episode instead.
Here is theTVDB page for this episode https://www.thetvdb.com/series/saturday ... es/6681723
Here is the official NBC page indicating S43E21 is the Donald Glover episode https://www.nbc.com/saturday-night-live ... er/3712034
Here is theXEM page showing how it fixes this issue http://thexem.de/xem/show/4294
Finally, here is the output from when I run filebot:
and here is a screenshot of this episode playing in Plex showing Donald Glover. https://imgur.com/a/eii2huU
this is a screengrab from inside Plex, showing this episode identified as Tina fey https://imgur.com/a/NpdUXns
However, the scene numbering for S43E21 is really the Donald Glover/Childish Gambino episode. the TVDB has this as the Tina Fey/Nicki Minage episode instead.
Here is theTVDB page for this episode https://www.thetvdb.com/series/saturday ... es/6681723
Here is the official NBC page indicating S43E21 is the Donald Glover episode https://www.nbc.com/saturday-night-live ... er/3712034
Here is theXEM page showing how it fixes this issue http://thexem.de/xem/show/4294
Finally, here is the output from when I run filebot:
Code: Select all
Run script [fn:amc] at [Mon Jul 16 23:43:36 CEST 2018]
Parameter: ut_dir = /home/hd1/torrents/data/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta.mkv/.
Parameter: subtitles = en
Parameter: artwork = false
Parameter: extractFolder = /home/hd1/torrents/extracted
Parameter: skipExtract = n
Parameter: ut_label = TV
Parameter: music = y
Parameter: clean = y
Input: /home/hd1/torrents/data/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta.mkv
Group: [tvs:saturday night live] => [Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta.mkv]
Get [English] subtitles for 1 files
Looking up subtitles by hash via OpenSubtitles
No matching subtitles found: /home/hd1/torrents/data/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta.mkv
Rename episodes using [TheTVDB]
Auto-detected query: [Saturday Night Live]
Fetching episode data for [Saturday Night Live]
Fetching episode data for [SNL Poland]
Fetching episode data for [Saturday Night Live (JP)]
Fetching episode data for [Saturday Night Live Korea]
Fetching episode data for [Weekend Update]
[OVERRIDE] Delete [/home/hd1/media/TV Shows/Saturday Night Live/Season 43/Saturday Night Live - S43E21 - Tina Fey Nicki Minaj.mkv]
[COPY] From [/home/hd1/torrents/data/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta/Saturday.Night.Live.S43E21.1080p.HEVC.x265-MeGusta.mkv] to [/home/hd1/media/TV Shows/Saturday Night Live/Season 43/Saturday Night Live - S43E21 - Tina Fey Nicki Minaj.mkv]
Processed 1 files
this is a screengrab from inside Plex, showing this episode identified as Tina fey https://imgur.com/a/NpdUXns
Re: support for XEM?
Thanks for the logs. I always need the original file paths so I can add them to my test dataset.
Support for XEM won't make it into the upcoming release, but maybe the next.
Support for XEM won't make it into the upcoming release, but maybe the next.
Re: support for XEM?
+1 for interest in this.
Any update on if we'll see it in a future release?
Any update on if we'll see it in a future release?
Re: support for XEM?
It's on my TODO list, but I'm not quite sure on how to implement this well in a consistent way that fits into how things currently work. It won't be in 4.8.5 but possibly in the next release, especially if if there's lots of demand for this particular feature.
Re: support for XEM?
Great thanks!
I'd note that I don't think you'll have a lot of vocal requests, just because I think most people have no idea it exists.
But it would certainly help a lot of people with their problems with having to manually correct mismatches.
Demands there, people just aren't aware that there's a centralized solution to it.
I'd note that I don't think you'll have a lot of vocal requests, just because I think most people have no idea it exists.
But it would certainly help a lot of people with their problems with having to manually correct mismatches.
Demands there, people just aren't aware that there's a centralized solution to it.
Re: support for XEM?
I usually ask people to include file paths when reporting mismatches, so that should give some indication what kind of files are troublesome and if XEM could be a solution for those.
Re: support for XEM?
+1 for this feature
American Dad is an example:
Scene releases include all episodes up to 23 in the first season, TVDB splits first and second season in 7 episodes and the rest. XEN contains the mapping from scene to TVDB http://thexem.de/xem/show/4606
American Dad is an example:
Code: Select all
$ filebot -rename --action test American\ Dad\ -\ 108\ Bullocks\ to\ Stan.mkv --format '{plex}' --q 'American Dad' --filter 's == 1' --db TheTVDB
Rename episodes using [TheTVDB]
Fetching episode data for [American Dad!]
Apply filter [s == 1] on [263] items
Include [American Dad! - 1x01 - Pilot]
Include [American Dad! - 1x02 - Threat Levels]
Include [American Dad! - 1x03 - Stan Knows Best]
Include [American Dad! - 1x04 - Francine's Flashback]
Include [American Dad! - 1x05 - Roger Codger]
Include [American Dad! - 1x06 - Homeland Insecurity]
Include [American Dad! - 1x07 - Deacon Stan, Jesus Man]
Failed to match files to episode data
Failure (°_°)
I only work in black and sometimes very, very dark grey. (Batman)
Re: support for XEM?
Adding my +1 to this, I found out only today that sonarr uses thexem to map anime from anidb to their TVDB counterpart which I think would be a wonderful feature for filebot.
Re: support for XEM?
Please post the original file name, and the desired file name, so I can add that to my test data.
Re: support for XEM?
So I think a good example of what I'm talking about is a series like Monogatari:
https://www.thetvdb.com/series/monogatari/seasons/all
Where season 1 is known as Bakemonogatari, season 2 is Nisemonogatari etc, each season that is listed in TVDB is actually a completely seperate series so if you want the series named according TVDB metadata AMC just doens't really know what to do with a file called Bakemonogatari 01.mkv or Nisemonogatari 01.mkv (Monogatari S01E01 and S02E01 respectively).
Cross referencing anime on AniDB with thexem entries should help to solve that problem to rename the file to match TVDB metadata.
Have I effectively answered your request? When I made that comment I didn't have anything in mind in particular just the fact that aniDB and tvdb entries vary wildly and anime releases are more commonly named in such a way that makes matching to anidb more consistent than matching to TVDB.
https://www.thetvdb.com/series/monogatari/seasons/all
Where season 1 is known as Bakemonogatari, season 2 is Nisemonogatari etc, each season that is listed in TVDB is actually a completely seperate series so if you want the series named according TVDB metadata AMC just doens't really know what to do with a file called Bakemonogatari 01.mkv or Nisemonogatari 01.mkv (Monogatari S01E01 and S02E01 respectively).
Cross referencing anime on AniDB with thexem entries should help to solve that problem to rename the file to match TVDB metadata.
Have I effectively answered your request? When I made that comment I didn't have anything in mind in particular just the fact that aniDB and tvdb entries vary wildly and anime releases are more commonly named in such a way that makes matching to anidb more consistent than matching to TVDB.
Re: support for XEM?
Alright, we wanna rename this:
to this:
By first matching to AniDB as per usual, and then using XEM to find the corresponding TheTVDB entry.
Code: Select all
Nisemonogatari 01.mkv
Code: Select all
Monogatari S02E01.mkv
Re: support for XEM?
I'd like to also appropriate this thread for possible solution.
* How would XEM support look like on the CLI?
* How would it look like in the GUI?
* How would XEM support look like on the CLI?
* How would it look like in the GUI?
Re: support for XEM?
At first glance XEM doesn't seem to qualify as standalone DB as it links between them mainly:
AniDB::XEN::TheTVDB.
This could potentially solve point 2, 3 but likely not 1 and 4.
For number 4 the above notation would be used like AniDB::XEN to specify that the matching process should query XEN for all available names before renaming, however the response doesn't seem to return a preferred name or be filterable by show (it returns all names in the XEN database).
Still no ideas about point 1 which is the fundamental one (mapping a single file to a specific episode or multiple episodes).
- provide mapping between scene releases and databases
- provide source of truth between database for same show with different names according to season
- provide mapping between AniDB order and TheTVDB order
- provide name mapping between different shows
AniDB::XEN::TheTVDB.
This could potentially solve point 2, 3 but likely not 1 and 4.
For number 4 the above notation would be used like AniDB::XEN to specify that the matching process should query XEN for all available names before renaming, however the response doesn't seem to return a preferred name or be filterable by show (it returns all names in the XEN database).
Still no ideas about point 1 which is the fundamental one (mapping a single file to a specific episode or multiple episodes).
I only work in black and sometimes very, very dark grey. (Batman)
Re: support for XEM?
American Dad!
http://thexem.de/xem/show/4606
Update? =
But I made this so maybe you can use it:
Output:
http://thexem.de/xem/show/4606
Update? =
if so and not updated the day a new episode comes out it's USELESS2012-03-26 20:47:14 UTC / 2019-02-18 03:47:19 UTC
But I made this so maybe you can use it:
Code: Select all
{
def currentSeason = 16
def currentEpisode = 4
def tvdbSeason, tvdbEpisode
def id = 73141
def show = 'American Dad!'
def slurper = new groovy.json.JsonSlurper()
def url = "http://thexem.de/map/all?id=$id&origin=tvdb"
def data = any{slurper.parseText( new URL( url ).text ).data
.find{ (it.scene.season == currentSeason && it.scene.episode == currentEpisode) || (it.tvdb.season == currentSeason && it.tvdb.episode == currentEpisode)}
.collectEntries{println it; [ (it.key) : [ 'season' : it.getValue().season, 'episode' : it.getValue().episode ] ] }}{}
any{"$show - (TVDb - Season ${data.tvdb.season.pad(2)} Episode ${data.tvdb.episode.pad(2)}) vs (Scene - Season ${data.scene.season.pad(2)} Episode ${data.scene.episode.pad(2)})"}{'Wrong info'}
}
FYI: when scaped from tvdb just replace the first part to:American Dad! - (TVDb - Season 16 Episode 04) vs (Scene - Season 15 Episode 04)
Code: Select all
def currentSeason = s
def currentEpisode = e
def tvdbSeason, tvdbEpisode
def id = id
def show = n
Re: support for XEM?
Anime
you can do a lookup for "alias" e.g. Nisemonogatari
http://thexem.de/map/allNames?origin=anidb
and then
http://thexem.de/map/all?id=6327&origin=anidb
Saturday Night Live
http://thexem.de/xem/show/4294
it only has season 43 ? USELESS
you can do a lookup for "alias" e.g. Nisemonogatari
http://thexem.de/map/allNames?origin=anidb
and then
http://thexem.de/map/all?id=6327&origin=anidb
Saturday Night Live
http://thexem.de/xem/show/4294
it only has season 43 ? USELESS
Re: support for XEM?
1.
My idea is a new xem.[AniDB, TheTVDB, Scene, ...] binding, so that we can map the output via xem for naming, but use the original value for matching.
2.
Additionally, a new --xem option that'll let you re-map each original to a new episode, or multi-episode, or nothing at all, before matching, via Groovy code and existing bindings of course:
e.g. map to itself, and effectively do nothing
e.g. allow arbitrary Groovy code to come up with new Episode objects:
e.g. make the standard use case easy:
3.
A use case for matching with scene, but renaming with original TheTVDB data, might then look like this:
Or just this, depending on whatever we decide to pass the xem'ed episodes on to the formatter, or really just use them for matching, but reverse-map them to the original Episode object before formatting:
My idea is a new xem.[AniDB, TheTVDB, Scene, ...] binding, so that we can map the output via xem for naming, but use the original value for matching.
2.
Additionally, a new --xem option that'll let you re-map each original to a new episode, or multi-episode, or nothing at all, before matching, via Groovy code and existing bindings of course:
e.g. map to itself, and effectively do nothing
Code: Select all
--xem episode
Code: Select all
--xem "new Episode(/Name/, 1, 1, /Title/)"
Code: Select all
--xem "episode.derive(1, 1)"
Code: Select all
--xem "xem.scene"
3.
A use case for matching with scene, but renaming with original TheTVDB data, might then look like this:
Code: Select all
--xem "xem.scene" --format "{xem.thetvdb.plex}"
Code: Select all
--xem "xem.scene" --format "{plex}"
Re: support for XEM?
Is your posted example solution relying on thexem for matching in that case? That is to say if in your example the show you're matching to doesn't have an entry in thexem will it fail? Or will it fallback to TVDB/aniDB?
Because thexem only does mappings for shows where actually required so it won't be a complete catalogue like TVDB for example.
Just making sure you are aware of that aspect.
Because thexem only does mappings for shows where actually required so it won't be a complete catalogue like TVDB for example.
Just making sure you are aware of that aspect.
Re: support for XEM?
I assume the implementation of {xem} would just return the original episode object if it can't be mapped to a new one.
Re: support for XEM?
Yeah that sounds pretty much ideal.
Re: support for XEM?
How would I match Owarimonogatari (2017) - 01 to Monogatari - S05E01 using the available API methods?
I only get back numbers. But I also need to somehow match AniDB ID to TheTVDB ID / SEASON.
e.g.
http://thexem.de/map/all?id=13033&origin=anidb
http://thexem.de/map/single?id=13033&or ... absolute=1
How do I know which "Episode 1" belongs to series AniDB::13033 since I'm not sure how to make sense of the numbers.
I only get back numbers. But I also need to somehow match AniDB ID to TheTVDB ID / SEASON.
e.g.
http://thexem.de/map/all?id=13033&origin=anidb
http://thexem.de/map/single?id=13033&or ... absolute=1
How do I know which "Episode 1" belongs to series AniDB::13033 since I'm not sure how to make sense of the numbers.