Matching problem on TheTVDB

Any questions? Need some help?
kim
Power User
Posts: 1251
Joined: 15 May 2014, 16:17

Re: Matching problem on TheTVDB

Post by kim »

hmm...
1. Rating ?
"SeriesRating: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SeriesRating: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]"
but the do have
<Rating>7.9</Rating><RatingCount>15</RatingCount>
http://thetvdb.com/api/694FAD89942D3827 ... 521/en.xml

<Rating>7.9</Rating><RatingCount>9</RatingCount>
http://thetvdb.com/api/694FAD89942D3827 ... 210/en.xml

2. can i generate this data, how ?

3. is it not better to have and 100% name match get the win:
file: "Colony" but tvdb have "Colony" AND "The Colony" = "Colony" = 100% AND "The Colony" = e.g. 50%
this way is the users error not the program ?
and then all the other checks: do the show even have an season 2 or an ep. 6
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

1.
9 VS 15 ratings is not significantly different, and your "correct" match is the one with only 9 ratings. That definitely doesn't help you. :D

2.
Do you mean "How do I get detailed logging like this?" You can't, it's hard to interpret and won't help you fix whatever problem you seem to have.

3.
Sadly, FileBot thinks more like me, and I couldn't tell the "good" match from the "bad" match. "The" is not meaningful (most of the time). To me, "The Colony" and "Colony" is exactly the same thing. :D

"Apples VS Banana Apples" is not the same as "Colony VS The Colony". One is more different, one is less different.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

Less talk, more code. You don't need to understand why something is not working, you just need to know how to nudge it into the right direction. That's what --q and --filter are for.

You use --q when you know exactly what you want. You use --filter rules to add same vague hints about what you want, like "not this show", "only new shows", "only ongoing shows", etc.

Code: Select all

$ filebot -rename * --action test --q "Colony" --db TheTVDB
Fetching episode data for [Colony]
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Code: Select all

$ filebot -rename * --action test --q "The Colony" --db TheTVDB -non-strict
Fetching episode data for [The Colony]
Fetching episode data for [The Colony (Australia)]
[TEST] Rename [Colony.S01E06.mkv] to [The Colony - 1x06 - Loss and Communication.mkv]

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter "n != /The Colony/"
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {n != /The Colony/}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter "age < 30"
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {age < 30}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

AMC wrote:e.g. if you're a typical user, and find a simple age filter too restrictive, then this slightly augmented age filter rule might be the magic bullet that fixes all your problems:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }" --def "ut_label=TV"
--filter rules are extremely flexible so more complex scenarios, such as applying certain filters only for certain labels or files, is fully supported.

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter 'age < 7 || !model.any{ it.age < 7 }'
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {age < 7 || !model.any{ it.age < 7 }}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

In this case, FileBot is too smart for it's own good. The same logic that makes filebot work better than Kodi/Plex/etc most of the time, makes it fail in this particular case. However, FileBot is self-healing due to the popularity score, so this problem will fix itself after a while.
:idea: Please read the FAQ and How to Request Help.
kim
Power User
Posts: 1251
Joined: 15 May 2014, 16:17

Re: Matching problem on TheTVDB

Post by kim »

How do you get a higher/lower score ?
FilePath: 0.2154: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
FilePath: 0.2308: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
The filter is ok if i get 1-2 wrong (if i even know is wrong show), but if i have like e.g. 20 shows with lookup problem what then ?

What I want is a way to get 100% correct all the time and no need to use filter... if I can help with testing, i will and im sure many on forum will too?

how do I help fix problem?
I tried to give low/high rating on thetvdb but one more vote did not change rating

btw: I agree with you on:
""Apples VS Banana Apples" is not the same as "Colony VS The Colony". One is more different, one is less different."

but that does not mean you/it should ignore "the"... just give "the" a better/less score/chance if the folder and/or filename have or don't have "the" in it.... the "the" must be there for a reason
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

What I want is a way to get 100% correct all the time and no need to use filter...
If that was possible, the --filter option wouldn't be there. :lol: The --filter option is like solving the Gordian Knot with a sword, making an impossible problem solvable given certain restrictions.

e.g. 20 shows with lookup problem what then?
So far we have 1 very special lookup problem here. I'm sure you can find another, but it's gonna be very special in a very different way.

but that does not mean you/it should ignore "the"... just give "the" a better/less score/chance if the folder and/or filename have or don't have "the" in it.... the "the" must be there for a reason
There will be at least 1-2 very special cases where this exact logic will give you the wrong match.


I've adjusted the the popularity metric with r3553 to give a higher score to shows that have higher ratings-per-day ratios. In this case it'll definitely give you the match you want, and in general this match should also be the more likely one.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

Just use this filter and see what happens:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }"
I have yet to find a case or situation where this doesn't just work.
:idea: Please read the FAQ and How to Request Help.
kim
Power User
Posts: 1251
Joined: 15 May 2014, 16:17

Re: Matching problem on TheTVDB

Post by kim »

Well that did it, thx :D
not sure what you mean by "higher ratings-per-day ratios" ?
i just need to know that i can do if (when) i get problem again ?
give high rating i guess, but the series or the episode ?
I've adjusted the the popularity metric with r3553 to give a higher score to shows that have higher ratings-per-day ratios. In this case it'll definitely give you the match you want, and in general this match should also be the more likely one.
btw: I was thinking if filebot could check for <Status>Continuing</Status> or <Status>Ended</Status> to in the "the popularity metric"... prefer "Continuing" over "Ended" ?
because i think most ppl scrape like this:
"Continuing" = 1 episode a week
"Ended" = 1 or all seasons (and when multiple episodes in one go, filebot can look at them as a whole = better match)
it can also be used like the age filter:
--filter "Status" = "Continuing" or "Ended"
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

1.
I've added a new score, the "(number of ratings / days since startDate) * rating" score. Good for old shows with many ratings, and new shows with some ratings. Bad for old shows with few ratings, or new shows with no ratings.


2.
You can give me the paths. Sometimes it can be fixed. Sometimes I may choose not to fix certain corner cases when I think that a fix could cause more harm than good.


3.
The VoteRate/Timestamp metrics should prioritize newer shows to a certain degree. But assuming that Continuing IS BETTER THAN Ended is not an assumption you can just make for everyone.

However, these small hints will give you virtually perfect matching assuming that you only process "Continuing" shows:

Code: Select all

--filter "info.status == /Continuing/"
If you put in a little bit of manual effort setting labels you can pass the Continuing/Ended status you want via the label:

Code: Select all

--filter "label.contains(info.status)"
If you download the latest episode within 7 days, then this will always work. This will even work if the SxE numbers are wrong because there should only be one single correct match:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }"
The only downside is that you probably can't process previous episodes/seasons of shows that are currently airing (i.e. where an episode aired within the last 7 days).

The logic applies a "must be newer than 7 days" logic, but only if there are any episodes that were aired within 7 days, and otherwise just accept all episode, so it works as if there's no filter so you can process older shows and episodes as well. This logic is probably superior (or at least less manual effort) than using Continuing/Ended for eliminating misleading matches.


4.
Your idea could be written like this. In this case we assume that the torrent title contains "Season N" in the title, and just pass everything through if that's the case. Otherwise we assume it's a single episode and apply the Continuing filter:

Code: Select all

--filter "defines.ut_title =~ /Season.[0-9]+/ ? true : info.status == /Continuing/"

Code: Select all

--filter "defines.ut_dir.toFile().getFiles{ it.isVideo() }.size() > 1 ? true : info.status == /Continuing/"
Just need to find a way to figure out if it was a single episode torrent, and if yes, apply an info.status == /Continuing/ filter.

You can't access any of the runtime data of the amc script, but all the information that is passed in from the command-line is available in the format and/or filter.
:idea: Please read the FAQ and How to Request Help.
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

Hi,

after the update from FileBot 4.6.1 (r3596) to FileBot 4.6.1 (r3655) I found this new matching problem.

FileBot 4.6.1 (r3596): The episode "Man Seeking Woman - 02 - 10 - Balloon" was matched using the list command with filter
FileBot 4.6.1 (r3655): The episode "Man Seeking Woman - 02 - 10 - Balloon" was not matched using the list command with filter

So I rolled back. Please check this issue.

Kind regards,

Sheldon
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

Looks good to me:

Code: Select all

Input: Man Seeking Woman - 02 - 10 - Balloon.mp4
Man Seeking Woman - 02 - 10 - Balloon.mp4 [series: man seeking woman, movie: The Man (2005)]
Exclude Movie: The Man (2005)
Group: [tvs:man seeking woman, mov:null, anime:null] => [Man Seeking Woman - 02 - 10 - Balloon.mp4]
Rename episodes using [TheTVDB]
Auto-detected query: [man seeking woman]
Fetching episode data for [Man Seeking Woman]
[COPY] Rename [Man Seeking Woman - 02 - 10 - Balloon.mp4] to [TV Shows/Man Seeking Woman/Season 02/Man Seeking Woman - S02E10 - Balloon.mp4]
:idea: Please read the FAQ and How to Request Help.
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

Not for me. The list command fails.
And due to the fact that it works fine with r3596 the new release r3655 must be the culprit.

The same is true for the new series "Undergroud". If I try to list season 1 episdo 1 in english the command fails with r3655.

This is the parametrized CLI batch call I use to get the episode.

java -Dunixfs=false -DuseExtendedFileAttributes=true -DuseCreationDate=false -Djava.net.useSystemProxies=false -Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=60000 -Djna.nosys=true -Dapplication.deployment=portable -Dapplication.analytics=true "-Dapplication.dir=D:\Portable Applications\Platform\PortableApps\FileBot" "-Duser.home=D:\Portable Applications\Platform\PortableApps\FileBot" "-Djava.io.tmpdir=D:\Portable Applications\Platform\PortableApps\FileBot\Temp" "-Djna.library.path=D:\Portable Applications\Platform\PortableApps\FileBot\Lib" "-Djava.library.path=D:\Portable Applications\Platform\PortableApps\Filebot\Lib" -Dnet.filebot.AcoustID.fpcalc="D:\Portable Applications\Platform\PortableApps\FileBot\Lib\fpcalc.exe" -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory "-Dnet.filebot.util.prefs.file=D:\Portable Applications\Platform\PortableApps\FileBot\prefs.properties" -jar "D:\Portable Applications\Platform\PortableApps\FileBot\FileBot.jar" -list --db %2 --q %3 --lang %4 --filter %5 --format %6 > %7 %*
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

What's the (not parameterized) command you're running ? What's the output?
:idea: Please read the FAQ and How to Request Help.
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

The command is:

-list - db "thetvdb " -q "Man Seeking Woman - lang "en" --filter "s>=2 && s<=2 && e>=10 && e <= 10 " --format "{n} - {s.pad(2)} - {e.pad(2)} - {t}"

The result (r3655) is:

Using persistent disk cache D:\Portable Applications\Platform\PortableApps\FileBot\cache\0
Create cache: thetvdb_search_en_3
Create cache: thetvdb_1
Create cache: data_0
Create cache: thetvdb_data_0_en_3
Man Seeking Woman - 02 - 10 - Balloon

The expected and most recent result (r3596) is:

Man Seeking Woman - 02 - 10 - Balloon
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

What's the difference? (other than the logging which you can disable with --log OFF)

Code: Select all

r3655 => Man Seeking Woman - 02 - 10 - Balloon
r3596 => Man Seeking Woman - 02 - 10 - Balloon
:idea: Please read the FAQ and How to Request Help.
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

Thanks a lot!!!
The option --log OFF solves the problem. It wasn't needed until now. This seems to be new since r3655.
The problem was that the result is redirected to a text file from that I fetch the first line.
If the log is present in that file the matching fails. Now it's fine again. :-)
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

Hi,

with r3725 I recognized another small issue. The list command doesn't get

Code: Select all

Um Himmels Willen - 15 - 13 - Geschenk des Himmels
at all. I added the data at TheTVDB so I know it's present there.
Please have a look at this.

Kind regards,

Sheldon
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

Works for me:

Code: Select all

$ filebot -list --lang de --q "Um Himmels Willen" | grep "Geschenk des Himmels"
Um Himmels Willen - 15x13 - Geschenk des Himmels
If you add information, then it'll take a while until you'll be able to see it via the API with filebot due server- and client-side caching.
:idea: Please read the FAQ and How to Request Help.
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

I see! Now I have it in my list. Thanks a lot!
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

Hi,

I recognized a new matching problem with r3995.

The shows

After-the-Black-S02E03
After-the-Black-S02E04
After-the-Black-S02E05

are matched to

After-the-Cash-S02E03
After-the-Cash-S02E04
After-the-Cash-S02E05.

One reason migtht be that at TheTVDB the current season 2 is wrongly defined as season 1..
But Filebot should not match to the wrong series.

Kind regards,

Sheldon
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

In -non-strict mode, especially with if there is no "correct" match to begin with, there's no guarantees about anything. Looking at what it came up with as its "best guess", on first glance it looks pretty reasonable to me, even though it's wrong. :lol:
:idea: Please read the FAQ and How to Request Help.
User avatar
Sheldon
Posts: 188
Joined: 14 Apr 2014, 14:39

Re: Matching problem on TheTVDB

Post by Sheldon »

Sorry, but no! Whatever logic you apply the only correct result in this case is no result. Everything else is wrong.
This is not a matter of taste but a matter of logic.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Matching problem on TheTVDB

Post by rednoah »

The whole point of -non-strict is that it will always find the "best" match. If you want a "confirmed correct" match you must not use -non-strict.

Most likely, the logic that fails here, is the exact same logic that allows you to get good matches for really bad filenames. So any changes here will do more harm than good IMHO.
:idea: Please read the FAQ and How to Request Help.
Post Reply