Matching problem on TheTVDB

[kim]

hmm...
1. Rating ?
"SeriesRating: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SeriesRating: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]"
but the do have
<Rating>7.9</Rating><RatingCount>15</RatingCount>
http://thetvdb.com/api/694FAD89942D3827 ... 521/en.xml

<Rating>7.9</Rating><RatingCount>9</RatingCount>
http://thetvdb.com/api/694FAD89942D3827 ... 210/en.xml

2. can i generate this data, how ?

3. is it not better to have and 100% name match get the win:
file: "Colony" but tvdb have "Colony" AND "The Colony" = "Colony" = 100% AND "The Colony" = e.g. 50%
this way is the users error not the program ?
and then all the other checks: do the show even have an season 2 or an ep. 6

[rednoah]

1.
9 VS 15 ratings is not significantly different, and your "correct" match is the one with only 9 ratings. That definitely doesn't help you.

2.
Do you mean "How do I get detailed logging like this?" You can't, it's hard to interpret and won't help you fix whatever problem you seem to have.

3.
Sadly, FileBot thinks more like me, and I couldn't tell the "good" match from the "bad" match. "The" is not meaningful (most of the time). To me, "The Colony" and "Colony" is exactly the same thing.

"Apples VS Banana Apples" is not the same as "Colony VS The Colony". One is more different, one is less different.

[rednoah]

Less talk, more code. You don't need to understand why something is not working, you just need to know how to nudge it into the right direction. That's what --q and --filter are for.

You use --q when you know exactly what you want. You use --filter rules to add same vague hints about what you want, like "not this show", "only new shows", "only ongoing shows", etc.

Code: Select all

$ filebot -rename * --action test --q "Colony" --db TheTVDB
Fetching episode data for [Colony]
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Code: Select all

$ filebot -rename * --action test --q "The Colony" --db TheTVDB -non-strict
Fetching episode data for [The Colony]
Fetching episode data for [The Colony (Australia)]
[TEST] Rename [Colony.S01E06.mkv] to [The Colony - 1x06 - Loss and Communication.mkv]

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter "n != /The Colony/"
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {n != /The Colony/}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter "age < 30"
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {age < 30}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

[rednoah]

e.g. if you're a typical user, and find a simple age filter too restrictive, then this slightly augmented age filter rule might be the magic bullet that fixes all your problems:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }" --def "ut_label=TV"

--filter rules are extremely flexible so more complex scenarios, such as applying certain filters only for certain labels or files, is fully supported.

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter 'age < 7 || !model.any{ it.age < 7 }'
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {age < 7 || !model.any{ it.age < 7 }}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

[kim]

This is how Kodi get info i hope you can use it to make filebot better or just as good as KODI to get the correct info:

http://thetvdb.com/api/GetSeries.php?se ... anguage=en
http://thetvdb.com/api/1D62F2F90030C444 ... all/en.zip
http://thetvdb.com/banners/graphical/284210-g2.jpg
http://thetvdb.com/banners/fanart/original/284210-2.jpg
http://thetvdb.com/banners/posters/284210-2.jpg
http://thetvdb.com/banners/episodes/284210/5463227.jpg

http://thetvdb.com/api/GetSeries.php?se ... anguage=en
http://thetvdb.com/api/1D62F2F90030C444 ... all/en.zip
http://thetvdb.com/banners/graphical/105521-g4.jpg
http://thetvdb.com/banners/fanart/original/105521-3.jpg
http://thetvdb.com/banners/posters/105521-2.jpg
http://thetvdb.com/banners/episodes/105521/1028861.jpg

[rednoah]

In this case, FileBot is too smart for it's own good. The same logic that makes filebot work better than Kodi/Plex/etc most of the time, makes it fail in this particular case. However, FileBot is self-healing due to the popularity score, so this problem will fix itself after a while.

[kim]

How do you get a higher/lower score ?

FilePath: 0.2154: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
FilePath: 0.2308: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]

The filter is ok if i get 1-2 wrong (if i even know is wrong show), but if i have like e.g. 20 shows with lookup problem what then ?

What I want is a way to get 100% correct all the time and no need to use filter... if I can help with testing, i will and im sure many on forum will too?

how do I help fix problem?
I tried to give low/high rating on thetvdb but one more vote did not change rating

btw: I agree with you on:
""Apples VS Banana Apples" is not the same as "Colony VS The Colony". One is more different, one is less different."

but that does not mean you/it should ignore "the"... just give "the" a better/less score/chance if the folder and/or filename have or don't have "the" in it.... the "the" must be there for a reason

[rednoah]

What I want is a way to get 100% correct all the time and no need to use filter...

If that was possible, the --filter option wouldn't be there. The --filter option is like solving the Gordian Knot with a sword, making an impossible problem solvable given certain restrictions.

e.g. 20 shows with lookup problem what then?

So far we have 1 very special lookup problem here. I'm sure you can find another, but it's gonna be very special in a very different way.

but that does not mean you/it should ignore "the"... just give "the" a better/less score/chance if the folder and/or filename have or don't have "the" in it.... the "the" must be there for a reason

There will be at least 1-2 very special cases where this exact logic will give you the wrong match.


I've adjusted the the popularity metric with r3553 to give a higher score to shows that have higher ratings-per-day ratios. In this case it'll definitely give you the match you want, and in general this match should also be the more likely one.

[rednoah]

Just use this filter and see what happens:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }"

I have yet to find a case or situation where this doesn't just work.

[kim]

Well that did it, thx
not sure what you mean by "higher ratings-per-day ratios" ?
i just need to know that i can do if (when) i get problem again ?
give high rating i guess, but the series or the episode ?

I've adjusted the the popularity metric with r3553 to give a higher score to shows that have higher ratings-per-day ratios. In this case it'll definitely give you the match you want, and in general this match should also be the more likely one.

btw: I was thinking if filebot could check for <Status>Continuing</Status> or <Status>Ended</Status> to in the "the popularity metric"... prefer "Continuing" over "Ended" ?
because i think most ppl scrape like this:
"Continuing" = 1 episode a week
"Ended" = 1 or all seasons (and when multiple episodes in one go, filebot can look at them as a whole = better match)
it can also be used like the age filter:
--filter "Status" = "Continuing" or "Ended"

[rednoah]

1.
I've added a new score, the "(number of ratings / days since startDate) * rating" score. Good for old shows with many ratings, and new shows with some ratings. Bad for old shows with few ratings, or new shows with no ratings.


2.
You can give me the paths. Sometimes it can be fixed. Sometimes I may choose not to fix certain corner cases when I think that a fix could cause more harm than good.


3.
The VoteRate/Timestamp metrics should prioritize newer shows to a certain degree. But assuming that Continuing IS BETTER THAN Ended is not an assumption you can just make for everyone.

However, these small hints will give you virtually perfect matching assuming that you only process "Continuing" shows:

Code: Select all

--filter "info.status == /Continuing/"

If you put in a little bit of manual effort setting labels you can pass the Continuing/Ended status you want via the label:

Code: Select all

--filter "label.contains(info.status)"

If you download the latest episode within 7 days, then this will always work. This will even work if the SxE numbers are wrong because there should only be one single correct match:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }"

The only downside is that you probably can't process previous episodes/seasons of shows that are currently airing (i.e. where an episode aired within the last 7 days).

The logic applies a "must be newer than 7 days" logic, but only if there are any episodes that were aired within 7 days, and otherwise just accept all episode, so it works as if there's no filter so you can process older shows and episodes as well. This logic is probably superior (or at least less manual effort) than using Continuing/Ended for eliminating misleading matches.


4.
Your idea could be written like this. In this case we assume that the torrent title contains "Season N" in the title, and just pass everything through if that's the case. Otherwise we assume it's a single episode and apply the Continuing filter:

Code: Select all

--filter "defines.ut_title =~ /Season.[0-9]+/ ? true : info.status == /Continuing/"

Code: Select all

--filter "defines.ut_dir.toFile().getFiles{ it.isVideo() }.size() > 1 ? true : info.status == /Continuing/"

Just need to find a way to figure out if it was a single episode torrent, and if yes, apply an info.status == /Continuing/ filter.

You can't access any of the runtime data of the amc script, but all the information that is passed in from the command-line is available in the format and/or filter.

[Sheldon]

Hi,

after the update from FileBot 4.6.1 (r3596) to FileBot 4.6.1 (r3655) I found this new matching problem.

FileBot 4.6.1 (r3596): The episode "Man Seeking Woman - 02 - 10 - Balloon" was matched using the list command with filter
FileBot 4.6.1 (r3655): The episode "Man Seeking Woman - 02 - 10 - Balloon" was not matched using the list command with filter

So I rolled back. Please check this issue.

Kind regards,

Sheldon

[rednoah]

Looks good to me:

Code: Select all

Input: Man Seeking Woman - 02 - 10 - Balloon.mp4
Man Seeking Woman - 02 - 10 - Balloon.mp4 [series: man seeking woman, movie: The Man (2005)]
Exclude Movie: The Man (2005)
Group: [tvs:man seeking woman, mov:null, anime:null] => [Man Seeking Woman - 02 - 10 - Balloon.mp4]
Rename episodes using [TheTVDB]
Auto-detected query: [man seeking woman]
Fetching episode data for [Man Seeking Woman]
[COPY] Rename [Man Seeking Woman - 02 - 10 - Balloon.mp4] to [TV Shows/Man Seeking Woman/Season 02/Man Seeking Woman - S02E10 - Balloon.mp4]

[Sheldon]

Not for me. The list command fails.
And due to the fact that it works fine with r3596 the new release r3655 must be the culprit.

The same is true for the new series "Undergroud". If I try to list season 1 episdo 1 in english the command fails with r3655.

This is the parametrized CLI batch call I use to get the episode.

java -Dunixfs=false -DuseExtendedFileAttributes=true -DuseCreationDate=false -Djava.net.useSystemProxies=false -Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=60000 -Djna.nosys=true -Dapplication.deployment=portable -Dapplication.analytics=true "-Dapplication.dir=D:\Portable Applications\Platform\PortableApps\FileBot" "-Duser.home=D:\Portable Applications\Platform\PortableApps\FileBot" "-Djava.io.tmpdir=D:\Portable Applications\Platform\PortableApps\FileBot\Temp" "-Djna.library.path=D:\Portable Applications\Platform\PortableApps\FileBot\Lib" "-Djava.library.path=D:\Portable Applications\Platform\PortableApps\Filebot\Lib" -Dnet.filebot.AcoustID.fpcalc="D:\Portable Applications\Platform\PortableApps\FileBot\Lib\fpcalc.exe" -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory "-Dnet.filebot.util.prefs.file=D:\Portable Applications\Platform\PortableApps\FileBot\prefs.properties" -jar "D:\Portable Applications\Platform\PortableApps\FileBot\FileBot.jar" -list --db %2 --q %3 --lang %4 --filter %5 --format %6 > %7 %*

[rednoah]

What's the (not parameterized) command you're running ? What's the output?

[Sheldon]

The command is:

-list - db "thetvdb " -q "Man Seeking Woman - lang "en" --filter "s>=2 && s<=2 && e>=10 && e <= 10 " --format "{n} - {s.pad(2)} - {e.pad(2)} - {t}"

The result (r3655) is:

Using persistent disk cache D:\Portable Applications\Platform\PortableApps\FileBot\cache\0
Create cache: thetvdb_search_en_3
Create cache: thetvdb_1
Create cache: data_0
Create cache: thetvdb_data_0_en_3
Man Seeking Woman - 02 - 10 - Balloon

The expected and most recent result (r3596) is:

Man Seeking Woman - 02 - 10 - Balloon

[rednoah]

What's the difference? (other than the logging which you can disable with --log OFF)

Code: Select all

r3655 => Man Seeking Woman - 02 - 10 - Balloon
r3596 => Man Seeking Woman - 02 - 10 - Balloon

[Sheldon]

Thanks a lot!!!
The option --log OFF solves the problem. It wasn't needed until now. This seems to be new since r3655.
The problem was that the result is redirected to a text file from that I fetch the first line.
If the log is present in that file the matching fails. Now it's fine again.

[Sheldon]

Hi,

with r3725 I recognized another small issue. The list command doesn't get

Code: Select all

Um Himmels Willen - 15 - 13 - Geschenk des Himmels

at all. I added the data at TheTVDB so I know it's present there.
Please have a look at this.

Kind regards,

Sheldon

[rednoah]

Works for me:

Code: Select all

$ filebot -list --lang de --q "Um Himmels Willen" | grep "Geschenk des Himmels"
Um Himmels Willen - 15x13 - Geschenk des Himmels

If you add information, then it'll take a while until you'll be able to see it via the API with filebot due server- and client-side caching.

[Sheldon]

I see! Now I have it in my list. Thanks a lot!

[Sheldon]

Hi,

I recognized a new matching problem with r3995.

The shows

After-the-Black-S02E03
After-the-Black-S02E04
After-the-Black-S02E05

are matched to

After-the-Cash-S02E03
After-the-Cash-S02E04
After-the-Cash-S02E05.

One reason migtht be that at TheTVDB the current season 2 is wrongly defined as season 1..
But Filebot should not match to the wrong series.

Kind regards,

Sheldon

[rednoah]

In -non-strict mode, especially with if there is no "correct" match to begin with, there's no guarantees about anything. Looking at what it came up with as its "best guess", on first glance it looks pretty reasonable to me, even though it's wrong.

[Sheldon]

Sorry, but no! Whatever logic you apply the only correct result in this case is no result. Everything else is wrong.
This is not a matter of taste but a matter of logic.

[rednoah]

The whole point of -non-strict is that it will always find the "best" match. If you want a "confirmed correct" match you must not use -non-strict.

Most likely, the logic that fails here, is the exact same logic that allows you to get good matches for really bad filenames. So any changes here will do more harm than good IMHO.