Matching problem on TheTVDB

Sheldon · Post by **Sheldon** » 08 Feb 2016, 11:36

Hi,

if I send a search request for

series: Colony
season: 1
episode: 1
language: en

to TheTVDB via FileBot CLI I receive the episode info for

series: The Colony
season: 1
episode: 1.
anguage: en

Both series exist at TheTVDB but if I send the exact series name I should get the right one.
How can I automatically match the right series?

Kind regards,

Sheldon

Post by **rednoah** » 08 Feb 2016, 17:10

1.
Search for the colony works as expected:

Code: Select all

filebot -list --db thetvdb --q "the colony"

Code: Select all

The Colony - 1x01 - Arrival and Survival
The Colony - 1x02 - Power Struggle
The Colony - 1x03 - Comfort in Chaos
…

2.
You probably mean to say Some file did not get matched correctly! to which I say What's the output?

FAQ wrote:Include file paths as text when reporting mismatches

@see viewtopic.php?f=6&t=1868#p10869

Sheldon · Post by **Sheldon** » 08 Feb 2016, 17:27

There is a misunderstanding. The series "The Colony" and "Colony" are 2 different series.
If I search for "Colony" I get the data for "The Colony".

Post by **rednoah** » 08 Feb 2016, 17:28

No worries, I got your drift, I tried that as well:

Code: Select all

filebot -list --db thetvdb --q "colony"

Code: Select all

Colony - 1x01 - Pilot
Colony - 1x02 - A Brave New World
Colony - 1x03 - 98 Seconds
…

Sheldon · Post by **Sheldon** » 08 Feb 2016, 17:35

I do the same and receive the data for "The Colony".
I use the jar version 3.6.1 (r3508).

Post by **rednoah** » 08 Feb 2016, 19:00

Alright, I can confirm that it doesn't work with the latest revision. Thanks for reporting. I'll have a look at it.

Post by **rednoah** » 08 Feb 2016, 19:15

Is this merely an issue of calling filebot -list --db thetvdb --q "colony" or do you actually get any Colony files mismatched as The Colony when renaming?

Sheldon · Post by **Sheldon** » 08 Feb 2016, 19:39

It's about using the list command including language, filter and format option.

Post by **rednoah** » 08 Feb 2016, 22:40

Please try r3516.

Sheldon · Post by **Sheldon** » 09 Feb 2016, 07:37

I tested r3516 and now I receive the correct data for the series "Colony". Thanks for fixing it!

But I recognized another similar issue. At TheTVDB you find the following 4 series

The Odd Couple - English
Männerwirtschaft - German (german version of The Odd Couple - English)
The Odd Couple (2015) - English
Odd Couple - German (german version of The Odd Couple (2015) - English)

If I search for episodes for series Odd Couple - German I receive the data for Männerwirtschaft - German. That's wrong
But if I search for episodes for series The Odd Couple (2015) - German I receive the data for Odd Couple - German. That's correct.

Post by **rednoah** » 09 Feb 2016, 09:38

Try r3520.

Sheldon · Post by **Sheldon** » 09 Feb 2016, 10:55

Great! Now I receive the data for series "Odd Couple - German" by both searches
"Odd Couple - German" and "The Odd Couple (2015) - German". That's perfect!
I also cross-checked the series "Colony" again and it works fine.
So thanks a lot for fixing the issues!

Post by **kim** » 21 Feb 2016, 16:17

Somewhere between version FileBot 4.6.1 (r3436) and FileBot 4.6.1 (r3541) something is wrong:

CLI: AMC and show "Colony"
r3436:
in strict mode Colony --> lookup yes and correct

r3541:
in strict mode Colony --> lookup NO
in NON-strict mode Colony --> lookup yes, BUT --> "The Colony" (WRONG show)

Sheldon · Post by **Sheldon** » 21 Feb 2016, 17:14

I cannot copy this.
For FileBot CLI (r3541) strict I get
Colony -> Colony (correct matching)

Post by **kim** » 21 Feb 2016, 17:48

Are you using filter ?
I'm not

FileBot 4.6.1 (r3541) / Java(TM) SE Runtime Environment 1.8.0_73

-script fn:amc ... --log-file amc.log --action move --def unsorted=y --conflict auto --encoding UTF-8 --def artwork=y --def clean=y --def skipExtract=y

Rename episodes using [TheTVDB]
Auto-detected query: [Colony]
CmdlineException: Multiple options: Force auto-select requires non-strict matching: [The Colony, Colony, The Colony (Australia), Roanoke: Search for the Lost Colony]

Sheldon · Post by **Sheldon** » 21 Feb 2016, 18:01

Yes, I use a filter to find the corresponding episode.

Post by **kim** » 21 Feb 2016, 18:25

test with Colony.S01E06 as the Colony.S01E05 and down looks ok in NON-strict only, NOT strict as worked in r3436 with NO filter

As you can see thetvdb has the data "Colony - 1x06 - Yoknapatawpha" so why any diff. from 1x05 ?

FileBot.exe -list --db thetvdb --q "Colony"
Colony - 1x01 - Pilot
Colony - 1x02 - A Brave New World
Colony - 1x03 - 98 Seconds
Colony - 1x04 - Blind Spot
Colony - 1x05 - Geronimo
Colony - 1x06 - Yoknapatawpha
Colony - 1x07 - Broussard
Colony - 1x08 - From the Cold
Colony - 1x09 - Zero Day
Colony - 1x10 - Gateway
Colony - 0xSpecial 1 - Behind the Wall

Code: Select all

http://thetvdb.com/api/GetSeries.php?seriesname=Colony&language=en
http://thetvdb.com/api/694FAD89942D3827/series/105521/all/en.xml
http://thetvdb.com/api/694FAD89942D3827/series/284210/all/en.xml
http://thetvdb.com/api/694FAD89942D3827/series/142011/all/en.xml
http://thetvdb.com/api/GetSeries.php?seriesname=The+Colony&language=en
http://thetvdb.com/api/694FAD89942D3827/series/105521/banners.xml

Post by **rednoah** » 22 Feb 2016, 04:11

1.
Different people may have different cache states. If you want reliable test results you may need to run -clear-cache once first.

2.
I will not investigate these issues unless you give me a file path that I can use for testing.

Looks legit:

Code: Select all

Group: [tvs:colony] => [Colony.S01E06.mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Colony]
Fetching episode data for [The Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony (Australia)]
[TEST] Rename [Colony.S01E06.mkv] to [TV Shows/Colony/Season 01/Colony - S01E06 - Loss and Communication.mkv]
Processed 1 files

Sheldon · Post by **Sheldon** » 22 Feb 2016, 08:32

I use the list command and not the rename command. Maybe that's a hint.

Post by **kim** » 22 Feb 2016, 15:09

This is from the WRONG show --> "THE Colony" and not just "Colony"

Colony - S01E06 - Loss and Communication

the correct is

Colony - 1x06 - Yoknapatawpha

file path dont matter... just "Colony.S01E06.mkv" or Colony/Colony.S01E06.mkv = wrong show

btw: i use " -clear-cache" every time i TEST

Sheldon · Post by **Sheldon** » 22 Feb 2016, 15:20

So maybe the 2 commands (list and rename) lead to different results.
Kim is right. The result should be "Colony - 1x06 - Yoknapatawpha".
I get this using the list command.

Post by **rednoah** » 22 Feb 2016, 17:29

Something changed... seems to be working as expected now...

Code: Select all

Group: [tvs:colony] => [Colony.S01E06.mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [colony]
Fetching episode data for [The Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony (Australia)]
[TEST] Rename [Colony.S01E06.mkv] to [TV Shows/Colony/Season 01/Colony - S01E06 - Yoknapatawpha.mkv]

EDIT:

I think I may have used --lang de in the previous test... This one is using --lang en.

Post by **kim** » 22 Feb 2016, 18:11

I'm sorry the PATH do matter:

you see i use "unsorted=y" first in strict mode so the path get to be
"D:\_DoWnLoaD\Done\Unsorted\Colony.S01E06.mp4"

and this gets this WRONG show...
and if you dont see this then PLEASE tell me how you test it ?

Post by **rednoah** » 22 Feb 2016, 18:51

Yes, FilePath matters, but not in any meaningful way. Might as well throw a coin.

Code: Select all

EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x01 - Arrival and Survival]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x02 - Power Struggle]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x03 - Comfort in Chaos]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x04 - Safety and Security]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x05 - A Stranger Among Us]
EpisodeFunnel: 1.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x07 - A Test of Faith]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x08 - Recon Mission]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x09 - Breaking Point]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 1x10 - Exodus]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x01 - New Beginning]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x02 - After the Fall]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x03 - Trust]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x04 - To Have and Have Not]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x05 - The Abduction]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony - 2x06 - Skout]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x07 - Tick]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x08 - Defense]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x09 - The Virus]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, The Colony - 2x10 - Don't Look Back]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x01 - Pilot]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x02 - A Brave New World]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x03 - 98 Seconds]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x04 - Blind Spot]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x05 - Geronimo]
EpisodeFunnel: 1.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x07 - Broussard]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x08 - From the Cold]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x09 - Zero Day]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, Colony - 1x10 - Gateway]
EpisodeFunnel: -1.0000: [Colony.S01E06.mkv, Colony - 0xSpecial 1 - Behind the Wall]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony (Australia) - 1x01 - When Worlds Collide]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony (Australia) - 1x02 - Survival]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony (Australia) - 1x03 - This Land is Ours]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony (Australia) - 1x04 - Rebellion]
EpisodeFunnel: 0.5000: [Colony.S01E06.mkv, The Colony (Australia) - 1x05 - After the Storm]
EpisodeFunnel: 1.0000: [Colony.S01E06.mkv, The Colony (Australia) - 1x06 - Back to Reality]
EpisodeBalancer: 1.1000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
EpisodeBalancer: 1.1000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
EpisodeBalancer: 0.1000: [Colony.S01E06.mkv, The Colony (Australia) - 1x06 - Back to Reality]
AirDate: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
AirDate: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
MetaAttributes: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
MetaAttributes: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
SubstringFields: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SubstringFields: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
SeriesNameBalancer: 1.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SeriesNameBalancer: 1.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
SeriesName: 1.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SeriesName: 1.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
RegionHint: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
RegionHint: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
Numeric: 1.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
Numeric: 1.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
NumericSequence: 1.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
NumericSequence: 1.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
SeriesRating: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SeriesRating: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
TimeStamp: 1.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
TimeStamp: 1.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
FilePathBalancer: 0.2500: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
FilePathBalancer: 0.2500: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]
FilePath: 0.2154: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
FilePath: 0.2308: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]

Same show name, both fairly new, neither series particularly more popular than the other ... just use a --filter to get rid of "The Colony", or wait until lots of people give "Colony" lots of high ratings.

Sheldon · Post by **Sheldon** » 22 Feb 2016, 19:04

But that's confusing me a bit. If he sends the correct series "Colony" he should never get episodes for "The Colony" if you simply match the series names strictly. It looks like you do some kind of partial matching here. Is it?

Post by **kim** » 22 Feb 2016, 19:13

hmm...
1. Rating ?
"SeriesRating: 0.0000: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
SeriesRating: 0.0000: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]"
but the do have
<Rating>7.9</Rating><RatingCount>15</RatingCount>
http://thetvdb.com/api/694FAD89942D3827 ... 521/en.xml

<Rating>7.9</Rating><RatingCount>9</RatingCount>
http://thetvdb.com/api/694FAD89942D3827 ... 210/en.xml

2. can i generate this data, how ?

3. is it not better to have and 100% name match get the win:
file: "Colony" but tvdb have "Colony" AND "The Colony" = "Colony" = 100% AND "The Colony" = e.g. 50%
this way is the users error not the program ?
and then all the other checks: do the show even have an season 2 or an ep. 6

Post by **rednoah** » 22 Feb 2016, 19:56

1.
9 VS 15 ratings is not significantly different, and your "correct" match is the one with only 9 ratings. That definitely doesn't help you.

2.
Do you mean "How do I get detailed logging like this?" You can't, it's hard to interpret and won't help you fix whatever problem you seem to have.

3.
Sadly, FileBot thinks more like me, and I couldn't tell the "good" match from the "bad" match. "The" is not meaningful (most of the time). To me, "The Colony" and "Colony" is exactly the same thing.

"Apples VS Banana Apples" is not the same as "Colony VS The Colony". One is more different, one is less different.

Post by **rednoah** » 22 Feb 2016, 20:09

Less talk, more code. You don't need to understand why something is not working, you just need to know how to nudge it into the right direction. That's what --q and --filter are for.

You use --q when you know exactly what you want. You use --filter rules to add same vague hints about what you want, like "not this show", "only new shows", "only ongoing shows", etc.

Code: Select all

$ filebot -rename * --action test --q "Colony" --db TheTVDB
Fetching episode data for [Colony]
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Code: Select all

$ filebot -rename * --action test --q "The Colony" --db TheTVDB -non-strict
Fetching episode data for [The Colony]
Fetching episode data for [The Colony (Australia)]
[TEST] Rename [Colony.S01E06.mkv] to [The Colony - 1x06 - Loss and Communication.mkv]

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter "n != /The Colony/"
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {n != /The Colony/}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter "age < 30"
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {age < 30}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Post by **rednoah** » 22 Feb 2016, 20:28

AMC wrote:e.g. if you're a typical user, and find a simple age filter too restrictive, then this slightly augmented age filter rule might be the magic bullet that fixes all your problems:
Code: Select all
--filter "age < 7 || !model.any{ it.age < 7 }" --def "ut_label=TV"
--filter rules are extremely flexible so more complex scenarios, such as applying certain filters only for certain labels or files, is fully supported.

Code: Select all

$ filebot -rename * --action test --db TheTVDB -non-strict --filter 'age < 7 || !model.any{ it.age < 7 }'
Auto-detected query: [Colony]
Fetching episode data for [Colony]
Fetching episode data for [The Colony]
Apply Filter: {age < 7 || !model.any{ it.age < 7 }}
[TEST] Rename [Colony.S01E06.mkv] to [Colony - 1x06 - Yoknapatawpha.mkv]

Post by **kim** » 22 Feb 2016, 20:29

This is how Kodi get info i hope you can use it to make filebot better or just as good as KODI to get the correct info:

http://thetvdb.com/api/GetSeries.php?se ... anguage=en
http://thetvdb.com/api/1D62F2F90030C444 ... all/en.zip
http://thetvdb.com/banners/graphical/284210-g2.jpg
http://thetvdb.com/banners/fanart/original/284210-2.jpg
http://thetvdb.com/banners/posters/284210-2.jpg
http://thetvdb.com/banners/episodes/284210/5463227.jpg

http://thetvdb.com/api/GetSeries.php?se ... anguage=en
http://thetvdb.com/api/1D62F2F90030C444 ... all/en.zip
http://thetvdb.com/banners/graphical/105521-g4.jpg
http://thetvdb.com/banners/fanart/original/105521-3.jpg
http://thetvdb.com/banners/posters/105521-2.jpg
http://thetvdb.com/banners/episodes/105521/1028861.jpg

Post by **rednoah** » 22 Feb 2016, 20:34

In this case, FileBot is too smart for it's own good. The same logic that makes filebot work better than Kodi/Plex/etc most of the time, makes it fail in this particular case. However, FileBot is self-healing due to the popularity score, so this problem will fix itself after a while.

Post by **kim** » 22 Feb 2016, 20:45

How do you get a higher/lower score ?

FilePath: 0.2154: [Colony.S01E06.mkv, The Colony - 1x06 - Loss and Communication]
FilePath: 0.2308: [Colony.S01E06.mkv, Colony - 1x06 - Yoknapatawpha]

The filter is ok if i get 1-2 wrong (if i even know is wrong show), but if i have like e.g. 20 shows with lookup problem what then ?

What I want is a way to get 100% correct all the time and no need to use filter... if I can help with testing, i will and im sure many on forum will too?

how do I help fix problem?
I tried to give low/high rating on thetvdb but one more vote did not change rating

btw: I agree with you on:
""Apples VS Banana Apples" is not the same as "Colony VS The Colony". One is more different, one is less different."

but that does not mean you/it should ignore "the"... just give "the" a better/less score/chance if the folder and/or filename have or don't have "the" in it.... the "the" must be there for a reason

Post by **rednoah** » 22 Feb 2016, 22:09

What I want is a way to get 100% correct all the time and no need to use filter...

If that was possible, the --filter option wouldn't be there.

The --filter option is like solving the Gordian Knot with a sword, making an impossible problem solvable given certain restrictions.

e.g. 20 shows with lookup problem what then?

So far we have 1 very special lookup problem here. I'm sure you can find another, but it's gonna be very special in a very different way.

but that does not mean you/it should ignore "the"... just give "the" a better/less score/chance if the folder and/or filename have or don't have "the" in it.... the "the" must be there for a reason

There will be at least 1-2 very special cases where this exact logic will give you the wrong match.

I've adjusted the the popularity metric with r3553 to give a higher score to shows that have higher ratings-per-day ratios. In this case it'll definitely give you the match you want, and in general this match should also be the more likely one.

Post by **rednoah** » 22 Feb 2016, 22:13

Just use this filter and see what happens:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }"

I have yet to find a case or situation where this doesn't just work.

Post by **kim** » 23 Feb 2016, 01:36

Well that did it, thx

not sure what you mean by "higher ratings-per-day ratios" ?
i just need to know that i can do if (when) i get problem again ?
give high rating i guess, but the series or the episode ?

I've adjusted the the popularity metric with r3553 to give a higher score to shows that have higher ratings-per-day ratios. In this case it'll definitely give you the match you want, and in general this match should also be the more likely one.

btw: I was thinking if filebot could check for <Status>Continuing</Status> or <Status>Ended</Status> to in the "the popularity metric"... prefer "Continuing" over "Ended" ?
because i think most ppl scrape like this:
"Continuing" = 1 episode a week
"Ended" = 1 or all seasons (and when multiple episodes in one go, filebot can look at them as a whole = better match)
it can also be used like the age filter:
--filter "Status" = "Continuing" or "Ended"

Post by **rednoah** » 23 Feb 2016, 06:19

1.
I've added a new score, the "(number of ratings / days since startDate) * rating" score. Good for old shows with many ratings, and new shows with some ratings. Bad for old shows with few ratings, or new shows with no ratings.

2.
You can give me the paths. Sometimes it can be fixed. Sometimes I may choose not to fix certain corner cases when I think that a fix could cause more harm than good.

3.
The VoteRate/Timestamp metrics should prioritize newer shows to a certain degree. But assuming that Continuing IS BETTER THAN Ended is not an assumption you can just make for everyone.

However, these small hints will give you virtually perfect matching assuming that you only process "Continuing" shows:

Code: Select all

--filter "info.status == /Continuing/"

If you put in a little bit of manual effort setting labels you can pass the Continuing/Ended status you want via the label:

Code: Select all

--filter "label.contains(info.status)"

If you download the latest episode within 7 days, then this will always work. This will even work if the SxE numbers are wrong because there should only be one single correct match:

Code: Select all

--filter "age < 7 || !model.any{ it.age < 7 }"

The only downside is that you probably can't process previous episodes/seasons of shows that are currently airing (i.e. where an episode aired within the last 7 days).

The logic applies a "must be newer than 7 days" logic, but only if there are any episodes that were aired within 7 days, and otherwise just accept all episode, so it works as if there's no filter so you can process older shows and episodes as well. This logic is probably superior (or at least less manual effort) than using Continuing/Ended for eliminating misleading matches.

4.
Your idea could be written like this. In this case we assume that the torrent title contains "Season N" in the title, and just pass everything through if that's the case. Otherwise we assume it's a single episode and apply the Continuing filter:

Code: Select all

--filter "defines.ut_title =~ /Season.[0-9]+/ ? true : info.status == /Continuing/"

Code: Select all

--filter "defines.ut_dir.toFile().getFiles{ it.isVideo() }.size() > 1 ? true : info.status == /Continuing/"

Just need to find a way to figure out if it was a single episode torrent, and if yes, apply an info.status == /Continuing/ filter.

You can't access any of the runtime data of the amc script, but all the information that is passed in from the command-line is available in the format and/or filter.

Sheldon · Post by **Sheldon** » 10 Mar 2016, 09:06

Hi,

after the update from FileBot 4.6.1 (r3596) to FileBot 4.6.1 (r3655) I found this new matching problem.

FileBot 4.6.1 (r3596): The episode "Man Seeking Woman - 02 - 10 - Balloon" was matched using the list command with filter
FileBot 4.6.1 (r3655): The episode "Man Seeking Woman - 02 - 10 - Balloon" was not matched using the list command with filter

So I rolled back. Please check this issue.

Kind regards,

Sheldon

Post by **rednoah** » 10 Mar 2016, 11:55

Looks good to me:

Code: Select all

Input: Man Seeking Woman - 02 - 10 - Balloon.mp4
Man Seeking Woman - 02 - 10 - Balloon.mp4 [series: man seeking woman, movie: The Man (2005)]
Exclude Movie: The Man (2005)
Group: [tvs:man seeking woman, mov:null, anime:null] => [Man Seeking Woman - 02 - 10 - Balloon.mp4]
Rename episodes using [TheTVDB]
Auto-detected query: [man seeking woman]
Fetching episode data for [Man Seeking Woman]
[COPY] Rename [Man Seeking Woman - 02 - 10 - Balloon.mp4] to [TV Shows/Man Seeking Woman/Season 02/Man Seeking Woman - S02E10 - Balloon.mp4]

Sheldon · Post by **Sheldon** » 10 Mar 2016, 17:16

Not for me. The list command fails.
And due to the fact that it works fine with r3596 the new release r3655 must be the culprit.

The same is true for the new series "Undergroud". If I try to list season 1 episdo 1 in english the command fails with r3655.

This is the parametrized CLI batch call I use to get the episode.

java -Dunixfs=false -DuseExtendedFileAttributes=true -DuseCreationDate=false -Djava.net.useSystemProxies=false -Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=60000 -Djna.nosys=true -Dapplication.deployment=portable -Dapplication.analytics=true "-Dapplication.dir=D:\Portable Applications\Platform\PortableApps\FileBot" "-Duser.home=D:\Portable Applications\Platform\PortableApps\FileBot" "-Djava.io.tmpdir=D:\Portable Applications\Platform\PortableApps\FileBot\Temp" "-Djna.library.path=D:\Portable Applications\Platform\PortableApps\FileBot\Lib" "-Djava.library.path=D:\Portable Applications\Platform\PortableApps\Filebot\Lib" -Dnet.filebot.AcoustID.fpcalc="D:\Portable Applications\Platform\PortableApps\FileBot\Lib\fpcalc.exe" -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory "-Dnet.filebot.util.prefs.file=D:\Portable Applications\Platform\PortableApps\FileBot\prefs.properties" -jar "D:\Portable Applications\Platform\PortableApps\FileBot\FileBot.jar" -list --db %2 --q %3 --lang %4 --filter %5 --format %6 > %7 %*

Post by **rednoah** » 10 Mar 2016, 20:04

What's the (not parameterized) command you're running ? What's the output?

Sheldon · Post by **Sheldon** » 10 Mar 2016, 20:37

The command is:

-list - db "thetvdb " -q "Man Seeking Woman - lang "en" --filter "s>=2 && s<=2 && e>=10 && e <= 10 " --format "{n} - {s.pad(2)} - {e.pad(2)} - {t}"

The result (r3655) is:

Using persistent disk cache D:\Portable Applications\Platform\PortableApps\FileBot\cache\0
Create cache: thetvdb_search_en_3
Create cache: thetvdb_1
Create cache: data_0
Create cache: thetvdb_data_0_en_3
Man Seeking Woman - 02 - 10 - Balloon

The expected and most recent result (r3596) is:

Man Seeking Woman - 02 - 10 - Balloon

Post by **rednoah** » 11 Mar 2016, 03:26

What's the difference? (other than the logging which you can disable with --log OFF)

Code: Select all

r3655 => Man Seeking Woman - 02 - 10 - Balloon
r3596 => Man Seeking Woman - 02 - 10 - Balloon

Sheldon · Post by **Sheldon** » 11 Mar 2016, 15:02

Thanks a lot!!!
The option --log OFF solves the problem. It wasn't needed until now. This seems to be new since r3655.
The problem was that the result is redirected to a text file from that I fetch the first line.
If the log is present in that file the matching fails. Now it's fine again.

Sheldon · Post by **Sheldon** » 19 Mar 2016, 14:29

Hi,

with r3725 I recognized another small issue. The list command doesn't get

Code: Select all

Um Himmels Willen - 15 - 13 - Geschenk des Himmels

at all. I added the data at TheTVDB so I know it's present there.
Please have a look at this.

Kind regards,

Sheldon

Post by **rednoah** » 19 Mar 2016, 22:01

Works for me:

Code: Select all

$ filebot -list --lang de --q "Um Himmels Willen" | grep "Geschenk des Himmels"
Um Himmels Willen - 15x13 - Geschenk des Himmels

If you add information, then it'll take a while until you'll be able to see it via the API with filebot due server- and client-side caching.

Sheldon · Post by **Sheldon** » 20 Mar 2016, 09:39

I see! Now I have it in my list. Thanks a lot!

Sheldon · Post by **Sheldon** » 17 May 2016, 16:06

Hi,

I recognized a new matching problem with r3995.

The shows

After-the-Black-S02E03
After-the-Black-S02E04
After-the-Black-S02E05

are matched to

After-the-Cash-S02E03
After-the-Cash-S02E04
After-the-Cash-S02E05.

One reason migtht be that at TheTVDB the current season 2 is wrongly defined as season 1..
But Filebot should not match to the wrong series.

Kind regards,

Sheldon

Post by **rednoah** » 17 May 2016, 19:27

In -non-strict mode, especially with if there is no "correct" match to begin with, there's no guarantees about anything. Looking at what it came up with as its "best guess", on first glance it looks pretty reasonable to me, even though it's wrong.

Sheldon · Post by **Sheldon** » 17 May 2016, 19:42

Sorry, but no! Whatever logic you apply the only correct result in this case is no result. Everything else is wrong.
This is not a matter of taste but a matter of logic.

Post by **rednoah** » 17 May 2016, 20:00

The whole point of -non-strict is that it will always find the "best" match. If you want a "confirmed correct" match you must not use -non-strict.

Most likely, the logic that fails here, is the exact same logic that allows you to get good matches for really bad filenames. So any changes here will do more harm than good IMHO.

Sheldon · Post by **Sheldon** » 17 May 2016, 20:08

But I didn't set the -non-strict option in my command.

Post by **rednoah** » 17 May 2016, 20:19

Well there, can you guess what my next question will be then?

Sheldon · Post by **Sheldon** » 17 May 2016, 20:33

Nope! You know my command already but here you are.

java -Dunixfs=false -DuseExtendedFileAttributes=true -DuseCreationDate=false -Djava.net.useSystemProxies=false -Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=60000 -Djna.nosys=true -Dapplication.deployment=portable -Dapplication.analytics=true "-Dapplication.dir=D:\Portable Applications\Platform\PortableApps\FileBot" "-Duser.home=D:\Portable Applications\Platform\PortableApps\FileBot" "-Djava.io.tmpdir=D:\Portable Applications\Platform\PortableApps\FileBot\Temp" "-Djna.library.path=D:\Portable Applications\Platform\PortableApps\FileBot\Lib" "-Djava.library.path=D:\Portable Applications\Platform\PortableApps\Filebot\Lib" -Dnet.filebot.AcoustID.fpcalc="D:\Portable Applications\Platform\PortableApps\FileBot\Lib\fpcalc.exe" -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory "-Dnet.filebot.util.prefs.file=D:\Portable Applications\Platform\PortableApps\FileBot\prefs.properties" -jar "D:\Portable Applications\Platform\PortableApps\FileBot\FileBot.jar" -list --db %2 --q %3 --lang %4 --filter %5 --format %6 --log OFF > %7 %*
)

Post by **rednoah** » 17 May 2016, 20:41

You know your command. I absolutely do not.

Actually, I still don't know the command that was executed since I don't know your %1..9 parameters. More importantly, I also don't know the output.

Sheldon · Post by **Sheldon** » 17 May 2016, 20:57

Yes, you do because we did this already several times. But anyway.

-list --db %2 --q %3 --lang %4 --filter %5 --format %6 --log OFF > %7 %*

%2 = thetvdb
%3 = After the Black
%4 = en
%5 = "s=2 && s<=2 && e>= 5 && e <= 5"
%6 = "{n} - {s.pad(2)} - {e.pad(2)} - {t}"
%7 = output file name

result = After the Catch - 02 - 05 - Real Life

Post by **rednoah** » 17 May 2016, 21:14

1.
If you're doing all of that, then you might as well add another n == /After the Black/ to your filter.

2.
How does a -list call result in badly renamed files?

Sheldon · Post by **Sheldon** » 17 May 2016, 21:23

I thought the series is set by the -q option. Do I have to add it to the filter as well? What is the difference?

The result of the list command is redirected to the output file.
The renaming is not done via Filebot. I fetch the name from the output file

Post by **rednoah** » 17 May 2016, 22:10

Code: Select all

$ filebot -list --q "After the Black" --db TheTVDB
After the Black - 1x01 - Episode 1
After the Black - 1x02 - Episode 2
After the Black - 1x03 - Episode 3
...

Sheldon · Post by **Sheldon** » 18 May 2016, 07:32

Yes, that's right! But I mentioned already that this data at TheTVDB is wrong.
After the Black is currently in season 2. They defined it to be season 1.
So you have to search for season 2 and their episode numbers (see my example) to get the wrong result (After the Catch).

Post by **kim** » 19 May 2016, 13:34

this show ?
http://thetvdb.com/?tab=series&id=309608&lid=7

it's not locked = make account and update it yourself

http://www.bbcamerica.com/shows/after-the-black

Sheldon · Post by **Sheldon** » 19 May 2016, 15:13

Yes, that's the right show. Thanks for your hint.
i know that the TheTVDB data can be modified. I did this already for other shows.
The problem is with FileBot. FileBot must not return wrong data in this case. But it does!

Post by **kim** » 19 May 2016, 21:33

looks like there is a problem with your filter?

Apply Filter: {s=2 && s<=2 && e>= 5 && e <= 5}

try: --filter "s==2 && s<=2 && e>= 5 && e <= 5"
or --filter "n != /After the Catch/"

filebot.exe -list --db thetvdb --q "After the Black" --filter "s=2 && s<=2 && e>= 5 && e <= 5" --format "{n} - {s.pad(2)} - {e.pad(2)} - {t}"
Using persistent disk cache C:\Program Files\FileBot\cache\1
Fetch resource: https://api.thetvdb.com/search/series?n ... +the+Black
Received 515 bytes
Rank [After the Black] => [After the Black, After the Catch, Project Runway, The First 48]
Rank [After the Black] => [After the Black, After the Catch]
Fetch resource: https://api.thetvdb.com/series/309608
Received 849 bytes
Fetch resource: https://api.thetvdb.com/series/309608/episodes?page=1
Received 3,626 bytes
Apply Filter: {s=2 && s<=2 && e>= 5 && e <= 5}
Include [After the Black - 1x05 - Episode 5]
After the Black - 01 - 05 - Episode 5

but I don't see the problem ?

Post by **rednoah** » 20 May 2016, 05:07

'=' is the assignment operator. '==' is the equals operator.

@see http://groovy-lang.org/operators.html#_ ... _operators

Sheldon · Post by **Sheldon** » 20 May 2016, 13:16

Sorry, my post was wrong.
I use s>=2 && s <=2 && e>=5 && e<= 5.
and this leads to "After the Catch."
That's the problem.

Post by **rednoah** » 20 May 2016, 15:44

It leads to no results because episode 2x05 doesn't exist:

Code: Select all

$ filebot -list --q "After the Black" --db TheTVDB --filter "s == 2 && e == 5"
Apply Filter: {s == 2 && e == 5}
$

Sheldon · Post by **Sheldon** » 20 May 2016, 15:56

It should lead to no result but it leads to the show "After the Catch". That's the issue!
I didn't change the data at TheTVDB so far to be able to show the bug to you.

Post by **rednoah** » 20 May 2016, 16:04

If I give you an executable command with full output that proofs that it yields no results, then you need to provide equally strong proof (command + console output) if you want to convince me that it works differently for you.

Sheldon · Post by **Sheldon** » 21 May 2016, 12:32

After some research i found the culprit that causes the problem and it's not FileBot in the first place.
So consider this to be solved! Sorry for bothering you!

Post by **rednoah** » 21 May 2016, 22:06

That's what I thought.

Sheldon · Post by **Sheldon** » 22 May 2016, 09:20

This is what happened. The command

Code: Select all


java -Dunixfs=false -DuseExtendedFileAttributes=true -DuseCreationDate=false -Djava.net.useSystemProxies=false -Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=60000 -Djna.nosys=true -Dapplication.deployment=portable -Dapplication.analytics=true "-Dapplication.dir=D:\Portable Applications\Platform\PortableApps\FileBot" "-Duser.home=D:\Portable Applications\Platform\PortableApps\FileBot" "-Djava.io.tmpdir=D:\Portable Applications\Platform\PortableApps\FileBot\Temp" "-Djna.library.path=D:\Portable Applications\Platform\PortableApps\FileBot\Lib" "-Djava.library.path=D:\Portable Applications\Platform\PortableApps\Filebot\Lib" -Dnet.filebot.AcoustID.fpcalc="D:\Portable Applications\Platform\PortableApps\FileBot\Lib\fpcalc.exe" -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory "-Dnet.filebot.util.prefs.file=D:\Portable Applications\Platform\PortableApps\FileBot\prefs.properties" -jar "D:\Portable Applications\Platform\PortableApps\FileBot\FileBot.jar" -list --db TheTVDB --q "After-the-Black"  --lang en --filter "s >= 2 && s <= 2 && e >= 5 && e <= 5" --format "{n} - {s.pad(2)} - {e.pad(2)} - {t}" >> test.txt

leads to this output

Code: Select all


Apply Filter: {s >= 2 && s <= 2 && e >= 5 && e <= 5}
Include [After the Catch - 2x05 - Real Life]
After the Catch - 02 - 05 - Real Life

This is some kind of bug because it returns a wrong result even if the input "After-the-Black" is incorrect.
There seems to be a problem with the dashes in the name.
Maybe this is of interest for you and you want to avoid such mismatches.

Post by **rednoah** » 22 May 2016, 10:13

I guess if you pass in a query than that's exactly what will be passed on to the database and then you get what you get.

For FileBot that's not an issue because internally the query will always be normalised before a search request is sent.

Sheldon · Post by **Sheldon** » 22 May 2016, 10:29

But probably this normalization doesn't take place if the request is sent from the command line like above.

Post by **rednoah** » 22 May 2016, 10:32

Of course not. FileBot will not mess with your query.

Sheldon · Post by **Sheldon** » 23 May 2016, 11:19

I guess you changed something because with r4020 the mismatch is gone. Great! Thanks!