[Bug] Pattern matching worse since 4.7

All your suggestions, requests and ideas for future development
Post Reply
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

[Bug] Pattern matching worse since 4.7

Post by Hemloc »

Hi there,

Since upgrading to 4.7 I have found the pattern matching actually worse than before. So much so that I have gone back to 4.6.1 which does not have the same problems. There are several issues I have seen with the new pattern matching :

1. For some reason FileBot 4.7.5 cannot identify a series from a scene standard filename, e.g. Humans.S02E05.1080p.AMZN.WEBRip.DD2.0.x264-KiNGS_0.mkv. There is only one series on TheTVDB with this name, and yet FileBot fails to identify the name. It shouldn't since it confirms to scene standards. This works fine in 4.6.1.
2. FileBot should not use the directory for pattern matching if everything is already in the filename. An example is the name above, and yet FileBot always checks the directory name no matter what. It should only do this if it cannot find a series name from the filename.
3. There should be a way of switching off checking directory names completely. This is a new feature, and unwanted by several it seems, not just me. Using a blacklist is a very inefficient design when a simple directory name like "Humans - Season 2" confuse the program. As it is, I use temporary directory names that do not conform to any standard and asking you to add them to a blacklist seems silly. A simple switch would, I am sure, solve the problem for many users.

Regards
Andrew

PS : And now that I have downgraded, every time I open the program I am asked if I want to upgrade. Why is there no option to switch this off?

C:\Program Files\FileBot>filebot -script fn:sysinfo
FileBot 4.7.5 (r4600)
JNA Native: 4.0.1
MediaInfo: 0.7.88
7-Zip-JBinding: 9.20
Chromaprint: 1.1.0
Extended Attributes: OK
Script Bundle: 2016-11-26 (r465)
Groovy: 2.4.7
JRE: Java(TM) SE Runtime Environment 1.8.0_66
JVM: 32-bit Java HotSpot(TM) Client VM
CPU/MEM: 12 Core / 247 MB Max Memory / 11 MB Used Memory
OS: Windows 7 (x86)
Package: MSI
Data: C:\Users\Hemloc\AppData\Roaming\FileBot
uname: CYGWIN_NT-6.1 Hemloc-PC 1.7.28(0.271/5/3) 2014-02-09 21:06 x86_64 Cygwin
Done ?(?????)?
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
Scene "standards" include names such as TWD and GoT so they're pretty useless. Fortunately, there is people who add these mangled names as alias to the series record and that's why FileBot can deal with most of them.

I have no idea why this specific example doesn't work, will be doing some debugging. The full path of the file that doesn't work would be helpful.


EDIT:

Works for me. I need the full file path to run additional tests.

Code: Select all

$ filebot -rename Humans.S02E05.1080p.AMZN.WEBRip.DD2.0.x264-KiNGS_0.mkv --db TheTVDB -non-strict --action TEST
Rename episodes using [TheTVDB]
Auto-detected query: [Humans]
Fetching episode data for [Humans]
Fetching episode data for [Real Humans]
Fetching episode data for [Humans Mutants]
Fetching episode data for [Humans & Households]
[TEST] Rename [Humans.S02E05.1080p.AMZN.WEBRip.DD2.0.x264-KiNGS_0.mkv] to [Humans - 2x05 - Episode 5.mkv]
Processed 1 files

2.
FileBot is and has always been considering the entire file path (file name, folder name, folders folder name, etc) so nothing new here.

FileBot doesn't have settings, because it needs to work out of the box. FileBot will not search for folder names that are on the blacklist, so as long as you use folders such as "Complete", "Downloads", etc it'll just work.

If in some rare cases the folder name makes more sense to FileBot than the filename, then you can post issues here in the forums a things will be fixed, possibly by adding entries to the blacklist. Certainly inconvenient for the individual user, but important for making FileBot work out of the box for most people (especially the less capable ones).


PS: There is such an option: viewtopic.php?f=8&t=2112
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

1. Actually, it is not just that specific example, I had the same problem with Criminal.Minds.S12E07.Mirror.Image.PROPER.1080p.AMZN.WEBRip.DD5.1.x264-VISUM_0.mkv. I didn't bother to try the various other ones I had as it seemed they would all have a similar problem. I didn't give the path because I tried this in "L:\Downloads" and that is a blacklisted directory name so I didn't think it would make a difference.

2. Ok... but then why when I upgraded from 4.6.1 to 4.7 for files I have been renaming in the same directory "L:\Downloads\Newsleecher" for over a year, does it suddenly start complaining that it cannot identify "Newsleecher". Something must have changed.

3. I still believe there must be some better way of handling "strange" directory names. However, it seems I have three main directory names causing FileBot problems :
3.1. "L:\Downloads\Newsleecher"
3.2. I use a sub-directory called "L:\Downloads\Newsleecher\t" and the "t" also causes problems. I use "t" instead of "temp" for many things like this.
3.3. Finally I name my series names like "Humans - Season 2", and potentially the " - Season 2" is causing problems.
So if you were to add "Newsleecher", "t" and " - Season \d+" to your blacklist, that would probably solve most of my directory problems :)

Aha! My apologies, I never actually checked to see if there was a way of switching off updates... which if you solve the above problems I potentially won't need!

Anyway, many thanks for the time and effort you put into both the program and help on these forums.
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

I've added "Newsleecher" to the blacklist. 1-2 character terms or numbers have always been blacklisted. I don't think FileBot will consider "t" a valid query or show name (that's also why the TV Series "V" won't work well).

EDIT: "T" is considered a valid TV Show name similar to "V" because there is a show called "T" (actually "T." but . is ignored). Using "temp" or any other blacklisted term as folder name would be better.

FileBot understands patterns like SeriesName - Season [0-9] so that shouldn't be an issue.


Search requests for "T" are not ideal, it does seem to work:

Code: Select all

Input: Downloads/Newsleecher/t/Humans - Season 2/Humans.1x01.mp4
Group: [tvs:humans] => [Humans.1x01.mp4]
Rename episodes using [TheTVDB]
Auto-detected query: [Humans, T]
Fetching episode data for [Humans]
Fetching episode data for [Real Humans]
Fetching episode data for [Humans Mutants]
Fetching episode data for [Humans & Households]
Fetching episode data for [T.]
Fetching episode data for [T@gged]
Fetching episode data for [T. Bag And The Revenge Of The T. Set]
Fetching episode data for [T and T]
Fetching episode data for [T in the Park]
[COPY] Rename [Downloads/Newsleecher/t/Humans - Season 2/Humans.1x01.mp4] to [TV Shows/Humans/Season 01/Humans - S01E01 - Episode 1.mp4]
Processed 1 files

The latest revision r4643 should fix the "folder name lookup issues if folder name uses dots instead of spaces" regression bug.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Thanks for the update. I noticed that my 3.3. problem is not actually related to the directory name, but the actual series name, see below.

The new version seems a handle the directory names better, so I tried a general rename of a lot of files. Now the following is reported :

1. The.Librarians.2014.S03E03.And.the.Reunion.of.Evil.1080p.WEB-DL.DD5.1.H264-Oosh_0.mkv
This name is not matched uniquely, however the list FileBot brings up shows "The Librarians (2014)" as the first entry, which matches exactly to the series name and year. It looks like FileBot does not match on year as well, but should as it would find a unique match then.
Same occurs with Travelers.2016.S01E08.Donner.1080p.WEB-DL.DD5.1.H.264-TRV_0.mkv.

2. NCIS.S14E07.1080p.Home.of.the.Brave.WEB-DL.DD5.1.H264-FENIKS_0.mkv
Humans.S02E06.1080p.AMZN.WEBRip.DD2.0.x264-KiNGS_0.mkv
Gotham.S03E10.Time.Bomb.1080p.WEBRip.DD5.1.x264-CasStudio_0.mkv
These names are not matched uniquely, potentially because there are many series with the series name in them, however, there is only one series actually just named "NCIS" or "Gotham", so FileBot should match it uniquely, and there is only one series that even starts with "Humans".
Note this works correctly in 4.6.1.

3. yonderland.s03e08.720p.hdtv.x264-tla_0.mkv
This is not matched uniquely although there is only one series with this name.

4. Sweet.Vicious.S01E03.Sucker.1080p.MTV.WEBRip.AAC2.0.x264-BTW.mkv
FileBot cannot find a match on this series, probably because of the backslash in the actual series name "Sweet/Vicious".

- When displaying that a series was failed to be identified, wouldn't it be better to display the filename that did not match instead of a list of all files, of which the one in question is not even listed (presumably in the "..." section)? See http://imgur.com/a/L6lDl as an example.

- And would still be nice having a switch to ignore directory matching, I often play in many different directories :)
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
The first option is the one that FileBot thinks is most likely, so you can just hit ENTER to click through the confirmation steps.

Just auto-selecting the first result without confirmation is a feature reserved for the command-line tools.

I could easily skip that step in the GUI if there's a name + year match, but name + year isn't actually very common with TV shows, so it'll only make a slight difference (confirming a preselected option) for a small number of shows.


2.
Same issue. It might seem obvious for you testing with 3 shows, but the same logic that would make life slightly easier for you would make life quite a bit harder for other people. FileBot has to deal with possible shows for all people that have all kinds of strangely named files.

e.g.
Doctor Who - 1x01.mkv by all likelihood is Doctor Who (2005) and not Doctor Who

:idea: You may get better results (or at least less manual confirmation steps) if you use Strict matching instead of Opportunistic matching.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

1. Every little bit helps in automation :)

2. Thanks, but that made no difference. I understand what you mean, but there should be a way of making a one to one match when the series name can be made uniquely. That is why I did not indicate "Once Upon a Time" which has the same year issue. However, as indicated, this worked fine in 4.6.1 but no longer in 4.7+.

3. Your explanation does not indicate why this fails as it should be a one to one match.

4. Similar to 3, there is only one series with the words "sweet" and "vicious" and FileBot does not even find it. Instead if matches only on "Vicious", for some reason ignoring "Sweet".
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Have you tried with Strict mode? That should be a solution for most of these issues.


Re: Sweet & Vicious

Without file paths I can run some tests on I have no idea of even guessing why it may or may not work.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Yes, I have tried Strict mode, and unfortunately, no it makes no difference. Here are images of the attempts :
NCIS - http://imgur.com/a/X1zLW,
Humans - http://imgur.com/a/1Dehy
Yonderland - http://imgur.com/a/xdTT7
Sweet/Vicious - http://imgur.com/a/4DMuj
All files are in "L:\Downloads\Newsleecher".
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
The first 3 do seem odd. Maybe some fine-tuning for these kind of cases would be good.


2.
TheTVDB has issued with shows that contain / because the query FileBot searches with won't contain slashes so TheTVDB won't return results.

If the show was more popular, then FileBot would index it as well it'd just work. Giving the show a good rating on TheTVDB will help the show get included in the FileBot index sooner.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

In the selection dialog, there is a toggle button you can set to Image and Image. ALL allows you to remember and auto-select the same result for the same query (e.g. "NCIS" ➔ TheTVDB::72108) so it won't ask the same question again.

It works the same in strict and non-strict mode, however it's not stored forever so you may need to make the same selection every few months (and you can clear your selections by clearing the cache).
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Hi rednoah,

Me again, wondering what happened this time. I am trying out 4.7.7 and (potentially using a very obscure series again, it's Swedish), wondering what happened this time?
http://imgur.com/a/QFIWt
I am not sure how it converted "Svartsjon" to "Sweet Vicious" since they are not really that close.

Also, you will notice that Sweet.Vicious.Season.1.Special.Too.Legit.1080p.MTV.WEBRip.AAC2.0.x264-BTW_0.mkv has not been highlighted. However, in 4.6.1 (http://imgur.com/a/AWkVP) you will see that it actually does locate and rename this correctly.

At least both versions completely misnamed Svartsjon (I am not really sure why since there is exactly one series Svartsjön on TheTVDB) so no difference there :)

Andrew
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
"Sweet Vicious" is one of the queries that is being considered. If "Svartsjon" yields no results then "Sweet Vicious" seems to be the next best match. That's what Match Mode: Opportunistic does.

2.
Please post filenames as text.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

1. My preferences are still set to "Strict".

2. Um, I'm not sure which filenames you would like as text. Presumably Svartsjon so examples would be :
Svartsjon.S01E01.720p.WEB-DL.AAC2.0.H.264.srt
Svartsjon.S01E01.720p.WEB-DL.AAC2.0.H.264_0 (1).mkv
Svartsjon.S01E02.SWEDiSH.720p.WEB-DL.AAC2.0.H.264-NORDiCUS.srt
Svartsjon.S01E02.SWEDiSH.720p.WEB-DL.AAC2.0.H.264-NORDiCUS_0 (1).mkv
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

TheTVDB doesn't yield any results for "Svartsjon" so it doesn't work:

Code: Select all

$ filebot -list --q "Svartsjon"
TheTVDB: no results
However, searching in Swedish seems to magically solve the issue:

Code: Select all

$ filebot -list --q "Svartsjon" --lang Swedish
Svartsjön - 1x01 - Avsnitt 1
Svartsjön - 1x02 - Avsnitt 2
Svartsjön - 1x03 - Avsnitt 3
Svartsjön - 1x04 - Avsnitt 4
Svartsjön - 1x05 - Avsnitt 5
Svartsjön - 1x06 - Avsnitt 6
Svartsjön - 1x07 - Avsnitt 7
Svartsjön - 1x08 - Avsnitt 8
You'll probably need to add "Svartsjon" as English alias for "Svartsjön" on TheTVDB so it'll work with all language preferences.

When testing in Strict mode, I cannot reproduce any issues with "Sweet Vicious" episode data spilling over to fill in the missing information for "Svartsjon" files.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

It being Swedish and missing an English alias makes sense that it cannot find it.

Anything I can do to help you find the issue?

Just tried "Svartsjon (Swe, Fre, Eng) S01E08.srt" and it found "Sweet Vicious S01E08 - Back to Black", so the Strict setting seems to be ignored. Are you doing this via CLI or via GUI?
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Have these files been processed with FileBot before? If yes, then they're tagged.

@see viewtopic.php?f=4&t=5#p5394
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Tried "filebot -script fn:xattr /path/to/files" and all it returned was "Done ?(?????)?"... so apparently not. Even wiped out the "AppData\Roaming\FileBot" directory and it still matched to "Sweet Vicious".
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Fails, as expected:

Code: Select all

$ filebot -rename "Svartsjon (Swe, Fre, Eng) S01E08.srt"
Rename episodes using [TheTVDB]
Auto-detected query: [svartsjon]
Failed to fetch episode data: [svartsjon]
Failed to match files to episode data
Succeeds, as expected:

Code: Select all

$ filebot -rename "Svartsjon (Swe, Fre, Eng) S01E08.srt" --lang Swedish
Rename episodes using [TheTVDB]
Auto-detected query: [svartsjon]
Fetching episode data for [Svartsjön]
[MOVE] Rename [Svartsjon (Swe, Fre, Eng) S01E08.srt] to [Svartsjön - 1x08 - Avsnitt 8.srt]
Processed 1 files
I've played with the GUI and CLI and everything works as far as I can tell.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Hmm, ok, I removed "C:\Program Files\FileBot" (please note the installer will not install to this location even though I don't want it in (x86) and tried to install into the directory indicated), removed "AppData\Roaming\FileBot", re-installed the program to the default location, wiped out my preferences using "filebot -script fn:preferences --action clear", reset preferences to Strict and tried the filename again in the GUI... no difference, still matches to "Sweet Vicious".
However, trying your command line does return the same results as you.
So it seems there is a problem with the GUI not using the Strict preference properly, but CLI does.

I tried using TheMovieDB (which returned Black Lake) and TVmaze (which returned Svartsjön), both much better results, but unfortunately they both return way too many series results for other titles.
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
Program Files => 64-bit Program Files
Program Files (x86) => 32-bit Program Files

Please use 64-bit FileBot on 64-bit Windows and not 32-bit FileBot.


2.
I've also tested the GUI with the paths you provided. In strict mode, I cannot reproduce the behaviour you're getting.
Image
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

1. Irrelevant.
1.1. You can stick any program you like in any directory you like, windoze simply does this for convenience to split the programs... which ultimately is pointless since a user will almost never install two different bit versions of a program. I never install a program into (x86) unless I have no choice (which is rare), and never have any issues (it is also much more convenient when you are trying to find program directories as you only have to look in one location instead of two).
1.2. Also, I cannot use 64-bit FileBot as I have 32-bit java required for other programs, and 64-bit FileBot will not work with 32-bit java. Also, this makes absolutely no difference to the general running of a program.

2. Isn't there any logging that can be checked to see what the GUI is performing in the background to see why it is not working?
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Yes, but I'm not sure if it'll be of any use in this case:
viewtopic.php?f=3&t=3913
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Where do I find the log files?
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

If you want a log file then you need to set the --log-file option. Otherwise it's all just console output.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Sorry, but I really don't have enough information here in order to do anything :
1. Setting "-Dnet.filebot.logging.debug=ALL" in filebot.launcher.l4j.ini does nothing.
2. There is no console or console output so I don't understand how this option is supposed to work.
3. Where do I set the --log-file option?
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Ok, this got stranger and stranger. In trying to figure where the logging option would be set for the GUI, I thought I would rerun the CLI command... and discovered that depending on which directory I run the CLI command in, I get different results :

l:\downloads>filebot -rename "Svartsjon (Swe, Fre, Eng) S01E08.srt"
Rename episodes using [TheTVDB]
Auto-detected query: [Sweet/Vicious, svartsjon]
Processing multiple shows at once requires -non-strict
Failure (°_°)

l:\>filebot -rename "Svartsjon (Swe, Fre, Eng) S01E08.srt"
Rename episodes using [TheTVDB]
Auto-detected query: [svartsjon]
Failed to fetch episode data: [svartsjon]
Failed to match files to episode data
Failure (°_°)

So I found the problem : The directory name "Downloads" causes the search to return different results.

If a file is dragged from a directory or sub-directory of a directory containing "downloads" (e.g. L:\Downloads\Newsleecher\Svartsjon.S01E01.720p.WEB-DL.AAC2.0.H.264_0 (1).mkv) the program returns two results. If you use another directory (e.g. root), it works just fine.

So there you go, either "downloads" or possibly your blacklist names are causing the "Strict" setting in the program to be ignored.
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
FileBot (console) (filebot.exe) will launch FileBot with console:
viewtopic.php?f=3&t=1961

Running filebot (without arguments) in CMD will do the same. You can do filebot -log-file /path/to/file if you want to log standard output to a text file as well.


2.
If you continue to narrow down the problem then you might find something interesting. Maybe a misleading NFO or URL file somewhere?

It works in a clean environment:

Code: Select all

$ filebot -rename "Downloads/Newsleecher/Svartsjon.S01E01.720p.WEB-DL.AAC2.0.H.264_0 (1).mkv"
Rename episodes using [TheTVDB]
Auto-detected query: [svartsjon]
Failed to fetch episode data: [svartsjon]
Failed to match files to episode data
Failure (°_°)

3.
Please run these commands and post the output:

Code: Select all

cd l:\downloads

Code: Select all

dir

Code: Select all

filebot -script fn:xattr "Svartsjon (Swe, Fre, Eng) S01E08.srt"
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

No, I have checked, you have to be physically in the directory when running the CLI, it does not show the problem otherwise :

c:\>filebot -rename "Downloads/Newsleecher/Svartsjon.S01E01.720p.WEB-DL.AAC2.0.H.264_0 (1).mkv"
Rename episodes using [TheTVDB]
Auto-detected query: [svartsjon]
Failed to fetch episode data: [svartsjon]
Failed to match files to episode data
Failure (°_°)

Output shows nothing :

l:\downloads>filebot -script fn:xattr "Svartsjon (Swe, Fre, Eng) S01E08.srt"
Done ?(?????)?
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

I have no idea. Let's try to narrow down the problem:

1. Delete l:\downloads
2. Create a new file "Svartsjon (Swe, Fre, Eng) S01E08.srt"
3. Try again
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

I wish I could but I have too much in that directory, I would rather rename the directory and create a new one. Also... this only seems to happen under "L:\Downloads". I created a sub-directory "t" here with nothing in it and got this output :

l:\downloads\t>dir

Volume label : General Serial number : B6FE-D044 File System : NTFS
Path : L:\Downloads\t\*.*

. <Dir> 24/01/2017 11:02p .
.. <Dir> 24/01/2017 11:02p ..

0 bytes in 2 file(s) 0 bytes allocated
117,109,800,960 bytes available 3,000,457,228,288 bytes on disk

l:\downloads\t>filebot -rename "Svartsjon (Swe, Fre, Eng) S01E08.srt"
Rename episodes using [TheTVDB]
Auto-detected query: [Sweet/Vicious, svartsjon]
Processing multiple shows at once requires -non-strict
Failure (°_°)

l:\downloads\t>filebot -script fn:xattr "Svartsjon (Swe, Fre, Eng) S01E08.srt"
Done ?(?????)?

Tried the same thing with "C:\Downloads" and could not recreate the result. So something about the "L" drive.
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Is there an *.nfo or *.url file in l:\ or l:\downloads?
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Sigh... yes, finally found this : L:\Downloads\Sweet.Vicious.S01E07.Heartbreaker.1080p.MTV.WEBRip.AAC2.0.x264-BTW.nfo

And yes, somehow having that file in that directory causes FileBot to match Svartsjon to "Sweet Vicious"... talk about obscure!
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Removing the file has solved the Svartsjon issue (though who knows in the future if another nfo file causes another strange rename), but this does not seem to have helped my Strict checking, if Strict checking is on, why does FileBot return many results for a series for which there is only one series by that name, e.g.

c:\>filebot -rename Salem.S03E09.1080p.AMZN.WEBRip.DD5.1.x264-TrollHD_0.mkv
Rename episodes using [TheTVDB]
Auto-detected query: [Salem]
Multiple options: Force auto-select requires non-strict matching: [Salem, Alzeersalem 2000, Salem's Lot]
Failure (°_°)

There is a one to one match on "Salem" only?
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Yes, and opportunistic matching will most likely guess correctly in this fairly obvious case.


However, strict means really strict, as in "doesn't work half the time because it's so strict" strict. If the query is "Salem" and there are many search results that contain "Salem" then there are many unfortunate corner cases that might lead to a bad match. FileBot cannot guarantee a good match in this case, thus the file does not get matched.


Consider this use case:

Code: Select all

Auto-detected query: [Doctor Who]
Multiple options: Force auto-select requires non-strict matching: [Doctor Who, Doctor Who (2005)]
...

You'll probably want to use opportunistic matching at all times. Strict matching is not meant to be used for the average kind of use case.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Ok,

1. Sorry, but an nfo sitting in a directory above that somehow causes a series to be renamed incorrectly is a bug.

2. Why can't there be a simple method of matching? I mean "Salem" is a unique series name so the match should be exact and not require guesswork. Also, whether I choose Strict or Opportunistic, both return a drop-down list, so no, the program does not do this correctly.

3. I tried renaming Salem in 4.6.1 and lo and behold it works as expected! So I repeat the subject of this topic "Pattern matching worse since 4.7". Will 4.7 ever work as well as 4.6.1? (BTW at least 4.6.1 also had the nfo bug :)
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
I've tested with strict mode and nfo files and files remain unmatched as they should be. Bad matching behaviour appears to occur only in Opportunistic mode, which is expected.


2.
That's Opportunistic matching. It's the default a reason and most users never ever change that. The selection dialog is intentional. But you can reduce user interaction by clicking the auto-repeat toggle so it'll remember "Salem" for next time.

FYI the options in the select dialog are sorted by relevance, so you can just hit ENTER without looking at it, if the filenames look easy enough. ;)


3.
For you specifically, possibly not. For the majority of everyone else, it already does work better than ever.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

1. No, matching should occur on the files being checked, not some obscure file not even located in the same directory or even with the same series name. Also, recall I was using Strict mode for all my testing. So definitely a bug.

2. Yes, that should help (though like I said, actually it shouldn't be needed since for these sorts of series there is only a 1-1 match). It appears 4.6.1 did this but 4.7+ does not (a simple string length and string compare across all names to check if there is only one series with the same length name is not a hard thing to add).
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

1.
Believe it or not, that was a feature that multiple people have specifically asked for: checking *.nfo files that are not selected and not in the same folder.

I did some additional testing with *.nfo files and I can't reproduce any bad matches:
Image
:idea: At this point it's a bit of a waste of time for everyone involved. Let's just leave it be. If there's a indeed a bug, then it'll happen again with a completely different set of files. Then we can have a look again.


2.
Experience has shown that a 1-1 match is not good enough to make a decision that cannot be changed later on. You hitting ENTER on a pre-selected item is a minor inconvenience, and you only have to do it once. However, FileBot making a selection and getting it wrong (e.g. Doctor Who) is a major issue because it'll incorrectly match the file every and won't give users an opportunity to change the selection.

Having weighed the pros and cons, the current behaviour is the best compromise and ideal behaviour for the majority of users.
:idea: Please read the FAQ and How to Request Help.
Hemloc
Posts: 25
Joined: 03 Dec 2016, 11:14

Re: [Bug] Pattern matching worse since 4.7

Post by Hemloc »

Whenever I code, I strongly believe in giving users options. If you cannot decide which way to go, then make it an option and let the user choose which way they want it, makes for more coding, but more happy users. Everyone is different and you can never make everyone happy, but adding options means you can minimise the unhappy users :)

Unfortunately you don't seem to follow this philosophy (I don't want it looking at directory names, I don't want it checking nfo files, but there are no options to turn these sorts of things off... and I didn't even know the software did this until I had problems) and that is why I had so many issues.

1. So this should be an option which I would switch off :) To recreate the problem you actually need the physical file (http://nfomation.net/info/1485365895.Sw ... 64-BTW.nfo), just a filename does not work, you need the contents of the file as well, but agreed, I found the problem, its resolved, moving on. You can solve the bug now if you want to :)

2. And I would have to ask : Why can't it be changed later on? That seems to indicate a flaw in the design :) We know users can far too easily do the wrong thing, accidentally choose the wrong option, so why is there no simple option to undo an incorrect decision?
Just so you know, that option on the list box to ask again is not in the FAQ, nor the tutorial, and I have never have even looked at the funny icon on the bottom right until you mentioned it, so it may be other users don't even know of this very useful feature :)

Anyway, it is what it is, the program works, it does what it says on the box, and barring a few issues for people like myself, most people will be very happy with the way it works.
User avatar
rednoah
The Source
Posts: 24698
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [Bug] Pattern matching worse since 4.7

Post by rednoah »

Yes, and IMHO manual series selection (in case there is room for error) is one of these options where users need to make the choice. ;)


1.
How to make sense of the files is not something an average user needs to be concerned about, as long as it just works. And it does, assuming that files can be matched in the first place. In this case, the core issue is not that the random NFO files, but that a search for "Svartsjon" yields no results. Ignoring NFO files will not help you match "Svartsjon". ;)

EDIT: If the series had more than 5 votes, then the show would have been indexed by FileBot with all international titles, which is especially useful for corner cases (of which there is many) where TheTVDB search fails.


2.
If FileBot makes a choice in the background, then users will not understand that there was a choice in the first place and think that it simply doesn't work. Being given a choice is self-explanatory, but having forcing manual selection via some option somewhere in the occasional case where auto-selection picks the wrong is definitely not exactly create design either. In this case, simplicity wins over arcane black magic. :lol:


:idea: I'm not opposed to having a billion configuration options, if and only if this is indeed necessary and desirable. See format expressions. :lol:
:idea: Please read the FAQ and How to Request Help.
Post Reply