xattr or folder structure resulting in mismatches

Running FileBot from the console, Groovy scripting, shell scripts, etc
Post Reply
Cinder
Posts: 20
Joined: 17 Jun 2015, 19:22

xattr or folder structure resulting in mismatches

Post by Cinder »

I have some movies that are repeatedly being mismatched. I'm running Filebot in Docker with the command:

Code: Select all

docker run -it --name=filebot --rm                                              \
        -v data:/data                                                           \
        -v /volume1/Movies:/volume1/Movies                                      \
        -v /volume1/scripts/filebot:/volume1/scripts/filebot                    \
        -e USER_ID=1024                                                         \
        -e GROUP_ID=100                                                         \
        -e DuseExtendedFileAttributes=false                                     \
        rednoah/filebot:latest                                                  \
                                                                                \
        -script fn:amc "/volume1/Movies/Seed"                                   \
        -no-xattr                                                               \
        -non-strict                                                             \
        --db TheMovieDB                                                         \
        --output "/volume1/Movies/Library"                                      \
        --log-file "/volume1/scripts/filebot/amc-movies.log"                    \
        --action hardlink                                                       \
        --conflict override                                                     \
        --def music=n artwork=n                                                 \
        --def "ut_label=movie"                                                  \
        --def "movieFormat=/volume1/Movies/Library/{n} ({y})/{fn}"              \
        --def subtitles=en                                                      \
        --def excludeList="/volume1/scripts/filebot/excludeList-movies.txt"     \
        --def minLengthMS=0                                                     \
        --def minFileSize=0                                                               
Recently I've found that some movies are not hardlinked in to their correct subfolders in the /volume1/Movies/Library structure, instead they are all put into a subfolder "/volume1/Movies/Library/David Attenborough and the Sea Dragon (2018)" because xattr incorrectly reads them as this match. So my output would be like this before going on to hardlink them into the incorrect subfolder just mentioned:

Code: Select all

Input: /volume1/Movies/Seed/movie1.mkv
Input: /volume1/Movies/Seed/movie2.mkv
Input: /volume1/Movies/Seed/movie3.mkv
xattr: [movie1.mkv] => [Attenborough and the Sea Dragon (2018) [CD1]]
xattr: [movie2.mkv] => [Attenborough and the Sea Dragon (2018) [CD3]]
xattr: [movie3.mkv] => [Attenborough and the Sea Dragon (2018) [CD2]]
Group: [mov:attenborough and the sea dragon 2018 cd1] => [movie1.mkv, movie2.mkv, movie3.mkv]
Obviously the movie files are named differently in reality, but you get the idea. So the question is, do the files somehow have incorrect xattr metadata, or is the xattr parser doing something strange? Setting the -no-xattr flat made no difference.

Also I am suspicious of the fact that the folder "/volume1/Movies/Unorganized" is not completely strict in the way it's structured, sometimes the .mkv files are directly under "/volume1/Movies/Unorganized" and sometimes they are in "/volume1/Movies/Seed/MovieName/". I think pretty much all the files I'm having issues with are not in subfolders named by the movie title, which I think is throwing Filebot off and mismatching against the folder structure. This is not something I am able to change either, sometimes files come in subfolders, sometimes they dont, thats what I have Filebot for, to create a perfectly organized, hardlinked, structure elsewhere.

Any suggestions on how to go forward?

In any case I would like to see an option to disable folder structure being a part of the movie/series detection. It's probably good for a lot of users, but my files always contain the title and year in the filename, and I use separate scripts (and folders) to handle tv series and movies, so being able to just disregard the folder structure would allow me to get better matches, I would assume.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: xattr or folder structure resulting in mismatches

Post by rednoah »

1.
One example of a real file path that was misidentified as David Attenborough and the Sea Dragon (2018) would be helpful for testing, or the logs of when that happened. Since I don't know why it happened in the first place, I can't recommend or implement any potential workarounds (since they underlying cause often isn't what it seems to be). Specific examples and test cases would be much appreciated.

:idea: FileBot generally always works regardless of folder structure, since FileBot just looks at everything, and picks whatever makes most sense. If the filename matches Name (Year) then this will be the most likely match. If it doesn't work, there might be a complex explanation, such as your names / years not matching the database, and thus throwing FileBot off track. Specific examples might tell us more.


2.
You can clear xattr like so:

Code: Select all

filebot -script fn:xattr --action clear /path/to/files
:arrow: viewtopic.php?f=4&t=5#p5394
:idea: Please read the FAQ and How to Request Help.
Cinder
Posts: 20
Joined: 17 Jun 2015, 19:22

Re: xattr or folder structure resulting in mismatches

Post by Cinder »

rednoah wrote: 06 Nov 2018, 06:11 1.
One example of a real file path that was misidentified as David Attenborough and the Sea Dragon (2018) would be helpful for testing, or the logs of when that happened. Since I don't know why it happened in the first place, I can't recommend or implement any potential workarounds (since they underlying cause often isn't what it seems to be). Specific examples and test cases would be much appreciated.

:idea: FileBot generally always works regardless of folder structure, since FileBot just looks at everything, and picks whatever makes most sense. If the filename matches Name (Year) then this will be the most likely match. If it doesn't work, there might be a complex explanation, such as your names / years not matching the database, and thus throwing FileBot off track. Specific examples might tell us more.


2.
You can clear xattr like so:

Code: Select all

filebot -script fn:xattr --action clear /path/to/files
:arrow: viewtopic.php?f=4&t=5#p5394
Sent you the details by PM. In short I tried your suggestion of clearing the xattr, but that didn't fix the mismatches even though I could see that the files being mismatched had the David Attenborough[...] stuff in their xattr metadata.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: xattr or folder structure resulting in mismatches

Post by rednoah »

e.g. due to xattr, this file has already been identified previously, and so subsequent runs are even more likely to pick that match that was selected previously for some reason:

Code: Select all

xattr: [2048.Nowhere.to.Run.2017.mkv] => [Lost City Raiders (2008)]
...
Group: [mov:lost city raiders 2008] => [2048.Nowhere.to.Run.2017.mkv]
...
So to re-process a file, you'll need to remove xattr before you can do a clean test run on these files:

Code: Select all

filebot -script fn:xattr --action clear /path/to/files
:arrow: viewtopic.php?f=4&t=5#p5394


:idea: When testing --action test is recommended as to avoid actually processing files during testing. Alternatively, filebot -revert can also be used to revert badly processed files. filebot -revert will also clear xattr metadata.
:idea: Please read the FAQ and How to Request Help.
Post Reply