filebot says duplicate but file name is different

All about user-defined episode / movie format expressions
Post Reply
jprokos
Posts: 107
Joined: 09 Oct 2016, 02:01

filebot says duplicate but file name is different

Post by jprokos »

I have been working on this for a few hours and can't quite seem to get what I want. Then I did but now having an issue with FileBot thinking two files are duplicates. I am sure there is an error in the syntax but can't figure it out... maybe time for some rest.

:?: I thought the index feature would index if the file names were exactly the same not if the file type was the same, but that appears wrong.

Code: Select all

/Volumes/PlexMedia/PlexServer_1/{plex.derive{' - '+allOf{tags}{audio.language}{vf}{vs}{vc}.join(' ')}}{if (dc > 1) '.'+di}
These two files are being renamed using the above syntax.

Code: Select all

Pom Poko (1994) - [en] 1080p BluRay x264.mp4
Pom Poko (1994) - [ja] 1080p BluRay x264.mp4
This is the result:

Code: Select all

Pom Poko (1994) - [en] 1080p BluRay x264 CD1.mp4
Pom Poko (1994) - [ja] 1080p BluRay x264 CD2.mp4
The below files are behaving correctly giving me the results I am looking for using {if (dc > 1) '.'+di}.
The below files sizes are different from one another. Whereas the above files are exactly the same (their only difference may be one is dubbed and one is not).

Code: Select all

Cimarron (1931) - 480p.XviD.3.avi
Cimarron (1931) - 360p.XviD.2.avi
Cimarron (1931) - 360p.XviD.1.avi
It looks like it's my formatting when audio.language is added to the filename?
User avatar
rednoah
The Source
Posts: 18986
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot says duplicate but file name is different

Post by rednoah »

Note to differentiate between "duplicate destination path" (as per your format) and "duplicate object" (as per {object} and {dc}). Using {dc} in the format ensures unique file paths, and thus eliminates "duplicate destination path" conflicts.


:idea: {di} and {dc} yield the same value regardless of what the rest of your format may or may not be doing. If your format adds unique information for each unique file to generate unique file paths, then {di} and {dc} may not work well for your use case. You may need to count "duplicate count per movie/codec/language combination" via {model} if you need your custom format to support more complex more intricate use cases:
viewtopic.php?t=9814


:idea: Alternatively, you can also use --conflict index which adds .1 .2 .3 to file paths as we're moving files, to resolve "duplicate destination path" conflicts even if the format doesn't account for (or can't easily account for) overlapping paths.
:idea: Please read the FAQ and How to Request Help.
jprokos
Posts: 107
Joined: 09 Oct 2016, 02:01

Re: filebot says duplicate but file name is different

Post by jprokos »

Plex formatting rules say anything after a "-" will be ignored so I figured that is the spot to add this additional information about the movies. Where I want to add tags, add audio language to the filename if it's not English (FRENCH, JAPANESE, GERMAN), and {vf}{vs}{vc} I don't want CD1 CD2 to appear for multiple srt files that have the same language (as in eng.srt) or instances where movie files are the same but where one is dubbed and one is not. It would be great if DUBBED could be added to the filename if the primary audio was english but the movie was not.

This is what I have in qBittorrent:

Code: Select all

filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" --action duplicate --conflict index  -non-strict --log-file amc.log --def excludeList=/Users/name/.filebot/amc.excludes --def unsorted=y --def music=y  "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"
This is the structure of files before they are touched with the above script.

Image

How would you go about avoiding the CD1 and CD2 on movies and srt files when they are copied from that file structure over to another drive and handling the above renaming scheme?
jprokos
Posts: 107
Joined: 09 Oct 2016, 02:01

Re: filebot says duplicate but file name is different

Post by jprokos »

It seems the issue is that the subtitles are in a subfolder and not named properly for FileBot. The way I have it set up now is two HDD. One for seeding and one for Plex. I have been copying directly to the Plex using that as my --output location in the amc script and keeping the original torrent on the seed drive untouched while seeding.

If I run a script beforehand to move and rename subtitles to the movie's root directory on the seed drive it would alter the structure of the seeding torrent - bad.

The only option would be to copy the files to the Plex drive, then move and rename subtitles (so they aren't "orphaned" before calling the filebot amc script. I could even delete non english .srt .idx .sub files at this point. This would avoid all of the issues that I am experiencing with subtitles being marked as movie files and indexed, etc..

That's basically re-writting the entire amc script just so I can have subtitles named properly. I don't even know where to start.
User avatar
rednoah
The Source
Posts: 18986
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot says duplicate but file name is different

Post by rednoah »

1.
Most importantly, does Plex even support multiple subtitles per language? And if yes, what is the recommended naming in this case?


2.
As for auto-detection DUBBED movies. You can of course do all kinds of auto-detection logic yourself in your custom format, but suffice it to say, it's going to be a bit intricate, and probably not much built-in magic that'd help make things easier for you here.


3.
My first though would be a standalone pre-processing script looks for all the subtitle files in Subs folders and then move/rename those according to the primary video file location/name.
:idea: Please read the FAQ and How to Request Help.
jprokos
Posts: 107
Joined: 09 Oct 2016, 02:01

Re: filebot says duplicate but file name is different

Post by jprokos »

rednoah wrote: 28 Sep 2020, 10:57 1.
Most importantly, does Plex even support multiple subtitles per language? And if yes, what is the recommended naming in this case?
2.
As for auto-detection DUBBED movies. You can of course do all kinds of auto-detection logic yourself in your custom format, but suffice it to say, it's going to be a bit intricate, and probably not much built-in magic that'd help make things easier for you here.
3.
My first though would be a standalone pre-processing script looks for all the subtitle files in Subs folders and then move/rename those according to the primary video file location/name.
I always appreciate your replies. Looking at other users posts I see I am relatively ignorant :(

1. Yes Plex does support multiple subtitles per language. Plex will load as many as exist in the folder.
Although they will display the same: English (SRT External)
Supported Subtitle Formats
The following formats are fully supported either as embedded tracks or external subtitle files. Full support means they are compatible with all Plex Apps, including clients that require transcoded media.
SRT (.srt)
SMI (.smi)
SSA (or ASS) (.ssa or .ass)
WebVTT (.vtt)

Code: Select all

/Movies
   /Avatar (2009)
      Avatar (2009).mkv
      Avatar (2009).eng.srt
      Avatar (2009).eng.forced.ass
2. I don't think I am up to doing anything "intricate" I am not one with the "FileBot Force" :lol:

3. I like the concept but read item 2 above. Would need some hand holding.

Would it be possible to post process only the english.srt files and move (mv) them to the newly created or pre-exisiting directory on my Plex drive. I don't want to alter the seeding torrent's structure by modifying it or it's contents.

Running this command on my seed drive finds all of the proper files but not directory specific.

Code: Select all

$ find . -print | grep -i eng.*.srt
If I could do this in one shot along with the following in my torrent client:

Code: Select all

filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" --action duplicate --conflict index  --log-file amc.log --def excludeList=/Users/John/.filebot/amc.excludes --def unsorted=y --def music=y  "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"
Could I use any filebot built in stuff to get paths and filenames along with command line arguments? Any suggestions on syntax?

Thank you
User avatar
rednoah
The Source
Posts: 18986
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot says duplicate but file name is different

Post by rednoah »

1.
The Plex naming scheme gives no advice on how to name same type / same language subtitle files as afar as I can see. You can add ".eng" so you get one per language. You can add ".forced" so you can have non-forced and forced per language. However, it is not documented that you can arbitrarily add ".123" at end too, though there may be undocumented features.


2. & 3.
Importing subtitles is tricky. Historically speaking, the easiest way is to just ignore subtitles altogether. Perhaps have FileBot or Plex fetch them on demand. Depending on your preferences, importing subtitles perfectly may not be worth the effort.


4.
Perhaps --action import is good enough for your use case. That'll simply copy along the Subs folder (and other extra files from the same folder) when processing the primary movie file:
viewtopic.php?t=11079
:idea: Please read the FAQ and How to Request Help.
jprokos
Posts: 107
Joined: 09 Oct 2016, 02:01

Re: filebot says duplicate but file name is different

Post by jprokos »

rednoah wrote: 28 Sep 2020, 18:49 1.
The Plex naming scheme gives no advice on how to name same type / same language subtitle files as afar as I can see. You can add ".eng" so you get one per language. You can add ".forced" so you can have non-forced and forced per language. However, it is not documented that you can arbitrarily add ".123" at end too, though there may be undocumented features.
Plex employee support in the forums
movie title (year).heb.srt
movie title (year).2.heb.srt
movie title (year).3.heb.srt
Unfortunately there is no immediate way to distinguish between a regular .srt file, a hearing impaired file, and a eng.forced.srt file in torrents because they often aren't properly named to begin with. Many times in order for them to be useful they need to be renamed to eng.srt, eng.ssa, and eng.forced.srt respectively.

I will look into --action import !! Can I specify which language or extensions?
User avatar
rednoah
The Source
Posts: 18986
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot says duplicate but file name is different

Post by rednoah »

1.
Assuming that ".eng.123.srt" works, using --conflict index will do the trick, at least for subtitle files that resolve to the same absolute destination path.


4.
--action import copies along companion files and folders. No more. No less. The code is not configurable or subtitle-aware so to say. You can of course take care of files you don't want before or after via your own custom pre- and post-processing scripts.
:idea: Please read the FAQ and How to Request Help.
jprokos
Posts: 107
Joined: 09 Oct 2016, 02:01

Re: filebot says duplicate but file name is different

Post by jprokos »

Would you help me understand...

I think I am running into occasional naming issues because of the net.filebot.metadata attributes on some existing files.

I have never had a subtitle that was not considered "orphaned" so I don't have an example of what the extended attributes would look like on a "non-orphaned" .srt file.

:?: Would you please provide an example of the net.filebot.metadata values on a non-orphaned .srt file

:idea: I ran a command to find net.filebot.metadata extended attributes on all of my .srt files:

Code: Select all

$ find . -iname "*.srt" | while read -r file; do `xattr -l "$file" | grep  "type" >> /Users/john/Desktop/filebotmeta1.txt` ; done
It shows I have these two types (MoviePart and Movie)

Code: Select all

net.filebot.metadata: {"@type":"MoviePart","partIndex":2,"partCount":2,"year":1988,"imdbId":94721,"tmdbId":4011,"language":"en","id":4011,"name":"Beetlejuice","aliasNames":[]}

Code: Select all

net.filebot.metadata {"@type":"Movie","year":1964,"imdbId":58777,"tmdbId":0,"id":58777,"name":"Zulu","aliasNames":[]}
Apparently at some point I changed the amc script (removed --conflict index -non-strict, or I post processed some of these files using the GUI) and now they — and I am assuming their associated .mp4 files — are "MoviePart" files.

:?: What is the best way to change these files back to "Movie" files? Edit or remove the xattr and reprocess?

In the GUI when I use {json} binding on the same files (where the CLI command xattr -l shows type "MoviePart") I get different values.
However, using the {xattr} binding on the same files shows "Beetlejuice (1982) [CD2]" for both the srt and the mp4. That would indicate it is a "MoviePart"
User avatar
rednoah
The Source
Posts: 18986
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot says duplicate but file name is different

Post by rednoah »

MoviePart is a Movie object, but with additional information. In this case, we have 2 distinct non-duplicate MoviePart objects:

Code: Select all

Avatar.CD1.mp4
Avatar.CD2.srt
Avatar.CD1.mp4
Avatar.CD2.srt

Here's a similar use case, but in this case we have the same movie twice, rather than one movie split in two parts, and so in this case we do have a movie duplicate, but neither movie has part information:

Code: Select all

Avatar.720p.mp4
Avatar.720p.srt
Avatar.1080p.mp4
Avatar.1080p.srt

:idea: Note that there are no subtitle-specific xattr. In the examples above, the video xattr and subtitle xattr will be exactly the same.


:idea: Note that MoviePart auto-detection relies heavily on the primary video file, which we don't have when dealing with orphaned subtitle files, and so how things work out in the latter case is rather undefined and depends on the situation.


:idea: You cannot modify xattr manually with FileBot without manually matching and processing files. You could do it manually with 3rd party tools though.
:idea: Please read the FAQ and How to Request Help.
Post Reply