Feature request: SRT files

[rednoah]

You could try --conflict index and see it works for you.


Otherwise you'd need to pass in your own custom format, that somehow yields unique file paths for unique subtitle files, how exactly depends on you though, {crc32} would do the trick but might not be what you want, option options like matching identifiers from the filename are possible. Your format. Your rules.
https://www.filebot.net/naming.html


EDIT:

e.g. this format works well:

Code: Select all

{plex.derive{if (dc > 1) '.'+di}}

[cheaters]

Thanks you. Sorry—a lot of questions here. Looks like you have provided an example already but I will post this anyway.

I am not clear on how to "pass in a custom format" in the amc script but would love to know. Is there a page on this site that explains formatting for that? Please provide an example using {crc32}

I have a generic Plex structure:

/Volumes/PlexMedia/Movies/Title/movieFiles and subtitles
/Volumes/PlexMedia/Seeding/Title/movieFiles/Subs/subtitles

Matching identifiers from the filename might do the trick, but something more robust would probably be better since I am not in control of the incoming file names.

I am currently using --conflict skip which causes the extra files to be put into the "unsorted directory" which means I have to manually rename them and add them into the movie's directory.

Not sure exactly what "--conflict index" does, but from two tests it appears to copy and append both subtitle files with "CD1" and "CD2".
CD stands for? I am guessing it's not for Compact Disk 1 and 2

[rednoah]

1.
Please read the amc script manual on how to use custom formats.


2.
--conflict index will add .1 .2 etc to the filename to make it unique. If you get CD1/2 then that's not the doing of --conflict index but multi-part movie handling, which is unrelated to --conflict resolution.


Make sure that your test setup is representative, i.e. real video files, real subtitles.

[cheaters]

1.
Please read the amc script manual on how to use custom formats.

2.
--conflict index will add .1 .2 etc to the filename to make it unique. If you get CD1/2 then that's not the doing of --conflict index but multi-part movie handling, which is unrelated to --conflict resolution.

Make sure that your test setup is representative, i.e. real video files, real subtitles.

Thanks for your patience,,, and help.

Since it worked for you I would like to use {plex.derive{if (dc > 1) '.'+di}} in my AMC script. I included my script below. Where do I place this?

1. I can't find literature on custom formats on modifying the AMC script. Are you saying don't call the AMC script at all—then re-create the AMC script then add my requirements? Or are you saying to call the AMC script and amend it. If it's the later, again, I have not seen a format for that in the manual or any examples of it.

2. I am not downloading multi-part movies. There is only one movie per directory; there are two .srt files in a "Subs" directory within that directory. This didn't happen until after I set the --conflict index flag. My script copies files to "Movies" directory and maintains the originals in "Seeding" directory. The "post amc script" subtitle files in "Movies" are now "moviename (date) CD1.eng.srt" and "moviename (date) CD2.eng.srt" and are the exact same size as the originals which were named "2_English.srt" and "3_English.srt". Their modify/added/created time are exactly the same. I can't think of another reason for it than the --conflict index flag.

This is my script:

Code: Select all

/usr/local/bin/filebot -script fn:amc --output "/Volumes/PlexMedia" --action duplicate --conflict index -non-strict --log-file amc.log --def excludeList=/Users/uname/.filebot/amc.excludes --def unsorted=y --def music=y --def subtitles=en --def minFileSize=0 --def minLengthMS=0 --def plex="127.0.0.1":"##############" "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"

I downloaded filebot with Homebrew and use the complete path to filebot.sh. This is the correct method these days?

[cheaters]

Could it be TheMovieDB that is renaming the srt files?

It looks like my open subtitles lookup fails every time. I see there is a guide on fixing that in the forums. Should I just remove --def subtitles=en from my script?

And I see my token is stale...

Code: Select all

Run script [fn:amc] at [Wed Mar 25 20:11:23 CDT 2020]
Parameter: excludeList = /Users/John/.filebot/amc.excludes
Parameter: unsorted = y
Parameter: music = y
Parameter: subtitles = en
Parameter: minFileSize = 0
Parameter: minLengthMS = 0
Parameter: plex = *****
Parameter: ut_dir = /Volumes/PlexMedia/Seeding/The.Movie.2019.1080p
Parameter: ut_kind = multi
Parameter: ut_title = The.Movie.2019.1080p
Parameter: ut_label = Movies
Use excludes: /Users/John/.filebot/amc.excludes (3)
Input: /Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/Subs/2_English.srt
Input: /Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/Subs/3_English.srt
Input: /Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/The.Movie.2019.1080p.mp4
Group: {Movie=true} => [2_English.srt, 3_English.srt, The.Movie.2019.1080p.mp4]
Get [English] subtitles for 1 files
Looking up subtitles by hash via OpenSubtitles
No matching subtitles found: /Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/The.Movie.2019.1080p.mp4
Rename movies using [TheMovieDB]
Auto-detect movie from context [/Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/The.Movie.2019.1080p.mp4]
Auto-detect movie from context [/Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/Subs/2_English.srt]
Auto-detect movie from context [/Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/Subs/3_English.srt]
[DUPLICATE] from [/Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/The.Movie.2019.1080p.mp4] to [/Volumes/PlexMedia/Movies/The Movie (2019)/The Movie (2019).mp4]
[DUPLICATE] from [/Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/Subs/2_English.srt] to [/Volumes/PlexMedia/Movies/The Movie (2019)/The Movie (2019) CD1.eng.srt]
[DUPLICATE] from [/Volumes/PlexMedia/Seeding/The.Movie.2019.1080p/Subs/3_English.srt] to [/Volumes/PlexMedia/Movies/The Movie (2019)/The Movie (2019) CD2.eng.srt]
Processed 3 files
Notify Plex: [host:127.0.0.1, token:################]
GET: http://127.0.0.1:32400/library/sections/all/refresh?X-Plex-Token=################
HttpClientError: http://127.0.0.1:32400/library/sections/all/refresh?X-Plex-Token=################ [401 Unauthorized]

[rednoah]

The --def subtitles=en option of the amc script will only perform Exact Search and is thus very likely to not return any results if your particular media file isn't very popular around the internet:
viewtopic.php?t=2615


The suball script as separate call is generally recommended, since that will give more options, including -non-strict mode:

Code: Select all

filebot -script fn:suball ...
filebot -script fn:amc ...

[cheaters]

I would rather keep what has been downloaded with the file than search on OpenSubtitles, but I need to keep both srt files not just one. You previously recommended using {plex.derive{if (dc > 1) '.'+di}} in my AMC script to avoid one of them being left behind—that looks like the solution. Not sure how to accomplish this...

[rednoah]

I would rather keep what has been downloaded with the file than search on OpenSubtitles

Then you just disable subtitles altogether. --def subtitles=en means "download additional subtitles" and does not enable or disable the processing of existing subtitles in general.

You previously recommended using {plex.derive{if (dc > 1) '.'+di}} in my AMC script to avoid one of them being left behind—that looks like the solution. Not sure how to accomplish this...

Please read up on custom formats if you're unfamiliar with custom formats:
https://www.filebot.net/naming.html

Please read the amc script manual if you need help integrating your custom format with the amc script.

[cheaters]

It's been a few weeks since I worked on this. I guess my question should have been why is "multipart movie handling" labeling .srt files as:

moviename CD1.eng
moviename CD2.eng?

The directory structure and contents before amc script:
Volumes/PlexMedia/Seeding/moviename/
--movie.mp4
Volumes/PlexMedia/Seeding/moviename/subs
--2_English.srt
--3_English.srt
--moviename.idx
--moviename.sub

Don't the srt files have to be in the same directory as the movie to be picked up by Plex?

[rednoah]

I see. That's because orphaned subtitles (subtitles without corresponding video file) are treated as movie file and processed accordingly, as opposed to ignoring them and doing nothing. This works well in some cases, but not others.


Notably, linked subtitles (subtitles with corresponding video file) are not separately processed, and will always be matched to the exact same movie object as their corresponding movie file:

Code: Select all

movie.mp4
movie.eng.srt
movie.deu.srt

EDIT:

The amc script will automatically ignore orphaned subtitles, only in strict mode however, due to popular request:
https://github.com/filebot/scripts/blob ... roovy#L238

[cheaters]

This is a tough.

Plex says the only way to view sidecar subs is to enable Local Media Assets. I have this disabled because you've mentioned it's the only way to show filebot titles in Plex and not metadata titles. I have Plex setup to download subs from OpenSubtitle but they aren't always accurate, especially for foreign language films. Since my subs were ripped from the DVD directly and are most accurate it seems they should be used.

If I set matching to "Opportunistic" this solves the subtitle naming issue??
Creating a script that runs before the amc script to move those sub files into the same folder as the movie might work, but it would have to finish before the amc script started and what happens if two movies download at aprox the same time? hmmm.

[rednoah]

1.
Well, moving Local Media Assets to the bottom of the list will do the trick as well, without actually disabling it.


2.
Opportunistic Mode creates the issue, by processing badly organized subtitles at all. Strict Mode avoids the issue altogether by not processing subtitles at all. As far as the amc script is concerned.


3.
Yes, if you could pre-process subtitles, and make certain assumptions about which subtitles belong to which video file, then that should do the trick. You'll still have to adjust your format to allow multiple subtitles per language, as {plex} does not take that into account by default.

[cheaters]

OK thanks for that information it's helpful.

I think what I am seeing is just because there are more than one eng.srt file? There doesn't appear to be anything I can do about it since FileBot won't create two files with the same name, and isn't programmed to append an index number for two eng.srt files... unless you consider CD1, CD2 an index number for subtitle files?

I guess the way it stands now will work but it would be better if they weren't renamed with ...CD1.eng and ...CD2.eng when there were duplicates of the same language...

These are the results of running the amc plex script on a movie folder with many subtitles when there are duplicates of certain languages:

Code: Select all

/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD1.chi.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD1.eng.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD1.por.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD1.spa.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD2.chi.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD2.eng.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD2.por.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD2.spa.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD3.eng.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019) CD3.por.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).ara.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).cze.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).dan.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).dut.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).fin.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).fre.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).ger.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).gre.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).heb.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).hun.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).ind.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).ita.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).jpn.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).kor.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).mp4
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).nor.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).pol.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).rum.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).rus.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).swe.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).tha.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).tur.srt
/Volumes/PlexMedia/Movies/The Two Popes (2019)/The Two Popes (2019).vie.srt

[cheaters]

I am totally confused. No matter what I do I can't get rid of CD1 in the Movie Format. Even when using your Syntax

Code: Select all

 {plex.derive{if (dc > 1) '.'+di}}

EDIT:



EDIT:"
Using this syntax in filebot"

Code: Select all

Movies/{n} ({y})/{n} ({y}){if (dc > 1) '.'+di}{'.' +lang}

I get "Binding "lang": undefined"

[rednoah]

1.
{plex} implicitly adds the CD1 part for MoviePart type objects.


2.
{lang} is always undefined for non-subtitle files, which explains why subtitle files get the .eng suffix, and video files don't.


3.
You'll want to view the values of {di} and {dc} first, to test your assumptions, and evaluate if these bindings can help you for your specific use case or not:

Code: Select all

[{di} of {dc}]

[cheaters]

EDIT:

e.g. this format works well:

Code: Select all

{plex.derive{if (dc > 1) '.'+di}}

You used {plex.derive} and I don't see CD1 or CD2 after the subtitles? I used exactly the same Syntax in filebot. I even changed the name of the subtitles to the name of the movie beforehand as you showed in our example above. But I am using the preset with the formula not the Match feature to test.

[rednoah]

You can use {json} to see the raw data for each information object.


In the screenshot above, we're dealing with Movie objects, not MoviePart objects. Because the files are already well-named and well-organized, the subtitles files are linked to their corresponding video file, which makes processing easier. The latter is not the case in your case.


Presumably, considering the screenshot above, if you were to delete the movie file, and then only process the subtitles, i.e. orphaned subtitles, then you would be able to reproduce the problem you're dealing with.


EDIT:

FileBot r7408 No grouping for orphaned sibling files (e.g. subtitles)

[ChefGregS]

I love this topic. LOL

I am sure there is some very simple way to do what I want. Maybe a text file read from the Format line... I got my FB to fix the SDH issue.

Code: Select all

{subt} {fn =~ /eng-SDH/ ? null : { fn =~ /sdh/ ? '-SDH' : null }}

So if the files are badly named (99% of all files are named in a very bad format I have found) Filebot can fix them...THANK YOU!!!!

Anyway, the reason for my post today: I want to upgrade my format to include other issues with obtained file names for subs. An example from a movie I just got:

Code: Select all

Bahasa Indonesia [Forced].ind.srt
Bahasa Indonesia.ind.srt
Bahasa Malaysia.may.srt
Deutsch [Forced].ger.srt
Deutsch.ger.srt
English [Forced].eng.srt
English.sdh-eng.srt

And on and on for almost 60 language files. Some of which Filebot subt doesn't even recognize (which is fine honestly as they are probably spoken by 10 people on some 1-mile square island somewhere - for example: Euskara.baq.srt and Galego.glg.srt)

So back to my desire and issue. With my code that works, I want to expand it to include some of these other issues. FORCED for one. Also, I noticed that in my code if the original file contains sdh and not SDH the case is not dealt with and thus it chooses null. Sort of a pain but not really as 99% of the files I have found are wrongly labeled but do use the capitalized version of SDH.
I have tried several versions of code but I get errors with every one.

Code: Select all

{fn =~ /Forced/ ? '-forced' : {fn =~ /sdh/ ? '-sdh' : {fn =~ /eng-sdh/ ? null}}}

Oddly enough this give an unexpected input "}" error. And if I remove one of those I get Missing Token "}" error. This makes me laugh but doesn't help me to know where the issue is. Can't I embed the commands? I mean, this is an 'IF X THEN Y ELSE Z" Embedded making it continue the search for the data we want: "IF X THEN Y ELSE IF C THEN D ELSE IF A THEN B ELSE Z" I think I am thinking of this the right way. I did so much of this type of testing when I used to do coding (back in the 80's and 90's) but it has been that long since I did this stuff regularly.

As stated above, I would love if this type of stuff is available in a text file, much as I use for fixing certifications

Code: Select all

{any{csv('S:/replacecert1.csv').get(certification)}{certification}{"NR"}

This would allow me to just write and add to a text file for all the stupid abominations I find in how others name their files.

I hope that this makes sense.

[rednoah]

e.g. custom detection logic for forced and SDH akin to Match information from the file path:

Format: Select all

{ '-' + fn.match(/sdh|forced/).lower() }

e.g. custom language detection for unsupported languages:

Format: Select all

{ any{ '.' + lang }{ '.' + fn.match(/baq|glg/).lower() } }

We can read the patterns from external files, or do replacements based on some external TSV file, if that is what is needed. But I kept the examples simple for the first step.


Keep in mind that the outermost {...} are semantically different from {...} blocks in Groovy code. Your code suggests that you don't differentiate when you copy-and-paste bits and pieces. See Learn how {expressions} work and useful Helper Functions for details.