How do I remove numbers at the end of the file name? e.g. "To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO - 32808360-0"

[boukmandutty]

I have some old sagetv recordings that are named like so:

Code: Select all

To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO - 32808360-0.mkv

I can get the right match only using the gui for filebot. Command line attempts match it to the wrong version of the show. Since the format is ok except for the end I was wondering if I could rename the file so that the appropriate directory and file is created without the numbers.

I am trying in other words to move from:

Code: Select all

To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO - 32808360-0.mkv

To

Code: Select all

To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO.mkv

In its appropriately named folder

I am trying to avoid any of the online databases since they dont get some of the series variation right. It is rather strange that they dont using filebot but they do when I feed the files into something like channelsdvr or kodi.

So far I have come up with this borrowing from what I was able to find on the forums:

Code: Select all

filebot -script fn:watcher "/mnt/Sagetv_NasRecordings/TESTING" --output "/mnt/mythtv_storage/Test" --action duplicate -non-strict --db xattr --format /mnt/mythtv_storage/Test/TV/"{fn}"

This still keeps the numbers at the end.

[rednoah]

e.g. use Plain File Mode to remove trailing " - 32808360-0" patterns:

Console Output: Select all

$ filebot -rename *.mkv --db file --format '{ fn.removeAll(/\s-\s\d+-\d+$/) }'
Rename files using [Plain File]
Stripping invalid characters from the output file path: To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO.mkv
[MOVE] from [To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO - 32808360-0.mkv] to [To Catch a Smuggler Madrid - S01E04 - Cocaine CEO.mkv]

Add -unixfs if you want to keep : colon characters. Please read Target Unix Filesystem for details.



Notes:

To Catch a Smuggler: Madrid is not a valid series name. There is no such series. Episode "Cocaine CEO" does not exist in the database. This file name cannot possibly work with Plex, Kodi, etc because the file name fundamentally does not make sense according to online database information. Kodi might display the file name as-is but certainly won't be able to link it to online database information. This is the reason why this file cannot be identified. The numbers at the end of the file name notably are not the reason and removing the numbers will not make the file identifiable.

{fn} is the current file name. {fn} will give you the current file name whatever it may be.

--db xattr is for processing files based on existing xattr metadata. This does not seem to be what you are trying to do. Please read Re-organize previously organized files using local xattr metadata for details.

-script fn:watcher is for monitoring folders for changes and processing files as they come in. This does not seem to be what you are trying to do. You'll want to do a one-off rename operation.

You may prefer to just use the FileBot Desktop application. Plain File Mode works best in the Desktop application because you'll want to use the Format Editor to prototype your format, and preview all the rename operations.

[boukmandutty]

Thanks for the help. Sorry if I wasn't as clear as I needed to be. I was trying and experimenting with quite a number of options trying to figure out what exactly to do with the recordings.

I still record stuff with SageTV and I am actually working out the best way to monitor the folder you see in the command. The goal is ultimately to figure out how to move the files into other folders once they have been remuxed to mkv.

The attempt to remove the numbers from the end of the filename was an option I thought about to help with the database searching part of things. What I discovered is what you pointed out.

The move then was to continue to use the data originally produced with the recording (I gather this is what kodi and Chanels dvr are doing when they successfully import the file with its correct data) while renaming the file and creating the folder without the number.

Will read through what you have suggested.

[rednoah]

The move then was to continue to use the data originally produced with the recording (I gather this is what kodi and Chanels dvr are doing when they successfully import the file with its correct data) while renaming the file and creating the folder without the number.

What additional data do you have alongside the file? *.nfo files of some kind? Can you provide sample file paths and sample file contents?

[boukmandutty]

So a typical recording produced would look like this:

TheOffice-S06E17-TheDelivery-35400686-0.ts

The properties file produced for it would contain the following:

Code: Select all

#Generator: CMT Metadata, written on Sunday, September 3, 2023 6:30 PM
SeasonNumber=6
Live=false
Subtitled=false
Director=
EpisodeCount=0
Guest\ Voice=
ExtendedRatings=
Actor=Steve Carell;Rainn Wilson;John Krasinski;Jenna Fischer;B.J. Novak;Ed Helms;Leslie David Baker;Brian Baumgartner;Kate Flannery;Angela Kinsey;Oscar Nunez;Phyllis Smith;Paul Lieberstein;Mindy Kaling;Creed Bratton;Craig Robinson;Ellie Kemper
Trivia=
CollectionName=
DiscNumber=0
MediaProviderID=tvdb
Host=
Stereo=false
Dolby=false
ScrapedDate=1693783005023
Writer=
Correspondent=
EpisodeNumber=17
Anchor=
Dubbed=false
PartNumber=1
Guest\ Star=Sarah Baker;Keeshan Giles
Title=The Office
MediaType=TV
Guest=
Taped=false
PropertiesWrittenBy=CMT
IMDBID=
3D=false
Executive\ Producer=Ben Silverman;Greg Daniels;Ricky Gervais;Stephen Merchant;Howard Klein
Letterbox=false
MediaTitle=The Office
Musical\ Guest=
ChannelPremiere=false
SAP=false
Producer=
AiringTime=
OriginalAirDate=1267682400000
SeasonFinal=false
CollectionOverview=
ParentalRating=TV14
SeriesInfoID=726133
MediaProviderDataID=78107
Description=When Pam's contractions begin, she and Jim try to wait things out as long as possible so they can have more time at the hospital; Michael anxiously awaits the baby's arrival; Erin makes Andy jealous when she has lunch with Kevin.
TrailerUrl=
Rated=
UserRating=0
Misc=
New=false
DD5.1=false
TotalParts=2
AiringDuration=
CollectionID=0
Language=
Contestant=
HDTV=false
Composer=
Choreographer=
Team=
Narrator=
SeriesFinale=false
Judge=
Premiere=false
SeasonPremiere=false
CC=false
Genre=Sitcom
ExternalID=EP7261330242
Voice=
RunningTime=1200000
SeriesPremiere=false
Year=
Quotes=
EpisodeName=The Delivery
Widescreen=false
Surround=false
TagLine=
ScrapedBy=Phoenix
SeasonFinale=false
AltEpisodeNumber=

It is not consistent though. Sometimes there is not much in the properties file from what I have seen. I will have to watch one of that was just created to see if it gets updated with info over time.

[rednoah]

What does the *.ts metadata file say for your To Catch a Smuggler: Madrid - S01E04 - Cocaine CEO - 32808360-0.mkv problem case?


TheOffice-S06E17-TheDelivery-35400686-0.ts is the video file. What is the properties file name? TheOffice-S06E17-TheDelivery-35400686-0.properties perhaps?


TheOffice-S06E17-TheDelivery contains lots of identifying information, but the file name alone is plenty identifiable in this particular case:

Properties: Select all

Title=The Office
EpisodeName=The Delivery
SeasonNumber=6
EpisodeNumber=17

You could write a Plain File Format akin to Rename files based on sibling XML files to rewrite file names based on nearby properties files.



EDIT:

I've added another Rename files based on sibling properties files example to the docs.

[boukmandutty]

Just getting a chance to return to this project. Using your direction I was able to come up with this:

Code: Select all

filebot -script fn:watcher "/home/jeanot/Processing" --output "/home/jeanot/Processing" --action move -non-strict --log-file amc.log --def movieFormat="/home/jeanot/Processing/Movies/{ ~plex }" seriesFormat="/home/jeanot/Processing/TV/{n}/Season {s}/{ fn.removeAll(/\s-\s\d+-\d+$/)}'"

Seems to be working from what I can see with my experiments.

I just noticed I hadnt adjusted the moves part. Since {~plex} was giving me some incorrect matched I tried to figure out how to build my title and season folders using --db file but couldnt. Lucky for me adding the fn.removeAll part you had suggested to the above seems to have helped with using the online database for series.

How would you get season and episode folders from --db file if you had to?

[rednoah]

Note that your custom format merely formats the given Movie / Episode match, which is completely unrelated to Movie / Episode auto-detection which happens beforehand. Please post file paths if you encounter mismatches in Movie Mode or Series Mode. Please read How to Request Help for details.




Plain File Mode is about rewriting local file paths. There is no Movie / Episode auto-detection and so there is no Movie / Episode match. You only have the file path. You can use your own custom code to extract information from the file path, and then use that information to generate new file paths.

e.g.


e.g.

You can use Groovy code and regular expressions to extract pieces of information from the file path, and then use those pieces to generate a new file path.

Format: Select all

{
	def n = folder.dir.name
	def s = folder.name.before(/[.]/)
	def c = folder.name.after(/[.]/)
	def e = fn.before(/[.]/)
	def t = fn.after(/[.]/)

	"$n/Season $s - $c/$n - S${s.pad 2}E${e.pad 2} - $t"
}

e.g.

If each of your media files is accompanied by a properties metadata file, then you can use that information to generate new file paths:

Console Output: Select all

$ ls
name.ts
name.properties

Properties: Select all

MediaProviderID=tvdb
MediaProviderDataID=78107
Title=The Office
SeasonNumber=6
EpisodeNumber=17
EpisodeName=The Delivery

Format: Select all

{
	def m = lines(folder / fn + '.properties').collectEntries{ it.split('=') }
	"$m.Title {$m.MediaProviderID-$m.MediaProviderDataID}/$m.Title - S${m.SeasonNumber.pad(2)}E${m.EpisodeNumber.pad(2)} - $m.EpisodeName"
}

Console Output: Select all

$ ls
The Office {tvdb-78107}/The Office - S06E17 - The Delivery.ts
The Office {tvdb-78107}/The Office - S06E17 - The Delivery.properties

[boukmandutty]

Thanks for the help. At the moment I have managed to get here:

Code: Select all

jeanot@jeanot-ubuntu:~$ filebot -rename -r /mnt/Sagetv_NasRecordings/Major\ Crimes\ -\ S02E16\ -\ Risk\ Assessment\ -\ 34839286-0.ts --db file --action copy --output Plex --format '{ 
        def n = fn.before(/[-]/)
        def s = fn.after(/[-]/).after(/[S]/).before (/[E]/)
        def t = fn.after(/[.]/)
        def e = fn.take (fn.lastIndexOf (" -"))
        
         "$n/Season $s/$e"
         
  }'
Rename files using [Plain File]
[COPY] from [/mnt/Sagetv_NasRecordings/Major Crimes - S02E16 - Risk Assessment - 34839286-0.ts] to [/home/jeanot/Plex/Major Crimes/Season 02/Major Crimes - S02E16 - Risk Assessment.ts]

Ignore the unused $t variable.

I tried the properties route and found out that some of the .properties files did not have the season and episode numbers.

[boukmandutty]

I have a question. Is it normal for the fn:watcher to wait until all recordings are done and rename them all in one go as a batch vs doing them each at a time after they are written?

I execute the following to watch for recordings to my sagetv folder.

Code: Select all

filebot -script fn:watcher "/mnt/Sagetv_NasRecordings/RecordingZ" --output "/mnt/Sagetv_NasRecordings/Plex_Hardlinks" --action duplicate  -non-strict --log-file amc.log --def seriesFormat="/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/{n}/Season {s}/{ fn.removeAll(/\s-\s\d+-\d+$/)}" --def ignore=".ts|.txt|.edl"

It ignores all the extensions indicated at the end and processes only the .mkv files that SageTV creates after remuxing the recordings. It also gets rid of the numbers I was having trouble with. The odd thing has been that it only renames all the files after all writing has ceased to the folder being watched. So if there are three episodes of the office in sequence being recorded, it will rename only after the last one is done. Is this by design?

I recorded all of these and would only execute the rename after they were all done:

Code: Select all

[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E10 - Christmas Party - 35718044-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E10 - Christmas Party.mkv]
[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E12 - The Injury - 35718046-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E12 - The Injury.mkv]
[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E13 - The Secret - 35718047-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E13 - The Secret.mkv]
[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E14 - The Carpet - 35719944-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E14 - The Carpet.mkv]
[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E15 - Boys and Girls - 35719945-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E15 - Boys and Girls.mkv]
[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E16 - Valentine's Day - 35719946-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E16 - Valentine's Day.mkv]
[DUPLICATE] from [/mnt/Sagetv_NasRecordings/RecordingZ/The Office - S02E17 - Dwight's Speech - 35719947-0.mkv] to [/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/The Office/Season 2/The Office - S02E17 - Dwight's Speech.mkv]

[rednoah]

Note that you have switched from Plain File Mode to watcher script / amc script / Episode Mode so this is a completely new topic.

The watcher script will only process files after after there are no longer any change events for some time. This is to avoid processing half-copied files.

If you're using Linux, then I recommend using the filebot-watcher.sh shell script watch for changes via the inotifywait tool. You will find that watching for file system events doesn't work the way you think, there is only a CREATED event followed by thousands of MODIFIED events over time, and notably no "file transfer complete" event.

[boukmandutty]

Note that you have switched from Plain File Mode to watcher script / amc script / Episode Mode so this is a completely new topic.

I know. I am experimenting to find the most reliable method and learning at the same time. For now my Plain File Mode script needs a bit more work to get to the point where it can distinguish between series, movies, and sports and create folders and rename accordingly.

The watcher script will only process files after after there are no longer any change events for some time. This is to avoid processing half-copied files.

Understood. Thanks for explaining.

[rednoah]

Any kind of file system monitoring will work best if you move files into the monitored folder in a single atomic move operation.

Make sure not to copy files or transcode files into the monitored folder. Always perform non-instant operations in a staging folder that is not monitored (but is located on the same filesystem as the monitored folder) and only ever move the completed unchanging files to the monitored folder.

[boukmandutty]

A related followup as I attempt to refine things:

How do I combine these two things? Is it possible at all:

First:

Code: Select all

{ fn.removeAll(/\s-\s\d+-\d+$/)}

To remove the trailing numbers in the original file name.

And Second:

Code: Select all

{ '.' + fn.match(/-([a-z]+)$/) }

To carry over renamed companion files.

I am trying to carry over the edl files so I don't have to scan for commercials a second time. So fart I have tired:

Code: Select all

filebot -script fn:watcher "/mnt/Sagetv_NasRecordings/RecordingZ" --output "/mnt/Sagetv_NasRecordings/Plex_Hardlinks" --action duplicate  -non-strict --log-file amc.log --def seriesFormat="/mnt/Sagetv_NasRecordings/Plex_Hardlinks/TV/{n}/Season {s}/{ fn.removeAll(/\s-\s\d+-\d+$/).match(/-([a-z]+)$/)}" --def ignore=".ts|.txt"

[rednoah]

What are the file paths you're testing with?

What does the console output say when you run the command above?

You can put bits and pieces together like so:

Format: Select all

{ fn.removeAll(/\s-\s\d+-\d+$/) }{ '.' + fn.match(/-([a-z]+)$/) }

** The code above is untested because you did not provide sample file paths nor console output and may or may not do what you want, depending on what it is that you want and the file names that you have.




EDIT:

Note that using {fn} in Episode Mode is generally a bad idea. FileBot has already identified the Episode, so you'll want to use {n} {s00e00} {t} etc for naming the file. Not using that information does not change the fact that FileBot has identified the file as that Episode. You cannot fix Episode mismatches at format time.

Are you perhaps trying to combine the watcher script and Plain File Mode? I'd just use inotifywait and filebot -rename for that.