Multiple audio tracks with different codecs and languages

[rednoah]

I've never seen a Title field for an audio stream:

Code: Select all

Title                            : Dolby Digital Plus Audio / 7.1 / 48 kHz / 1280 kbps

I guess this Title tag is supposed to show up when you select a given audio stream in your media player. Someone thought this is a good display value for this audio stream.

Might be useful, but probably won't be defined for the vast majority of files.

[devster]

My assumption was that the "title" field is an optional metadata tag and I wouldn't want to rely on it.

For naming purposes I found out that the AdditionalFeatures bascically means a hybrid stream.
In this case there seem to be:

• "base" 5.1 AC-3 stream, decodable by an AC-3 decoder. (The Format field which represents the "minimal" decoder required)
• Dependant E-AC-3 stream with 2 additional channels, optional and apparently interleaved with the first one (The Format/String apparently)

It seems to be part of E-AC-3 which allows embedding a core AC-3 stream.

The ChannelPositions/String2 seems a bug, a workaround in this specific case would be to tokenize by comma ChannelPositions (Front: L C R, Side: L R, Back: L R, LFE), strip whatever's before the colon, count number of words in the string (3,2,2,1) with the last one being the Low Frequency Effects.

[antisgae]

Just in case it helps to somebody i modified the code and added also flac audio.
Looks like this: "ESP ac3 2.0 ENG flac 1.0"
./1961 - Homicidal - Homicidio - William Castle/1961 - Homicidal - Homicidio - William Castle BDR 1080p 35.0Mbps ESP ac3 2.0 ENG flac 1.0 SUB ENG ESP.mkv

Code: Select all

{
	def codecList =
	[
	'MP3' : 'mp3',
	'FLAC' : 'flac',
	'PCM' : 'pcm',
	'AAC LC' : 'aac',
	'AAC LC SBR' : 'aac',
	'AC 3' : 'ac3',
	'AC 3 Dep' : 'eac3',
	'E AC 3' : 'eac3',
	'E AC 3 JOC' : 'eac3 Atmos',
	'DTS' : 'dts',
	'DTS 96 24' : 'dts',
	'DTS ES' : 'dtses',
	'DTS ES XXCH' : 'dtses',
	'DTS XBR' : 'dts',
	'DTS ES XBR' : 'dtses',
	'DTS ES XXCH XBR' : 'dtses',
	'DTS XLL' : 'dts',
	'DTS ES XLL' : 'dtses',
	'DTS ES XXCH XLL' : 'dtses',
	'DTS XLL X' : 'dtsx',
	'MLP FBA' : 'truehd',
	'MLP FBA 16 ch' : 'truehd Atmos'
	]
	def filter = { [it.lang, it.codec, it.ch, it.objects] }

def audioStreams = []
	def audioClean = { it.replaceAll(/[\p{Pd}\p{Space}]/, ' ').replaceAll(/\p{Space}{2,}/, ' ').slash(' ') }
	def channelClean = { it.replaceAll(/Debug.+|Object\sBased\s?\/?|(\d+)?\sobjects\s\/\s|0.(?=\d.\d)|20/).replaceAll(/6.0/,'5.1').replaceAll(/8.0/,'7.1')}
	def oneStream = { it.collect{ filter(it) }*.minus(null).unique().flatten().join(' ') }
	def dString = { it.toDouble().toString() }
	def toInt = { it.toInteger() }

	audio.collect{ au ->
		def codec = audioClean(any{ au['CodecID/Hint'] }{ au['Format'] })
		def format_profile = any{ audioClean(au['Format_AdditionalFeatures'])}{}
		def String ch = any{ channelClean(au.ChannelPositionsString2).tokenize('\\/')*.toDouble().toString().sum() }
			{ channelClean(dString(au.ChannelsOriginal)) } { channelClean(dString(au.Channels)) }

		def chFilter =	(( ( (ac == 'AAC'||ac == 'MP3') && ch != '2.0') || ( (ac == 'AC3'||ac == 'EAC3'||ac == 'DTS'||ac == 'TrueHD'||ac == 'MLPFBA') && ch != '5.1' ) ) ? ch : null)

		def combined = allOf{codec}{format_profile}.join(' ')
		audioStreams << ['index' : codecList.findIndexOf {it.key == combined}, 'default' : au['default'][0].toBoolean(),
		'codec' : codecList.get(combined, 'UNKNOWN_FORMAT'), 'combined' : combined, 'ch' : ch, 
		'bitrate' : any{toInt(au.BitRate)}{toInt(au.BitRate_Maximum)}{au.FrameRate.toDouble()}{null},
		'objects' : any{'[' + au['NumberOfDynamicObjects'] + ' Objs]'}{null}, 'lang' : any{au.'LanguageString3'.upper().replaceAll("SPA","ESP")}{null} ]
		return audioStreams
	}

	def allStreams = audioStreams.collect{ filter(it) }*.minus(null).unique()*.join(' ')
	def preferredStream = oneStream(audioStreams.findAll{ it.index == audioStreams.index.max() })
	def bestBitRate = oneStream(audioStreams.findAll{ it.bitrate == audioStreams.bitrate.max() })
	def defaultStream = oneStream(audioStreams.findAll{it.default == true})

	allStreams.join(' ').space(' ')
	preferredStream.space(' ')
	defaultStream.space(' ')
	bestBitRate.space(' ')
	[bestBitRate, preferredStream].unique().join(' ')
	[defaultStream, bestBitRate].unique().join(' ')
}

[kim]

added bestPreferredLang = to find best stream in your preferred language (preferredLang) or default as backup
e.g. German

Code: Select all

{
	def preferredLang = 'Deu'
	
	def codecList =
	[
	'MP3' : 'MP3',
	'PCM' : 'PCM',
	'AAC LC' : 'AAC',
	'AAC LC SBR' : 'AAC',
	'AAC LC SBR PS' : 'AAC',
	'AC 3' : 'AC3',
	'AC 3 Dep' : 'EAC3',
	'E AC 3' : 'EAC3',
	'E AC 3 JOC' : 'EAC3 Atmos',
	'DTS' : 'DTS',
	'DTS 96 24' : 'DTS 96-24',
	'DTS ES' : 'DTS-ES',
	'DTS ES XXCH' : 'DTS-ES',
	'DTS XBR' : 'DTS-HD HRA',
	'DTS ES XBR' : 'DTS-HD HRA',
	'DTS ES XXCH XBR' : 'DTS-HD HRA',
	'DTS XLL' : 'DTS-HD MA',
	'DTS ES XLL' : 'DTS-HD MA',
	'DTS ES XXCH XLL' : 'DTS-HD MA',
	'DTS XLL X' : 'DTS X',
	'MLP FBA' : 'TrueHD',
	'MLP FBA 16 ch' : 'TrueHD Atmos'
	]
	def filter = { [it.codec, it.ch, it.objects, it.lang] }

	def audioStreams = []
	def audioClean = { it.replaceAll(/[\p{Pd}\p{Space}]/, ' ').replaceAll(/\p{Space}{2,}/, ' ').slash(' ') }
	def channelClean = { it.replaceAll(/Debug.+|Object\sBased\s?\/?|(\d+)?\sobjects\s\/\s|0.(?=\d.\d)|20/).replaceAll(/6.0/,'5.1').replaceAll(/8.0/,'7.1')}
	def oneStream = { it.collect{ filter(it) }*.minus(null).unique().flatten().join(' ') }
	def dString = { it.toDouble().toString() }
	def toInt = { it.toInteger() }

	audio.collect{ au ->
		def codec = audioClean(any{ au['CodecID/Hint'] }{ au['Format'] })
		def format_profile = any{ audioClean(au['Format_AdditionalFeatures'])}{}
		def String ch = any{ channelClean(au.ChannelPositionsString2).tokenize('\\/')*.toDouble().toString().sum() }
			{ channelClean(dString(au.ChannelsOriginal)) } { channelClean(dString(au.Channels)) }

		def chFilter =	(( ( (ac == 'AAC'||ac == 'MP3') && ch != '2.0') || ( (ac == 'AC3'||ac == 'EAC3'||ac == 'DTS'||ac == 'TrueHD'||ac == 'MLPFBA') && ch != '5.1' ) ) ? ch : null)

		def combined = allOf{codec}{format_profile}.join(' ')

		audioStreams << ['index' : codecList.findIndexOf {it.key == combined}, 'default' : au['default'][0].toBoolean(),
		'codec' : codecList.get(combined, 'UNKNOWN_FORMAT'), 'combined' : combined, 'ch' : ch, 
		'bitrate' : any{toInt(au.BitRate)}{toInt(au.BitRate_Maximum)}{au.FrameRate.toDouble()}{null},
		'objects' : any{'[' + au['NumberOfDynamicObjects'] + ' Objs]'}{null}, 'lang' : any{au.'LanguageString3'.upperInitial()}{null} ]
		return audioStreams
	}

	def allStreams = audioStreams.collect{ filter(it) }*.minus(null).unique()*.join(' ')
	def preferredStream = oneStream(audioStreams.findAll{ it.index == audioStreams.index.max() })
	def bestBitRate = oneStream(audioStreams.findAll{ it.bitrate == audioStreams.bitrate.max() })
	def defaultStream = oneStream(audioStreams.findAll{it.default == true})
	def bestPreferredLang = any{audioStreams.findAll{it.lang == preferredLang }.sort{a, b -> b.bitrate <=> a.bitrate}.collect{ filter(it) }*.minus(null).unique().get(0).join(' ')}{defaultStream}

	bestPreferredLang.space('.')
}

[kim]

Added addToList = to make it more user friendly ( output e.g. [Add to "DTS XBR" codecList] )
Added useChFilter = to make it more user friendly ( true or false, makes it more "scene" like )

Code: Select all

{
	def preferredLang = 'Eng'
	def useChFilter = false
	def filter = { [it.codec, it.ch, it.objects, it.lang] }

	def codecList =
	[
	'MP3' : 'MP3',
	'PCM' : 'PCM',
	'AAC LC' : 'AAC',
	'AAC LC SBR' : 'AAC',
	'AAC LC SBR PS' : 'AAC',
	'AC 3' : 'AC3',
	'AC 3 Dep' : 'EAC3',
	'E AC 3' : 'EAC3',
	'E AC 3 JOC' : 'EAC3 Atmos',
	'AC 3 Dep JOC' : 'EAC3 Atmos',
	'DTS' : 'DTS',
	'DTS 96 24' : 'DTS 96-24',
	'DTS ES' : 'DTS-ES',
	'DTS ES XXCH' : 'DTS-ES',
	'DTS XBR' : 'DTS-HD HRA',
	'DTS ES XBR' : 'DTS-HD HRA',
	'DTS ES XXCH XBR' : 'DTS-HD HRA',
	'DTS XLL' : 'DTS-HD MA',
	'DTS ES XLL' : 'DTS-HD MA',
	'DTS ES XXCH XLL' : 'DTS-HD MA',
	'DTS XLL X' : 'DTS X',
	'MLP FBA' : 'TrueHD',
	'MLP FBA 16 ch' : 'TrueHD Atmos'
	]

	def audioStreams = []
	def audioClean = { it.replaceAll(/[\p{Pd}\p{Space}]/, ' ').replaceAll(/\p{Space}{2,}/, ' ').slash(' ') }
	def channelClean = { it.replaceAll(/Debug.+|Object\sBased\s?\/?|(\d+)?\sobjects\s\/\s|0.(?=\d.\d)|20/).replaceAll(/6.0/,'5.1').replaceAll(/8.0/,'7.1') }
	def oneStream = { it.collect{ filter(it) }*.minus(null).unique().flatten().join(' ') }
	def dString = { it.toDouble().toString() }
	def toInt = { it.toInteger() }

	any{audio.collect{ au ->
		def codec = audioClean(any{ au['CodecID/Hint'] }{ au['Format'] })
		def format_profile = any{ audioClean(au['Format_AdditionalFeatures'])}{}
		def String ch = any{ channelClean(au.ChannelPositionsString2).tokenize('\\/')*.toDouble().toString().sum() }
			{ channelClean(dString(au.ChannelsOriginal)) } { channelClean(dString(au.Channels)) }

		def chFilter =	( ( ( (ac == 'AAC'||ac == 'MP3') && ch != '2.0') || ( (ac == 'AC3'||ac == 'EAC3'||ac == 'DTS'||ac == 'TrueHD'||ac == 'MLPFBA') && ch != '5.1' ) ) ? ch : null )

		def combined = allOf{codec}{format_profile}.join(' ')

		audioStreams << ['index' : codecList.findIndexOf { it.key == combined }, 'default' : any {au['default'][0].toBoolean() }{ audio.size == 1 ? true : '' },
		'codec' : codecList.get(combined, 'Add to "' + combined + '" codecList'), 'combined' : combined, 'ch' : useChFilter ? chFilter : ch,
		'bitrate' : any{ toInt(au.BitRate) }{ toInt(au.BitRate_Maximum) }{ dString(au.FrameRate) }{null},
		'objects' : any{ '[' + au['NumberOfDynamicObjects'] + ' Objs]' }{null}, 'lang' : any{ au.'LanguageString3'.upperInitial() }{null} ]
		return audioStreams
	}

	def addToList = audioStreams.codec.findAll{ it.contains('Add to') }.unique().sort()
	def allStreams = audioStreams.collect{ filter(it) }*.minus(null).unique()*.join(' ')
	def preferredStream = oneStream(audioStreams.findAll{ it.index == audioStreams.index.max() })
	def bestBitRate = oneStream(audioStreams.findAll{ it.bitrate == audioStreams.bitrate.max() })
	def defaultStream = oneStream(audioStreams.findAll{ it.default == true })
	def bestPreferredLang = any{ audioStreams.findAll{ it.lang == preferredLang }.sort{ a, b -> b.bitrate <=> a.bitrate }.collect{ filter(it) }*.minus(null).unique().get(0).join(' ') }{}

	allStreams.join(' & ').space('.')
	preferredStream.space('.')
	defaultStream.space('.')
	bestBitRate.space('.')
	[defaultStream, bestBitRate].unique().join(' & ').space('.')
	[bestBitRate, preferredStream].unique().join(' & ').space('.')
	any{addToList}{bestPreferredLang}{defaultStream}{bestBitRate}{preferredStream}
	}{'NO_AUDIO'}
}

Only use one of these lines:

Code: Select all

	allStreams.join(' & ').space('.')
	preferredStream.space('.')
	defaultStream.space('.')
	bestBitRate.space('.')
	[defaultStream, bestBitRate].unique().join(' & ').space('.')
	[bestBitRate, preferredStream].unique().join(' & ').space('.')
	any{addToList}{bestPreferredLang}{defaultStream}{bestBitRate}{preferredStream}

useChFilter = this line:

Code: Select all

def chFilter =	( ( ( (ac == 'AAC'||ac == 'MP3') && ch != '2.0') || ( (ac == 'AC3'||ac == 'EAC3'||ac == 'DTS'||ac == 'TrueHD'||ac == 'MLPFBA') && ch != '5.1' ) ) ? ch : null )

= if AAC/MP3 2.0 OR AC3/EAC3/DTS/TrueHD/MLPFBA 5.1, then don't show the 2.0/5.1 part

allStreams = All Audio Streams
preferredStream = The last (best) match from codecList (order matters, from low to high quality / or what you prefer)
defaultStream = The Audio Stream tagged "default=Yes" or if only one Stream
bestBitRate = The Audio Stream with the highest BitRate (with BitRate_Maximum/FrameRate as backup)

[nartana]

I couldn't find an answer (bc it's probably obvious): Kim and others helped evolve the def codecList expression, in this thread, but I'm not sure where/how to use that long expression, that defines all the different possible codecs/formats. I want to make sure any movie with Atmos indicates as much, but that doesn't always happen.

I use CLI via SSH to rename media on my Synology (where I have FileBot installed.) It works great. I have a basic understanding of the expressions I've compiled. But is "codecList" a file in my FileBot package that's installed on my Syno? Do I edit that to add the many awesome codecs in the code Kim has evolved, here? Or is that long expression something I add to a CLI command each time I want to process files in my 'unsorted' folder?

[nartana]

Renaming output, CLI via Synology FB install (all packages updated):

Code: Select all

MovieTitle (2019) 118mins [18.0 Mbps] (1920x800) Dolby Digital 5.1 [6ch]

Renaming output, new Windows 10 installation of standalone FB app:

Code: Select all

SameMovieTitle 118mins (18.0 Mbps) (1920x800) TrueHD 7.1 Atmos [8ch]

According to MediaInfo, the second (Windows) output is correct: (8ch Atmos)

I would like to continue using CLI via Synology package - I’ve tried at least ten different expressions I’ve found on the forums, re. audio codec, but it always outputs 5.1 Dolby.

I know I'm not providing all the info you need, but if we know that all the modules are updated to latest, on the Syno server... is there a default reason why outputs differ?

[nartana]

This is the expression I use via CLI (which produces the incorrect codec/channel output, on rename):

Code: Select all

filebot -rename -r /volume1/Media/UNSORTED/Movies/ --db TheMovieDB -non-strict --conflict auto --output /volume1/Media/SORTED/ --format "Movies/{certification}/{n} ({y})/{n} ({y}) {minutes}mins [{mbps}] ({resolution}) {aco} {channels} [{af}]/{ny} @{mbps} {tags.upper[]} {' CD'+pi}" --action copy

[rednoah]

Compare filebot -script fn:sysinfo output. Different versions of MediaInfo may yield different values.

[nartana]

Thank you @rednoah (also for indulging sloppy question format.)

The output is below — I assume the next step is for me to check the version I'm using vs. the newest version available.

Code: Select all

FileBot 4.8.5 (r6224)
JNA Native: 5.2.2
MediaInfo: 19.09
7-Zip-JBinding: 9.20
Chromaprint: fpcalc version 1.4.3
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2019-05-15 (r565)
Groovy: 2.5.6
JRE: Java(TM) SE Runtime Environment 1.8.0_201
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 4 Core / 1 GB Max Memory / 40 MB Used Memory
OS: Linux (amd64)
HW: Linux ******** 3.10.105 #24922 SMP Wed Jul 3 16:37:24 CST 2019 x86_64 GNU/Linux synology_avoton_1815+
DATA: /volume1/@appstore/filebot/data/admin
Package: SPK
License: FileBot License P75********

[rednoah]

Well, it's not about being newer or older, but about being different. FileBot will bundle the latest version on platform-dependent builds, e.g. MSI package. But on platforms where there is more than one CPU architecture, the FileBot package will only bundle platform-independent components, i.e. SPK package, and rely on 3rd party dependencies for platform-dependent components, i.e. MediaInfo, Chromaprint, etc. These tend to be somewhat outdated on Synology because they're maintained by 3rd party volunteers.

MediaInfo: 19.09 is the latest one though, so that should work exactly the same as the latest beta, which bundles this version of MediaInfo as well:
viewtopic.php?t=1609

[nartana]

Thank you, I didn't know that about Syno packages. Now it makes more sense.

I'll test results running FB on a MacMini that's connected to the Syno (to see if my results differ vs. running the Syno FB build.)

Since I'll be trying FB on a diff. platform, it's a good time to ask:
Is the GUI (app) version of FB the only way to use these long { def codecList... } definitions of audio codecs? Like in the "Filter" field of the GUI version?

Or, if there's a thread, or keywords that'll help me search how to use these w/ the CLI-based FB operations...?

I know it's a real huckleberry question. I'm brain-blocked on the next step with this.

Thanks for helping me better understand these diff results (Syno FB vs. Windows FB.)

[rednoah]

What makes you think that format expressions work different in the GUI and the CLI? They're the same of course. Format expressions work the same in both GUI and CLI, it's literally the same code.


Except passing complex multi-line argument values on the command-line might be a little bit troublesome, doable, but annoying. Use the @file syntax for reading command-line arguments from external text files.


You can use filebot -script fn:mediainfo /files to dump raw media info tables, so you can see and compare the raw data that FileBot is getting back from libmediainfo.

[nartana]

Thanks again — that makes so much sense, that the interface is the interface (and expressions work the same, underneath.)

And thank you for the tools. Time to learn about syntax and to compare the data FileBot is getting back from libmediainfo.

Even in the short year I've been using FileBot, it has been really cool to see how you continuously evolve it, based on input.

Cheers @rednoah.

-Erin

[rednoah]

Even in the short year I've been using FileBot, it has been really cool to see how you continuously evolve it, based on input.

Keep the good ideas coming. I might not always implement exactly what you want, but it might inspire me to implement something even more interesting.