Embedded tags

All about user-defined episode / movie / file name format expressions
Post Reply
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Embedded tags

Post by UKenGB »

I'm trying to share media between an existing iTunes system and Plex that I want to be able to use. I'm having trouble with this as iTunes maintains the files' tags in sync with its own database and creates logical and appropriate folder and file names for all the media. It does a very good job of this and the media should then be sharable with any other system as every file contains all the identifying metadata. Except I am coming to realise that Plex ignores any correct tagging of video files and uses only folder and file naming which is unbelievably stupid, but not relevant here.

My problem is that iTunes creates one folder/file naming structure and Plex requires something different. Although iTunes can allow the files to be anywhere, it becomes harder to maintain. Anyway, my question about Filebot (and relevant to whether or not it is worth buying) is whether it can use embedded tags in video files that already contain all the required data rather than chasing around the 'Net to find what is already in the files' tags, OR, completely failing to find anything as it's not a commercially available video and hence will NEVER be found, yet the file's embedded tags already contain ALL the data. What then?

This is my big problem with Plex (and maybe Filebot too, hence the questions). How do they deal with a video file that contains everything needed in its embedded tags and that will never be found in any on-line media database? Plex appears to simply ignore it (a BIG problem), what does Filebot do? How does it name the file? How can it, unless it reads the embedded tags? I'm disappointed I have to ask about any of this since the embedded tag data ought to be the FIRST port of call for both Plex and Filebot to do their stuff, but Plex seems remarkably dumb in this regard. What about Filebot? Can it and does it read tag data in order to name video files?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

1.
AFAIK, iTunes is the only tool that uses iTunes tags. You probably won't find any software (Plex, Kodi, VLC, etc) that uses or cares about iTunes tags.

If MediaInfo supports iTunes embedded metadata, then FileBot will allow you to rename files using that information. Generic File Mode allows you to move/rename files based on any MediaInfo property and other locally available information.

Unfortunately, I don't have any iTunes tags files that I could test with, so I can't confirm if MediaInfo supports iTunes tags. Do you have any small sample files I could use for testing?

It's definitely a fringe use case though. Don't expect a "Rename based on iTunes metadata" button. You might be able to make one though. ;)


2.
If you use the normal Episode / Movie mode matching on unknown video files, then the files will either be ignored or misidentified, and anything that can't be identified by FileBot probably won't work in Plex.
:idea: Please read the FAQ and How to Request Help.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

It is a complete misconception that iTunes uses some proprietary tagging scheme. It mostly uses standard ID3 tags embedded in the files that anything else can read, edit or make use of in any way it sees fit. Any other trivial data that doesn't have a defined ID3 tag (like 'loved' and 'disliked' and some others) is then just maintained within its own database. So everything that is required to COMPLETELY name and identify any media file that has been tagged in iTunes (or any other real media file tagging application) is available for Plex or any other program to use for that exact same purpose. The idea of always having to then go and find this same info on the 'Net is dumb. As I said, a file's tags should be used first and on-line data only then used to fill in any blanks. The idea of such commercial media software ignoring a fully tagged file just because it cannot be found on-line is quite frankly preposterous.

Plex has an option to 'Use embedded tags' for music, so they are perfectly able to do the right thing, but they then completely ignore that for video files and offer no such option. It's simply unbelievable that such an apparently mature product is still so dumb.

I fully realise the prevelance of (basically pirated) media on the 'Net, as torrents etc and that have no embedded tags. So the ability to automatically grab this info from the 'Net is laudable, but this appears to be at the expense of any ability to make use of already fully tagged and correctly identified media. Tags are the intended way to identify such media. That's the way these files were designed. To ignore that standard methodology and rely instead on some kludge of just using folder and file names is incomprehensible. That should only be the last resort when there are no real tags to use.

As for iTunes media being a 'fringe' case, are you serious? It has to be THE most commonly used media manipulation software. Like it or loathe it, there are a LOT of users and other similar software would do well to be easily interoperable.

Anyway, since MediaInfo is perfectly able to read these tags (as written by iTunes), does that mean Filebot will also be able to use them to name the files so that dumbarse programs like Plex can have a chance at figuring out what the hell it is?

I have over a TB of music and video media files, fully tagged in iTunes. What do you need that will assist you?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

1.
ID3 is a standard that goes hand in hand with the MP3 file format. The same cannot be said for MP4 files. I'm not gonna go into the technical details, but if it was as easy and straight-forward as you think, why wouldn't FileBot and Plex implement iTunes tags? Why don't any of the other tools? Why wouldn't all the pirates tag their videos as they do for mp3 files? And if iTunes tags are so prevalent, how come you're the first person to complain in 8 years? It's simply not as easy and straight-forward as you think. ;)

FYI: FileBot stores full metadata as xattr after processing. Unfortunately that "standard" hasn't caught on yet.


2.
If MediaInfo works then your chances are good. If you give me a sample file (that is iTunes tagged) then I can have a look and see if it's possible to create a custom Preset to organize them by Plex standards based on the MediaInfo.
:idea: Please read the FAQ and How to Request Help.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

Be that as it may with ID3 tags actually being 'invented' for MP3 files (and of course with many different iterations for improvements over the years), but similar tagging schemes for (some) other music and video file formats have become essentially standard. I have never come across any media software incapable of understanding all such tags and the whole app genre of 'file tagger' has come into existence just because of such de-facto standards.

However, I would really appreciate it if you could look into this and here's a link to a TV Show in its folder hierarchy as created by iTunes:-

https://dl.dropboxusercontent.com/u/460 ... es%20I.zip

It seems to contain a good mix of tag data and MediaInfo reads all of it. I'm sure this can be found on-line, but this is just for you to study the internal tags as you requested. If e.g. Filebot does first read the painstakingly crafted embedded tags, there is then far less chance of mismatching it with on-line data when looking for any additional information.

Let me know how you get on.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

BTW, I usually use Subler to tag any video files missing them. This can attach any required metadata, including artwork, all of which is then read and used by iTunes - in place. iTunes doesn't play around with it as both apps are using the exact same tags. MediaInfo of course reads the same tags. It is a surprise to me to find media software like Plex that apparently doesn't use them. Here's hoping we can 'improve' Filebot in this way.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

1.
Feel free to make a feature request here and if there's multiple people are interested then I'll have a look at improving support for iTunes tagged files:
viewforum.php?f=6


2.
You're in luck. FileBot custom formats feat. MediaInfo is doing a good job.

This generic file format:

Code: Select all

{media.Album}/Season {media.Season}/{media.Album} - {media.PartID} - {media.Title}
will yield this path:

Code: Select all

QI XL: Series I/Season 1/QI XL: Series I - s01e10 - 10. Inland Revenue
This should be good enough in most cases, but processing the result one more time with TheTVDB and the {plex} format would make sure that the data is 100% perfect for Plex.

If you need to process files like this more than once, then it's worth making a Preset:
Image

@see viewtopic.php?f=3&t=3228&p=17872#p17872
:idea: Please read the FAQ and How to Request Help.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

Thanks for that, I'll have a go. I only wish that Plex was as flexible.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

You wouldn't happen to have a small movie file that is tagged with iTunes?

EDIT:

If you could copy and paste the MediaInfo General Table for a few files: TV Shows, Movies, etc that would be great.
:idea: Please read the FAQ and How to Request Help.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

I'll dig out a movie file for you. May be tomorrow now.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

Here's a small movie file, with some additional (hopefully self explanatory) info in some otherwise empty tags:-

https://dl.dropboxusercontent.com/u/460 ... 04.mov.zip

This was set as a 'Home Video' in iTunes. If you need anything else, just ask.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

A couple of issues that have come to mind when trying to use Filebot with embedded tags.

The Preset feature is great, but it actually is the only way to specify the source as 'Plain file'. The regular Fetch Data menu that you'd otherwise use to run the process does not have this option. So in order to use embedded tags, it's a convoluted process to Edit Presets, pick the required one to edit, then actually choose that one and then you get to edit it. After which you have to save it and then pick it from the menu to run which is a very time consuming workflow when having to be repeated until everything is how you want it.

It would really help if Plain File was an option in the Fetch Data menu. If you really want to improve the workflow, ensure the Episode Format dialog is not modal, so you could edit the format string, save and immediately run to see the result, then quickly adjust the format, save and run again. This would make figuring out complex formats far faster. Once it's as required, simply copy to a preset.

One question though, why is there a Load button below the RH pane. That shows the result of the current rename settings, why would loading new files there be appropriate?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

1.
You only have to create your custom button once, and then it's about as convenient as the Fetch Data Image menu.

Are you referring to the initial testing and prototyping? That should all happen in the Format Editor Image where you can set a sample file Image for testing.


:idea: The List panel can be used to prototype format expressions on multiple files at the same time, but that's more of an accidental feature. :lol:

EDIT: The List panel seems to be broken in 4.7.7 due to strange issues introduced during compile-deploy time. Scary. :shock:


2.
The fastest way for one-off jobs is actually to just drop in the files and then hit F2 and then double-clicking any File item to change the File format:
viewtopic.php?f=3&t=2072
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

One more example for a proper Movie file would be nice. Actually, one example for each unique {media.ContentType} would be great. (I only have "TV Show" so far)
:idea: Please read the FAQ and How to Request Help.
UKenGB
Posts: 49
Joined: 04 Jan 2017, 09:26

Re: Embedded tags

Post by UKenGB »

I understand that the Preset feature is just to store the tested code, for later re-use, but it seems to be the only way to test a format string on all the files using embedded tags. Just seeing the result on one is not enough as there's too great a chance of some variation in other files causing problems. Activating a preset means you can see the result for all the listed files. If you try to do that by editing the format and using the Fetch Data button, you cannot choose to use the embedded data. You can only run it getting external data.

I'll dig out some other media files. Movies tend to be rather large, but I'll find something.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

Here's how I'd prototype a new File format:

1. Drop all the test files into Original Files
2. Hit F2
3. Find the first New Names list item that doesn't look right and double-click it to open the Format Editor for that specific item
4. Fix the format and click Use Format to apply the new format to all items
5. Repeat 3. and 4. until it works for all files

(6.) Create a new Preset for easy access in the future
(7.) Share your new Preset here in the forums ;)
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Embedded tags

Post by rednoah »

UKenGB wrote:I'll dig out some other media files. Movies tend to be rather large, but I'll find something.
Just use copy & paste the information via Format Editor ➔ Bindings ➔ MediaInfo ➔ General ➔ CTRL+A, CTRL+C.

e.g.

Code: Select all

Video_Format_List	AVC
Video_Format_WithHint_List	AVC
Video_Codec_List	AVC
Audio_Format_List	AAC
Audio_Format_WithHint_List	AAC
Audio_Codec_List	AAC LC
FileName	Inland Revenue - [iTunes]
FileExtension	mp4
Format	MPEG-4
Format/String	MPEG-4
Format/Extensions	mp4 m4v m4a m4b m4p 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma f4v
Format_Commercial	MPEG-4
Format_Profile	Base Media
InternetMediaType	video/mp4
CodecID	isom
CodecID/String	isom (isom/iso2/avc1/mp41)
CodecID/Url	http://www.apple.com/quicktime/download/standalone.html
CodecID_Compatible	isom/iso2/avc1/mp41
Codec	MPEG-4
Codec/String	MPEG-4
Codec/Extensions	mp4 m4v m4a m4b m4p 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma f4v
FileSize	262120928
FileSize/String	250 MiB
FileSize/String1	250 MiB
FileSize/String2	250 MiB
FileSize/String3	250 MiB
FileSize/String4	250.0 MiB
Duration	2610080
Duration/String	43 min 30 s
Duration/String1	43 min 30 s 80 ms
Duration/String2	43 min 30 s
Duration/String3	00:43:30.080
Duration/String4	00:43:30:02
Duration/String5	00:43:30.080 (00:43:30:02)
OverallBitRate_Mode	CBR
OverallBitRate_Mode/String	Constant
OverallBitRate	803411
OverallBitRate/String	803 kb/s
FrameRate	25.000
FrameRate/String	25.000 FPS
StreamSize	1903219
StreamSize/String	1.82 MiB (1%)
StreamSize/String1	2 MiB
StreamSize/String2	1.8 MiB
StreamSize/String3	1.82 MiB
StreamSize/String4	1.815 MiB
StreamSize/String5	1.82 MiB (1%)
StreamSize_Proportion	0.00726
HeaderSize	1903211
DataSize	260217717
FooterSize	0
IsStreamable	Yes
Title	10. Inland Revenue
Collection	QI XL: Series I
Season	1
Album	QI XL: Series I
Album/Performer	BBC TV
Part	10
Part/Position	1
Track	10. Inland Revenue
Track/Position	10
Grouping	Entertainment,Games & Quizzes,Comedy
Performer	BBC Two
Composer	BBC iPlayer
Genre	Entertainment
ContentType	TV Show
Description	Sandi Toksvig, Al Murray, Dara O Briain and Alan Davies face some taxing questions.
Recorded_Date	UTC 2011-11-12 21:00:00
Encoded_Date	UTC 1904-01-01 00:00:00
Tagged_Date	UTC 2014-02-04 22:04:38
File_Modified_Date	UTC 2014-02-04 14:04:44
File_Modified_Date_Local	2014-02-04 22:04:44
Encoded_Application	Lavf54.63.104
Encoded_Application/String	Lavf54.63.104
Comment	Sandi Toksvig, Al Murray, Dara O Briain and Alan Davies face some taxing questions.
LongDescription	Stephen Fry asks some taxing questions about things that start with 'i'. Joining him are Sandi Toksvig, Al Murray, Dara O Briain and Alan Davies.
Part_ID	s01e10
TVNetworkName	BBC Two
:idea: Please read the FAQ and How to Request Help.
Post Reply