Metadata and Extended Attributes

[rednoah]

FileBot uses xattr (i.e. Extended Attributes / NTFS Alternative Streams) to store full metadata in filesystem attributes. These xattr are not to be confused with traditional tags that are embedded in the file content, but additional data streams stored in the file system for each file that are not part of the file itself. As such, xattr requires a filesystem that supports xattr, and xattr may be silently lost if files are moved to a filesystem that doesn't support xattr.


Details:

• [xattr:net.filebot.metadata] Episode or Movie Object (as json)
• [xattr:net.filebot.filename] Original filename
• [basic:creationTime] Episode airdate or Movie release year (disabled by default for compatibility reasons)

Usage:
FileBot and other programs have the original filename and all episode / movie metadata available regardless how the file is named. Sure nice to have full metadata instantly / offline available should you need it.


Disable this feature via the -no-xattr command-line flag:

Code: Select all

-no-xattr

or via system properties:

Code: Select all

filebot -script fn:properties --def useExtendedFileAttributes=false

Configure FileBot to store xattr metadata in plain text files instead of filesystem xattr streams:

Code: Select all

filebot -script fn:properties --def net.filebot.xattr.store=.xattr

If you require cross-platform compatibility, then using .xattr folders and files is highly recommended. The FileBot Xattr Metadata Scanners & Plug-ins for Plex
support both native filesystem xattr and .xattr folders by default.


View or Clear Extended Attributes
with the filebot command-line tools and xattr script:

Code: Select all

filebot -script fn:xattr /path/to/files

[rednoah]

1. Rename files with FileBot:

Code: Select all

$ filebot -rename *.mp4 --db TheMovieDB -non-strict
Rename movies using [TheMovieDB]
Auto-detect movie from context: [avatar.mp4]
[MOVE] From [avatar.mp4] to [Avatar (2009).mp4]
Processed 1 files

2. Confirm that FileBot metadata has been stored as xattr in the filesystem:

Code: Select all

$ filebot -script fn:xattr *.mp4
/path/to/Avatar (2009).mp4
	net.filebot.filename: avatar.mp4
	net.filebot.metadata: {"@type":"Movie","year":2009,"imdbId":499549,"tmdbId":19995,"language":"en","id":19995,"name":"Avatar","aliasNames":[]}

3. Use xattr metadata in filter and format expressions:

Code: Select all

$ filebot -mediainfo -r . --filter "y >= 2000" --format "{n} by {director}"
Avatar by James Cameron

4. Process xattr tagged files based on existing xattr metadata:

Code: Select all

filebot -rename -r . --db xattr --action copy --output /Volumes/USB --filter "episode && age < 30" --format "{n} - {sxe} - {t}"

[oneguynick]

Code: Select all

filebot -rename *.mp4 --db TVDB -non-strict
Illegal Argument: java.nio.file.InvalidPathException: Illegal char <*> at index 0: *.mp4 (*.mp4)
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4

^C

Example filename: Penoza - S01E02 - Unpleasant Surprises HDTV-720p.mkv

[rednoah]

*.mp4 refers to all mp4 files. If you're testing with mkv files then *.mkv will select those.

@see https://mywiki.wooledge.org/glob

[oneguynick]

My mistake. I forgot they were *.mkv, good catch. With that said I am now testing the same working commands against my samba share with the same issue. Copy locally just isn't an option. Is there a better way to do this? NFS or SFTP or something?

[oneguynick]

Alright so I took the network share out of the picture for troubleshooting and tried docker:

Code: Select all

docker run -it -v /tv:/volume1 -v /opt/docker/filebot:/data rednoah/filebot -script fn:xattr -r *
docker run -it -v /tv:/volume1 -v /opt/docker/filebot:/data rednoah/filebot -script fn:duplicates -r --action test *

Done ヾ(＠⌒ー⌒＠)ノ in both cases. I specifically copied a folder of shows to ensure there were exact duplicates.

I had considered that maybe it was the ZFS pool. I verified the pool is enabled for xattr:

Code: Select all

NAME                                                    PROPERTY  VALUE  SOURCE
share                                                   xattr     on     default

[rednoah]

NFS might work. SFTP probably doesn't preserve attributes, but RSYNC might (with --xattrs option).

If you're using docker with some host folder mounted into the docker, then there's a good chance xattr won't work. I remember filesystem events not working either. Not sure if that's something that can be enabled or if it's just not supported.

If you care about xattr, then the best solution is usually to process files on the NAS or server where reading/writing xattr usually works out of the box.

FileBot generally doesn't require xattr, it just stores some xattr info if possible, but will work just fine without. It's just that the duplicates script requires them to work, so it'll instantly know which file is which movie/episode without having to bother any web services.

[b2un0]

Hi,

is there any option to add filebot xattr to already sorted files?

eg, i have multiple series where on some seasons the xattr from filebot are missing, so ne the " filebot missing script" does not work correct and think the season is missing.

[rednoah]

You'll have to process files through FileBot again to ensure that xattr are set this time.

You can find files without xattr metadata like so:

Code: Select all

filebot -mediainfo . --filter '!f.metadata' --format {f}

[b2un0]

thanks for your fast reply!

thats the first thing i did, but filebot says "Skipped, already exists".

[rednoah]

1.
You can move all untagged files elsewhere first:

Code: Select all

filebot -mediainfo . --filter '!f.metadata' -exec mv -v {f} /path/to/untagged/files/{f.name}

2.
However, the general smart approach is to never touch files in your Media folder, and only ever move files there via FileBot. That way you can ensure that everything is always in perfect order.

i.e. just process all your files into a new folder, and then remove the old one if you're happy with the new one

Code: Select all

--format /path/to/new/folder/{plex}

[juan]

Can this be used to save the audios and subtitles mediainfo of the multimedia file?

It could be great to have this added to the xattr attributes (or file), so, Plex don't need to scan the multimedia file to get the audio and subtitle languages, the default audio, the forced subtitles, the duratión of the film....

This with a combination of an unionfs or mergerfs to save the xattr attributes, and rclone with a cloud provider (like a gsuite), could be perfect.

Or maybe apart of the xattr attributes, the data could be saved on an .nfo like the one that jellyfin creates, that have all this information!!

Code: Select all

  <fileinfo>
    <streamdetails>
      <video>
        <codec>h264</codec>
        <micodec>h264</micodec>
        <bitrate>10312878</bitrate>
        <width>1872</width>
        <height>1080</height>
        <aspect>26:15</aspect>
        <aspectratio>26:15</aspectratio>
        <framerate>23.97602</framerate>
        <scantype>progressive</scantype>
        <default>True</default>
        <forced>False</forced>
        <duration>46</duration>
        <durationinseconds>2780</durationinseconds>
      </video>
      <audio>
        <codec>ac3</codec>
        <micodec>ac3</micodec>
        <bitrate>192000</bitrate>
        <language>spa</language>
        <scantype>progressive</scantype>
        <channels>2</channels>
        <samplingrate>48000</samplingrate>
        <default>True</default>
        <forced>False</forced>
      </audio>
      <audio>
        <codec>dts</codec>
        <micodec>dts</micodec>
        <bitrate>1536000</bitrate>
        <language>eng</language>
        <scantype>progressive</scantype>
        <channels>6</channels>
        <samplingrate>48000</samplingrate>
        <default>False</default>
        <forced>False</forced>
      </audio>
      <subtitle>
        <codec>subrip</codec>
        <micodec>subrip</micodec>
        <language>spa</language>
        <scantype>progressive</scantype>
        <default>False</default>
        <forced>False</forced>
      </subtitle>
      <subtitle>
        <codec>subrip</codec>
        <micodec>subrip</micodec>
        <language>eng</language>
        <scantype>progressive</scantype>
        <default>False</default>
        <forced>False</forced>
      </subtitle>
    </streamdetails>
  </fileinfo>
  

Or maybe it's not even possible to send this information to plex.

[rednoah]

FileBot xattr metadata does not include MediaInfo, since that's readily available in the file itself. There are existing scripts for exporting MediaInfo, but there's no standardized way of passing any of that information on to Plex or Jellyfin. You could write your own integration though.

[devster]

Is there a script to convert from one type of xattr.store to another?
In particular I'm thinking about switching to plain text files instead of embedded for a filesystem change.

[rednoah]

--apply metadata can be used to export xattr to .xattr folders and files:
viewtopic.php?t=11079