Metadata and Extended Attributes

How-to guides, frequently asked questions, not-so-obvious features, etc
Post Reply
User avatar
rednoah
The Source
Posts: 14363
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Metadata and Extended Attributes

Post by rednoah » 24 Oct 2012, 11:16

FileBot r1255 adds support for using the new Java 7 FileAttribute interfaces to store metadata in the file attributes. That's NOT TAGS that are part of the filedata but filesystem attributes, so they might just get lost if you move files to a filesystem that doesn't support extended attributes.

Details:
* [xattr:metadata] Episode or movie object (as json)
* [xattr:filename] Original filename
* [basic:creationTime] Episode airdate or Movie release year (disabled by default for compatibility reasons)


Usage:
FileBot and other programs have the original filename and all episode/movie info regardless how the file is renamed. Sure nice to have that data available should you need it.


Enable / Disable this feature via Java System Properties:

Code: Select all

-DuseExtendedFileAttributes=[true|false]

FileBot/Groovy script for reading xattr from all files:

Code: Select all

filebot -script fn:xattr 
@see viewtopic.php?f=4&t=5#p5394
:idea: Please read the FAQ and How to Request Help.

User avatar
rednoah
The Source
Posts: 14363
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Usage Examples

Post by rednoah » 05 Apr 2017, 09:27

1. Rename files with FileBot:

Code: Select all

$ filebot -rename *.mp4 --db TheMovieDB -non-strict
Rename movies using [TheMovieDB]
Auto-detect movie from context: [avatar.mp4]
[MOVE] From [avatar.mp4] to [Avatar (2009).mp4]
Processed 1 files
2. Confirm that FileBot metadata has been stored as xattr in the filesystem:

Code: Select all

$ filebot -script fn:xattr *.mp4
/path/to/Avatar (2009).mp4
	net.filebot.filename: avatar.mp4
	net.filebot.metadata: {"@type":"Movie","year":2009,"imdbId":499549,"tmdbId":19995,"language":"en","id":19995,"name":"Avatar","aliasNames":[]}
3. Use xattr metadata in filter and format expressions:

Code: Select all

$ filebot -mediainfo -r . --filter "y >= 2000" --format "{n} by {director}"
Avatar by James Cameron
4. Process xattr tagged files based on existing xattr metadata:

Code: Select all

filebot -rename -r . --db xattr --action copy --output /Volumes/USB --filter "episode && age < 30" --format "{n} - {sxe} - {t}"
:idea: Please read the FAQ and How to Request Help.

oneguynick
Posts: 7
Joined: 06 Jan 2018, 13:45

Re: Metadata and Extended Attributes

Post by oneguynick » 07 Jan 2018, 12:58

Code: Select all

filebot -rename *.mp4 --db TVDB -non-strict
Illegal Argument: java.nio.file.InvalidPathException: Illegal char <*> at index 0: *.mp4 (*.mp4)
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
Failed to read xattr: InvalidPathException: Illegal char <*> at index 0: *.mp4
^C

Example filename: Penoza - S01E02 - Unpleasant Surprises HDTV-720p.mkv

User avatar
rednoah
The Source
Posts: 14363
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Metadata and Extended Attributes

Post by rednoah » 07 Jan 2018, 13:08

*.mp4 refers to all mp4 files. If you're testing with mkv files then *.mkv will select those.

@see https://mywiki.wooledge.org/glob
:idea: Please read the FAQ and How to Request Help.

oneguynick
Posts: 7
Joined: 06 Jan 2018, 13:45

Re: Metadata and Extended Attributes

Post by oneguynick » 08 Jan 2018, 10:36

My mistake. I forgot they were *.mkv, good catch. With that said I am now testing the same working commands against my samba share with the same issue. Copy locally just isn't an option. Is there a better way to do this? NFS or SFTP or something?

oneguynick
Posts: 7
Joined: 06 Jan 2018, 13:45

Re: Metadata and Extended Attributes

Post by oneguynick » 08 Jan 2018, 10:58

Alright so I took the network share out of the picture for troubleshooting and tried docker:

Code: Select all

docker run -it -v /tv:/volume1 -v /opt/docker/filebot:/data rednoah/filebot -script fn:xattr -r *
docker run -it -v /tv:/volume1 -v /opt/docker/filebot:/data rednoah/filebot -script fn:duplicates -r --action test *
Done ヾ(@⌒ー⌒@)ノ in both cases. I specifically copied a folder of shows to ensure there were exact duplicates.

I had considered that maybe it was the ZFS pool. I verified the pool is enabled for xattr:

Code: Select all

NAME                                                    PROPERTY  VALUE  SOURCE
share                                                   xattr     on     default

User avatar
rednoah
The Source
Posts: 14363
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Metadata and Extended Attributes

Post by rednoah » 08 Jan 2018, 17:03

NFS might work. SFTP probably doesn't preserve attributes, but RSYNC might (with --xattrs option).

If you're using docker with some host folder mounted into the docker, then there's a good chance xattr won't work. I remember filesystem events not working either. Not sure if that's something that can be enabled or if it's just not supported.

:idea: If you care about xattr, then the best solution is usually to process files on the NAS or server where reading/writing xattr usually works out of the box.

:arrow: FileBot generally doesn't require xattr, it just stores some xattr info if possible, but will work just fine without. It's just that the duplicates script requires them to work, so it'll instantly know which file is which movie/episode without having to bother any web services.
:idea: Please read the FAQ and How to Request Help.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests