Restore multi-episode match from xattr

All about user-defined episode / movie / file name format expressions
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Restore multi-episode match from xattr

Post by HellPhantom »

I have been trying to cope with plex dual playing of episodes by not naming my epsiodes exaclty like Plex wants, but at least have all the episode numbers and some information
in the filename less than a certain length.

My real problem is after I applied my original filenames and everything works the way I want I seem to have strayed to far away from what filebot can recognize as multi episodes and I can not run filebot on them again without loosing my multi episodes. Over the years I have altered or updated already named movies TV show many times and I want less intervention on multi episodes.

I do not want multi epsiodes to play multiple times from the start and I am no longer bothered that plex do not display all episodes, I have given up that dream and
plex is not going to ever change. I have settled on showing only the first episode of the bunch and having all info in filename.

I name my episodes like so: ( I use a _ instead of the - and omit the E)

Code: Select all

Spy vs. Spy - S01E01_02 - Bomb & Fake Dynamite (2010-09-06).[1080p.AVC.1.181Kbps.30.000Fps].[2.0.AAC].[eng]
Spy vs. Spy - S01E01_26 - Bomb & Fake Dynamite & Microbombs & Submarine & Splitting Torpedo & Dynamite & Elephant Ta (2010-09-06).[1080p.AVC.1.181Kbps.30.000Fps].[2.0.AAC].[eng]
This works the way I want in plex. it will play the episode only once and show as the first episode.

When I remember or catch it I run an upfront rename back from _ to -E before using filebot but I would like to eliminate this step before using filebot on consecutive renames. maybe there is a way to tell or make filebot aware of my format. O and I only use filebot online UI.

I have tried other acceptable formats that work with filebot but not plex but normally when it works for filebot it works for plex. also it must stay short
these don't work the way I want:

Code: Select all

 S01E01-02
 S01E01_E02
maybe someone does what I do with a simple solution.
I have read many of the other multi episode threads but it does not seem to be what I need.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Multi Episode tv shows & Plex

Post by rednoah »

:?: Do your files have xattr metadata?

:idea: FileBot will store xattr metadata when you rename files. So you can restore the match from there instantly by loading files and pressing F3 in the Desktop application or by using --db xattr in the command-line tool.

:arrow: Please see Plain File Mode and Metadata and Extended Attributes for details.



EDIT:
HellPhantom wrote: 30 Nov 2021, 07:24 O and I only use filebot online UI.
You mean the FileBot Node WebUI for Synology NAS and QNAP NAS? I'd use the Desktop application for tricky files. The FileBot Node WebUI is somewhat limited, when it comes to working with xattr metadata. The filebot command-line tool does have --db xattr but the FileBot Node WebUI doesn't have a UI for that.
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Multi Episode tv shows & Plex

Post by HellPhantom »

I am using the Desktop application for windows. I meant I am not using command line.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Multi Episode tv shows & Plex

Post by rednoah »

:?: Do your files have xattr metadata?

Image



If so, then you can restore the match from xattr metadata irregardless of the current file name, by pressing F3:

Image



Your custom format can then transmogrify the multi-episode information in any way you want. You don't need to worry about the file name at all, because FileBot can just read the information from the xattr metadata without even glancing at the file name.
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Multi Episode tv shows & Plex

Post by HellPhantom »

Ok, Interesting, I am trying to process this. it seems helpful. did not know the F3 and never used these xattr before, I am used to metadata with Mp3tag.

Like luck would have it most of my tv show does have xattr metadata, except my example SPy vs Spy :-)
I am investigating why? I have just renamed them fresh and still they did not have it, but let me see if I can make this work for me.
Last edited by HellPhantom on 30 Nov 2021, 08:00, edited 1 time in total.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Multi Episode tv shows & Plex

Post by rednoah »

tl;dr Use on Episode Mode ➔ TheTVDB when processing files for the first time. Use Smart Mode ➔ Attributes all subsequent times. The latter is instant, perfect (i.e. guaranteed same match as last time as stored to xattr metadata) and offline.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Multi Episode tv shows & Plex

Post by rednoah »

Extended Attributes are a file system feature of modern file systems, e.g. NTFS Alternative Streams on Windows. However, most users and programs are unaware of them, so they get lost easily, e.g. moving files to a file system that doesn't support NTFS Alternative Streams or some program (i.e. almost all programs that aren't Windows Explorer) copying the file but neglecting to copy the associated NTFS Alternative Streams as well.


If this is something you're worried about, then you'll want to configure FileBot to store xattr metadata to plain/text files instead of NTFS Alternative Streams.
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Multi Episode tv shows & Plex

Post by HellPhantom »

Excellent feature, no more worries when re-renaming ton's of movies that they will be matched incorrectly. Just update the naming with existing info Brilliant.

So far this would be perfect for 99% of tv shows, except Spy VS SPY which has 26 very short episodes in one episodes. it seems to stop working after 6 episodes max.
But I have not read everything or had time to play around enough with this.

I have saved xattr files from the "post processing" before but I did not fully understand why or what to do with it. I Will consider Using it.
I already embed many meta data points (collections, alternate titles and some genre and especially images) into my movies and shows because I am so tired of Plex making changes to agents and what not and then you loose all your hard work, your custom data is not precious to them and they have absolutely no effort to preserve them. It ends up being a great player but not so much a reliable safe organizer.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Multi Episode tv shows & Plex

Post by rednoah »

HellPhantom wrote: 30 Nov 2021, 08:25 So far this would be perfect for 99% of tv shows, except Spy VS SPY which has 26 very short episodes in one episodes. it seems to stop working after 6 episodes max.
Although xattr may have a physical limit, maybe as low as 4k on some platforms, it would either work or not work at all. So the match that was written to xattr likely was only a 6 episode multi-episode object.


I tried your sample file, and restoring a 26 multi-episode object from NTFS Alternative Streams works out-of-the-box:
Image
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

I have not found my bug yet but I have made copies of same file manipulated the amount of episodes in filebot it needs to match to but every time above x it no longer shows.
I chaged the 4 copies to have different numbers of episodes

1. 1-6 (works)
2. 1-7
3. 7-11 (works)
4. 7-12

Image

you can see in image the first and third one works. I don't know if my formatting expression can affect what is saved in xattr. I thought maybe the filename was more than 256 and it could be it but it is not, I also keep filenames lower than 256 by limiting the description length. I will create this scenario also on a different tv episode like flash to eliminate show and specific files and different HD.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Restore multi-episode match from xattr

Post by rednoah »

Yep, that means that 2 of the files just don't have any xattr at all. The xattr metadata is stored when you click Rename and the value is unrelated to whatever custom format you may or may not be using.


The log may or may not shed light on the issue:
viewtopic.php?t=1868


You can use the command-line to view raw xattr:
viewtopic.php?p=5394#p5394

Code: Select all

filebot -script fn:xattr /path/to/files

On an unrelated note, I made a video tutorial inspired by this thread:
https://www.youtube.com/watch?v=54h_Fb06drI
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

I created a flash episode containing all 23 episodes of season 01 for test, The xattr is not working on actual episode but if i look inside the file "net.filebot.metadata" created in folder
I can see the episode info is there up to episode 23. I just copied the last bit, so it is not nicely formatted but it is in there

Code: Select all

(2014)","season":1,"episode":23,"title":"Fast Enough","absolute":23,"airdate":{"year":2015,"month":5,"day":19},"id":5166511,"seriesInfo":{"database":"TheTVDB","order":"Airdate","language":"en","type":"TV Series","id":279121,"name":"The Flash (2014)"
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Restore multi-episode match from xattr

Post by rednoah »

Note that the .xattr folders are not relevant unless FileBot is explicitly configured to use them. By default, on Windows, FileBot will store xattr metadata into NTFS Alternative Streams.
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

I just want to drop some info before i continue with anything else:

System Info
Image

Log of Last episode i tried to create.
Image
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

I have moved the file to a shorter folder path in b:\1 and applied flash episodes 1-23 and it saved and works correctly under b:\1.

Code: Select all

b:\1\Flash (2014) - S01E01-E23 - Pilot & Fastest Man Alive & Things You Can't Outrun & Going Rogue & Plastique & Flash Is B (2014-10-07).[720p.x265.0.597Kbps.23.976Fps].[2.0.AAC].[eng].[eng]
I then copied the file back to where it needs to be with longer folder path

Code: Select all

f:\Multimedia\TV\Flash (2014) {tvdb-279121}\Season 01\
now the xattr no longer reads or works. I also cant apply it successfully in this longer folder. It must have something to do with either longer file path+name or maybe the path is not properly quoted in code. not sure yet.

If i copy the file back to b:\1 or any other shorter folder name it still works so it also does not break during copy.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Restore multi-episode match from xattr

Post by rednoah »

Looks like there are remote network drives in the mix. Is your F: drive a local drive or a remote drive? I'd run tests on a local NTFS drive first, e.g. on your Desktop. If you're using remote network drives via SMB (Windows incorrectly identifies those as NTFS too) then all bets are off as far as xattr metadata is concerned. Typically, xattr don't work at all in, depending on the SMB configuration on the remote server, and it's certainly conceivable that a xattr size limit is configured.


:idea: If you're using remote network drives, then it's best to configure FileBot to use .xattr folders instead of native file system extended attributes. Please read Optimizations for Remote File Systems for details.
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

No, they are all internal local drives. If I shorten path on F: or original J: it still works.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Restore multi-episode match from xattr

Post by rednoah »

HellPhantom wrote: 30 Nov 2021, 10:15 If i copy the file back to b:\1 or any other shorter folder name it still works so it also does not break during copy.
That could be the reason. However, do keep in mind that FileBot will cache xattr metadata in memory, so if the file path and the last-modified date don't change (e.g. by moving the file away and back again) then there's the possibility that FileBot isn't reading the information from the file system at all.


You definitely want to use the filebot -script fn:xattr command (each call has it's own process, so we don't need to worry about in-memory caching) to check xattr:
viewtopic.php?p=5394#p5394
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

Ok.

Filename:

Code: Select all

Flash (2014) - S01E01-E23 - Pilot & Fastest Man Alive & Things You Can't Outrun & Going Rogue & Plastique & Flash Is B (2014-10-07).[720p.x265.0.597Kbps.23.976Fps].[2.0.AAC].[eng].[eng]2

with this path on up to "s" it works

Code: Select all

b:\1\Multimedia\TV\Flash (2014) {tvdb-279121}\s
adding "e" to get to season 01 no more work.

Code: Select all

b:\1\Multimedia\TV\Flash (2014) {tvdb-279121}\se
It is probably total length path+filename exceeding 256. My files still works when exceeding 256 but many programs start giving issues so I keep them smaller including path
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Restore multi-episode match from xattr

Post by rednoah »

What happens if you configure FileBot to use .xattr folders as default storage engine for metadata?

Code: Select all

filebot -script fn:properties --def net.filebot.xattr.store=.xattr
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

OK.

With this setting flash xattr only reads or works when this episode is accompanied by its hidden .xatttr folder. I have tested this by removing this folder and moving it back.

with this setting Flash works in its original folder with their original folder lengths, it creates that hidden .xattr folder and the files inside. it is better with the folder method and filebot using it from folder but not perfect yet.

With Spy vs spy it does not work because the "Spy vs. Spy (2010) {tvdb-199861}" is longer than Flash name on folder and on each episode.

So I can copy flash to spy vs spy folder on J: along with its hidden xatrr folder and it no longer works. Total commander which a use to manage storage notifies me of the total name being longer than 259 it is actually 272 but this is not the actual episode that is to long but the xattr files inside the hidden .xattr folder that exceeds the limit.

Image

If I copy flash episode out to a slightly shorter folder again like say only "j:\Multimedia\TV\1\" they both work. as long as the hidden folder accompanies the file and the hidden files don't exceed 259.

Prob is the hidden folder and its subfolder of episode and then the file insode also adds X amount of character limiting the original filename you can use by quite a bit. another 8 characters i think

"net.filebot.metadata".
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

I have not made up my mind about what is best, having the xattr data stored separately or combined but something in the combined also runs into this length limit.

Does the combine store it inside the actual episode so I can copy it to another machine as long as it is on ntfs and you don't break it during copy.
The internal method should really be able to handle files up to the max of 259.
I understand the folder method adds additional folder and its files but internal is internal.

When I add run those config lines in the F5 Groovy pad is it permanently configured or only for that session.?
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

how do I undo this command so it no longer creates the folder .xattr

Code: Select all

filebot -script fn:properties --def net.filebot.xattr.store=.xattr
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Restore multi-episode match from xattr

Post by rednoah »

An empty value you should do the trick:

Code: Select all

filebot -script fn:properties --def net.filebot.xattr.store=
You can also manually edit the configuration file. Please read the console output of the command above to find the location of the configuration file.
:idea: Please read the FAQ and How to Request Help.
HellPhantom
Posts: 49
Joined: 15 Apr 2020, 10:45

Re: Restore multi-episode match from xattr

Post by HellPhantom »

Shot I got it. Thanks

For now I will try to work within the limits of the "native filesystem xattr"

Before I venture into the ".xattr folders" I need to have a better automated way to keep these hidden folders in sync because if I manually rename a file I also have to remember to manually rename its xattr counterpart folder name inside the hidden .xattr folder at the very least for .xattr data to stay intact. and to be 100% complete also have to change internally the "net.filebot.filename" so the data matches what I made the new name. My system is not ready to handle the folders, i will surely get these folders messed up and orphaned.

Still this is a great feature that will make my life tons easier. I also checked out your inspired tutorial.

Maybe this very useful and time saving feature or rather usage of it in this way is not advertised enough. Every body at some point has to revisit existing Files and I always re matched them and this inevitably leads to new problems or mismatches when working in huge batches.

Excellent work I don't know where I would be without Filebot and Mp3tag.
Post Reply