cmd queries / evaluating

[xhAJHUYe6uc]

Hi!

I am in the process of evaluating if filebot can fit my needs. Looking through the scripts and commands (only headless / terminal usage), I have some questions, is there any way to:

- query the internal database? examples: query db to determine if a candidate has been "seen", "What file(s) in this folder is not renamed / watched?" (No idea if internal db is used/whats used)
- export e.g. AMC script "unsorted=Y|N" as a result / log rather than moving data (looks like I need to script this)
- any way to set / tweak the "distance" for a match to occur other than strict non strict?


Come to think about it, I'm not sure what script I should use for my use case. I'm purely looking for a way to rename (by hard/soft link) if "highly confident", else make a record of that so my hooman slave can check it manually later. I am not looking for any metadata etc. "unsorted=Y|N" wont work (unless I script around fb features. I think).

Any insights / tips are appreciated

[rednoah]

1.
FileBot does not have an internal database. You can do pretty much any kind of testing without a license though, by using --action TEST (i.e. you can test if files get matched correctly, but you can't rename / move files).


1.1.
Once files are renamed (requires a license) FileBot will store metadata to xattr, so although there is no "internal database" the files themselves plus xattr will effectively be that, a way to get full metadata for each file instantly and offline:
viewtopic.php?t=324
viewtopic.php?t=4788


2.
You'd typically be using the same file system for input / output so that any move or hardlink operation is always instant. You don't want files left behind, as to avoid infinite loops when files get picked up over and over and over. In this case, moving files is cheap, and once files are moved, you have them all in one place, and if you need a text file with file paths for some reason, then you can just generate it on demand.


3.
There is no way to set / tweak "distance" because FileBot doesn't really work like that internally. The choice between strict and non-strict results in different algorithms being chosen and different code paths being followed so the concept of "distance" applies even less in this context.


4.
You'll probably want to use simple filebot -rename calls in a bash script, loop over each file, call filebot on each file, check the exit code for success or failure, proceed accordingly.

[xhAJHUYe6uc]

Many thanks for replying.

1) Considering data is on a read only system (move or rw is not an option unfortunately), storing to .xattr seems like an option. I can do some trickery with mergerfs so writing to .xattr is doable as I am pretty sure that xattr edit to original file would essentially trigger copy from read only system to rw system.

-If I understand you correctly does this allow multiple instances of Filebot (on different hosts) to work together due to .xattr ? F. ex: Instance A did rename, Instance B later renamed to some other scheme. (not a likely scenario, just trying to get some understanding of how it works )

2) Unfortunately not doable for my scenario. I have to have the data on read only file system A, and any sorting, by means of hard or soft links, on file system B. Note, file system A regularly gets updated with new entries. I was hoping that "excludes" could essentially filter out seen entries, but that's maybe only available for AMC? Could be scripted though.

[rednoah]

1.
Yes, xattr is a general purpose file system feature. It can be used by FileBot or other apps.

For compatibility reasons (e.g. processing files on a NAS via local file system, then again reading them remotely via SMB remote filesystem interface) configuring FileBot to use .xattr folders (i.e. store metadata in plain files) is generally a good idea.


2.
If you're using the amc script with the --def excludeList option, then that will indeed take care of processing files only once, simply by having a list of paths that have already been processed. It's simple but effective.


3.
hardlink / symlink operations are effectively instant, so you're good on that front. Though I'm not sure how hardlink would be possible with mergerfs or read-only filesystems (if it's just ro file & folder permissions, then you can of course create hardlinks on the same file system, by adjusting permissions for the output folder only).

[xhAJHUYe6uc]

3) Its not without exceptions, but a good read about it here https://github.com/trapexit/mergerfs#rename--link (rename & link paragraph) should you be curious.

I assume fn:properties is persistent so it only needs to be called once to be set?

Looks like Filebot can fit my use case assuming the strict matching works for a fair percentage of the data. Anyway, I'll get a license anyway just for you taking your time to answer here and so I can test it all. -cheap anyways so

thank you,

[xhAJHUYe6uc]

One more question. Just bought a license, installed it and saw that filebot defaults to "$HOME/.filebot". I tried to change the data location by:

Code: Select all

filebot -script fn:preferences --output SOMEOTHERPATH

, but I realized that was only for the preferences file. How can I tell Filebot to use some other directory than "$HOME/.filebot" ?

[rednoah]

e.g. do-stuff-with-filebot.sh

Code: Select all

#!/bin/sh
export HOME=/my/other/home
filebot ...

[xhAJHUYe6uc]

Aaaahhhh, well, thats one way to get around it

In case someone else reads this thread in future, same, just executed in a safer manner:

Code: Select all

HOME=/my/other/home filebot --license PATH

Will generate the directory at /my/other/home

I will however make a feature request for FILEBOT_HOME variable later

CORRECTION: is Filebot trying to read XDG_CONFIG_HOME ? I got two different locations as a result on two different systems: $HOME/.filebot and $HOME/.config/filebot

[rednoah]

Well, there's a FileBot-specific solution that works only for FileBot:

Code: Select all

export FILEBOT_OPTS="-Dapplication.dir=/home"

And a Java-specific solution which will work for all Java application:

Code: Select all

export JAVA_OPTS="-Duser.home=/home"

But IMHO a universal solution that'll work for everything and doesn't require Java or FileBot specific knowledge is best unless you need something more specific for some reason.


EDIT:

FileBot does not use XDG_* variables. But you can of course use them to set up the environment before calling FileBot according to your preferences.

[xhAJHUYe6uc]

Many thanks!

I'm almost there now.

I noticed that Filebot tries to store .xattr file with the original file, rather than storing the .xattr at the target folder (with the symlink). Is there any way to to achieve intended behaviour?

[rednoah]

I noticed that Filebot tries to store .xattr file with the original file, rather than storing the .xattr at the target folder (with the symlink). Is there any way to to achieve intended behaviour?

What exactly is the intended behaviour?


EDIT:

Ah. Perhaps you mean it's following symlinks? i.e. the newly created symlinks and thus adding xattr to the source file?

That's intended behaviour, because that's how native xattr works as well, you could specify your own absolute path for all xattr files though:

Code: Select all

net.filebot.xattr.store=/absolute/path/to/XattrStore

[xhAJHUYe6uc]

Ah. Perhaps you mean it's following symlinks? i.e. the newly created symlinks and thus adding xattr to the source file?

Yep!

I'm not sure how exactly xattr mechanics works and how Filebot and other 3rd party tools read this folder when its not stored with the file itself. I'm presuming that following symlink and storing xattr there works the same as storing the xattr without following symlinks.

F. ex: Folder B has Symlink to file X, in folder Z.

So storing xattr in Folder Z vs Folder B