Match issue due to special characters when file name contains < and >

[starsys]

After several trials, I finally understand why filebot do not recognize my file correctly.
Episode detection error comes from a special character : "<" or ">". When source file contains such character, filebot do not consider the filename at all but a strange random one like "GR2RV~.MKV" (and leads of course to matching issue)

As soon as I remove these characters from original filename, everything runs fine.

I'm using {plex} format.
Is there any solution for ignoring these special characters ?
Is it possible to ignore complete set of characters like " --> Torrent9.uno <--" (ignore pattern in original filename)
And thus help accurate episode detection ?

I'm french. Should I expect any issues with "éèàç..." characters ?

Thanks and congratulation for this great program (I bought the lifetime version

[rednoah]

< and > are not allowed on Windows, so you can't have those file paths in the first place, unless you're using a Linux-based NAS (which isn't mentioned in your OP) then it's allowed there of course, but if you then access those files from Windows via SMB shares, then the SMB share will "normalize" the names so Windows can handle them.

There's nothing FileBot can do on your Windows client, because even Windows itself probably doesn't know the real file path on the NAS.

You can run FileBot on the NAS itself (i.e. the CLI, not the GUI) and then it'll have direct access to the file system and see the real file paths, and not the fake compatibility names as seen through the SMB share.

You can also use Desktop Linux with NFS shares, keeping the pipeline all Linux, so you won't run into Windows limitations.

éèàç are allowed in Windows file paths so those should be fine.

[starsys]

Hello.
Thanks for your feedback.
I'm using filebot running on a synology DS918+.
Input files are in a remote folder (pointing my local internet provider box, where torrents are downloaded. I assume it's also a linux based system) mounted via CIFS in the NAS.

I didn't find if I can mount my box via NFS on my synology.
If you've any idea, thanks in advance.

[rednoah]

AFAIK, if you're on Windows, then SMB is your only option.

Depending on your use case, using FileBot Node running filebot locally on your NAS might work better:
viewtopic.php?f=13&t=1802

[starsys]

I'm not using windows.
I'm using my internet provider Box : "Freebox" for downloading : it's a Linux based system. (https://fr.wikipedia.org/wiki/Freebox#C ... s_internes)
I'm using Filebot node runnning on my Synology DS918+
The download drive of my Freebox is mounted in my NAS via CIFS (I can only link my Freebox to my NAS via CIFS or FTP, I cannot connect it via NFS)

I've seen that maybe editing file smb.conf would help (avoiding mangle and changing charset). Should I try ?
https://serverfault.com/questions/12461 ... -filenames

Thanks.

[rednoah]

I see. Well, CIFS is a Windows thing so it probably defaults to Windows compatibility options.

There could be options to make it work as expected. Worth a try. Please report back on your findings.

[starsys]

I tried to edit smb.conf with different charset : it doesn't work.
I tried to change an advanced SMB setting in my NAS (It really seemed to match my problem) :
"Enable VFS module to convert Mac special characters: Select to convert SMB-reserved characters (e.g. < > " * ?) in filenames to Basic Latin during file creation. Such files will be recognizable via AFP" : it doesn't work

So my original input file looks like something like "my-file-name--> Torrent9.uno <--.mkv" on my freebox and is transformed to "GRVGK2~W.MKV" on my NAS...
I'm going crazy ! Why I cannot just remove / transform / ignore this fuc*ing characters !

I'll try to contact synology and understand why the special characters options doesn't work

[starsys]

By the way : I just updated my NAS with the last firmware version : filebot node seems not to work anymore (Synology error window)

[starsys]

I had to repair the package. Now I can launch the filebot node but it get stucked in this state :
Locking /volume1/@appstore/filebot-node/data/filebot.log

and nothing happens

[rednoah]

That's fine. You might be running multiple instances of FileBot at the same time, and so it'll wait for previously invoked instances to finish first.

[rednoah]

My best guess is that the file names get mangled on the remote server, and so there's not much any device on your end can do about it:
https://www.oreilly.com/openbook/samba/ ... 05_04.html

You'll want to talk to however manages your Freebox thing rather than your Synology NAS.

[starsys]

Hello. Thanks for the info, really interesting.
Unfortunatly, NFS is not enabled in my Freebox : so I cannot use this protocol, and I've no access to its SMB.conf file.
Otherwise I would have tried this option :

Code: Select all

[data]
	mangled names = no

I think this setting has to be applied from the server side (My Freebox) not from client (My NAS) ?

[starsys]

I still have problem with filebot node since the last synology update.
I stuck for hours (no update):

Code: Select all

filebot -script 'fn:amc' '/volume1/freebox/disque/Téléchargements/' --output /volume1/freebox/disque --action TEST -non-strict --order Airdate --conflict auto --lang fr --def 'music=y' 'unsorted=y' 'clean=y' 'skipExtract=y' 'seriesFormat=series/{plex.tail}' 'movieFormat=films/{plex.name}' 'unsortedFormat=films/{plex.name}' 'excludeList=.excludes' --log all --log-file '/volume1/@appstore/filebot-node/data/filebot.log'

------------------------------------------

Locking /volume1/@appstore/filebot-node/data/filebot.log
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/volume1/@appstore/filebot/jar/groovy.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Run script [fn:amc] at [Wed May 15 09:47:09 CEST 2019]
Parameter: music = y
Parameter: unsorted = y
Parameter: clean = y
Parameter: skipExtract = y
Parameter: seriesFormat = series/{plex.tail}
Parameter: movieFormat = films/{plex.name}
Parameter: unsortedFormat = films/{plex.name}
Parameter: excludeList = .excludes
Argument[0]: /volume1/freebox/disque/Téléchargements
Use excludes: /volume1/freebox/disque/.excludes (5)

Nothing happens for more than 30 minutes........................

I also noticed that system detect a "OS: Linux (amd64)" whereas my NAS is using a Intel Celeron

Code: Select all

filebot -script 'fn:sysinfo'

------------------------------------------

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/volume1/@appstore/filebot/jar/groovy.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
FileBot 4.8.5 (r6224)
JNA Native: 5.2.2
MediaInfo: 17.12
7-Zip-JBinding: 9.20
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2019-05-05 (r564)
Groovy: 2.5.6
JRE: OpenJDK Runtime Environment 12.0.1
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 2 GB Max Memory / 37 MB Used Memory
OS: Linux (amd64)
HW: Linux DS918 4.4.59+ #24922 SMP PREEMPT Fri May 10 02:59:42 CST 2019 x86_64 GNU/Linux synology_apollolake_918+
DATA: /volume1/@appstore/filebot/data/admin
Package: SPK
License: FileBot License PX7914368 (Valid-Until: 2069-05-13)
Done ヾ(＠⌒ー⌒＠)ノ
------------------------------------------

[Process completed]

Is it normal ?

[rednoah]

1.

Nothing happens for more than 30 minutes........................

Is it normal ?

Not at all. No idea why it might get stuck at this point though, as not much is happening. I reckon that some file system operation (which is usually fairly instant) is somehow taking a very long time.

You could try running things via SSH yourself, to see if you can reproduce the issue. You can also use strace to track file system calls, which might help you figure out where exactly it's stuck:
https://linux.die.net/man/1/strace

You can also enable more logging in FileBot:
viewtopic.php?f=3&t=3913

But in this case my educated guess is definitely on blocking IO blocking things, so I gather it's a network filesystem issue rather than a filebot issue, so extra filebot logging might not help much. Worth a shot thing. See if gives us more fine-grained information as to what it's doing and where exactly it's stuck.


2.

I think this setting has to be applied from the server side (My Freebox) not from client (My NAS) ?

Yes, your local NAS probably only ever sees the mangled names, and has no way of knowing the unmangled names.

[starsys]

Can the CPU wrong detection (AMD instead of Intel) be an issue ?
I deleted my complete input folder and created it again (it seems it was full of corrupted hardlinks).
Now the log go a bit further but I still have an issue :

Code: Select all

Rename movies using [TheMovieDB]
Auto-detect movie from context: [/volume1/freebox/disque/Téléchargements/[ Torrent9.NZ ] La.Derniere.Folie.de.Claire.Darling.2018.FRENCH.1080p.WEB.H264-EXTREME.mkv]
CmdlineException: Failed to identify or process any files
Processing 1 unsorted files
[TEST] from [/volume1/freebox/disque/Téléchargements/[ Torrent9.NZ ] La.Derniere.Folie.de.Claire.Darling.2018.FRENCH.1080p.WEB.H264-EXTREME.mkv] to [/volume2/temp/Unsorted/freebox/disque/Téléchargements/[ Torrent9.NZ ] La.Derniere.Folie.de.Claire.Darling.2018.FRENCH.1080p.WEB.H264-EXTREME.mkv]

It seems no information is found for this movie whereas the windows version of filebot perfectly match and rename "Claire darling (2019)".
What is the search difference between CLI and windows version of Filebot ? Thanks.

[rednoah]

1.

Can the CPU wrong detection (AMD instead of Intel) be an issue ?

The CPU architecture is correctly displayed as amd64 which is a CPU instruction set originally invented by AMD and now adopted by all modern Intel and AMD CPUs:
https://en.wikipedia.org/wiki/AMD64


2.

I deleted my complete input folder and created it again (it seems it was full of corrupted hardlinks).

You can have corrupted data. You can't have corrupted or broken hardlinks. The concept you think of as "file" is actually a hardlink, so files and hardlinks are quite literally exactly the same thing.

You can have broken symlinks though.


3.
IDK. Works for me:

Code: Select all

[TEST] from [volume1/freebox/disque/Téléchargements/[ Torrent9.NZ ] La.Derniere.Folie.de.Claire.Darling.2018.FRENCH.1080p.WEB.H264-EXTREME.mp4] to [Movies/Claire Darling (2019)/Claire Darling (2019).mp4]

Make sure you're using the latest version. Other than that, only trial and error can help you figure out what's different.

Adding additional logging might help you figure things out: viewtopic.php?f=3&t=3913



Please create new threads for new issues unrelated to the OP: viewtopic.php?f=3&t=1868

[starsys]

I changed nothing and today it works !
Really strange...