String replacement

[UKenGB]

I have a number of files whose names end in ' (B&W)' or ' (Colour)' which I want to replace with ' - B&W' and ' - Colour' respectively. Simple with sed etc, but I cannot make anything work in Filebot.

I have 2 problems. It appears that one cannot use back references like '\1' and also I cannot match on '(' or ')' as they are treated as the special character, so it complains about unmatched brackets etc. I've tried to escape them '' or "" or with \ and all combinations thereof, but so far nothing I've tried has worked.

So, how can I replace bracketed text with something else, hopefully including what was previously inside the brackets?

[rednoah]

Java/Groovy syntax might be slightly different from what you're used to, but the concepts are the same:

Code: Select all

'Test (B&W)'.replaceAll(/\((.+?)\)/, '- $1')

FileBot has a built-in helper for this use case which makes things a bit more readable:

Code: Select all

'Test (B&W)'.replaceTrailingBrackets(' - $1')

[UKenGB]

Brilliant. I'd figured out the use of slashes by looking at the examples, but using $1 as the back reference will really help. Does that work in the general sense of each pair of () can be referenced as $1, $2 etc as per normal regex using \1, \2 etc.

Not previously even heard of Groovy. Just thought you were being hip

[UKenGB]

Something has just come up. I wanted to rename a bunch of .m4b files to .m4a. Aha, Filebot I thought. But couldn't get it to work. Turning on Extension Override and just using {fn} unsurprisingly resulted in Missing extension errors, so {fn}.m4a ought to work.

Except it didn't. It added .m4a twice. Made no sense to me why it would do that. Bug? In the end I gave up and just used the command line with 'find' etc. Shame as I thought it would be easy with Filebot.

[rednoah]

$1 will refer to the first capturing group (...) and so on.

Here's the complete technical documentation:
https://docs.oracle.com/javase/8/docs/a ... ttern.html

If you need more complex replacement logic, then String.replaceAll(CharSequence regex, Closure closure) is your friend.

[rednoah]

Extension: Override seems to have preview issues, showing the extension twice, but it's actually working when you click Rename. I'll have a look.

[JamesBlond007]

Hey guys,

i've a new problem. Hope this is the right Thread.
I now got the automated JDownloaderScript working and i tried it out with an episode that has a "ä" in it. So i got an Error:

Code: Select all

Input: /media/disc/Downloads/B S11E01 B u d L i d L T 1 G D W x-SOF/B S11E01 B u d L i d L T 1 G D W x-SOF/Bones.S11E01.Bones.und.die.Loyalitaet.in.der.Luege..Teil.1.GERMAN.DUBBED.WebHDRiP.x264-SOF/sof-the-bones-s11e01.mkv
Group: [tvs:bones] => [sof-the-bones-s11e01.mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Bones, the bones]
Fetching episode data for [Bones]
Fetching episode data for [El Barco]
Fetching episode data for [The Boys]
[MOVE] Rename [/media/disc/Downloads/B S11E01 B u d L i d L T 1 G D W x-SOF/B S11E01 B u d L i d L T 1 G D W x-SOF/Bones.S11E01.Bones.und.die.Loyalitaet.in.der.Luege..Teil.1.GERMAN.DUBBED.WebHDRiP.x264-SOF/sof-the-bones-s11e01.mkv] to [/media/disc/Serien/Bones/Staffel 11/Bones - 11x01 - Bones und die Loyalität in der Lüge (1).mkv]
Processed 0 files
Malformed input or input contains unmappable characters: /media/disc/Serien/Bones/Staffel 11/Bones - 11x01 - Bones und die Loyalität in der Lüge (1).mkv
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /media/disc/Serien/Bones/Staffel 11/Bones - 11x01 - Bones und die Loyalität in der Lüge (1).mkv
	at net.filebot.util.FileUtilities.moveRename(FileUtilities.java:79)
	at net.filebot.StandardRenameAction$1.rename(StandardRenameAction.java:17)
	at net.filebot.cli.CmdlineOperations.renameAll(CmdlineOperations.java:625)
	at net.filebot.cli.CmdlineOperations.renameSeries(CmdlineOperations.java:255)
	at net.filebot.cli.CmdlineOperations.rename(CmdlineOperations.java:110)
	at net.filebot.cli.ScriptShellBaseClass.rename(ScriptShellBaseClass.java:336)
	at Script1$_run_closure55.doCall(Script1.groovy:417)
	at Script1.run(Script1.groovy:403)
	at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:62)
	at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:72)
	at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:114)
	at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
	at net.filebot.Main.main(Main.java:120)
Processing 1 unsorted files
[MOVE] Rename [/media/disc/Downloads/B S11E01 B u d L i d L T 1 G D W x-SOF/B S11E01 B u d L i d L T 1 G D W x-SOF/Bones.S11E01.Bones.und.die.Loyalitaet.in.der.Luege..Teil.1.GERMAN.DUBBED.WebHDRiP.x264-SOF/sof-the-bones-s11e01.mkv] to [/media/disc/Unsorted/B S11E01 B u d L i d L T 1 G D W x-SOF/B S11E01 B u d L i d L T 1 G D W x-SOF/Bones.S11E01.Bones.und.die.Loyalitaet.in.der.Luege..Teil.1.GERMAN.DUBBED.WebHDRiP.x264-SOF/sof-the-bones-s11e01.mkv]
Processed 1 files

Then i saw the replaceAll function and thought i can extend all name and titlestrings with this:

Code: Select all

.replaceAll(/[`´‘’ʻ]/, /'/).replaceAll(/[!?.]+$/).replacePart(', Part $1').replaceAll('ä', 'ae').replaceAll('ü', 'ue').replaceAll('ö', 'oe')

i also tried only with this. The result is the same. I'll get no output.

Code: Select all

.replaceAll('ä', 'ae').replaceAll('ü', 'ue').replaceAll('ö', 'oe')

That's the resultlog:

Code: Select all

Parameter: seriesFormat = /media/41462120-ca5d-4cd0-8a29-6cb1db63d54b/Serien/{n}/{episode.special ? 'Special' : 'Staffel '+s}/{n} - {episode.special ? 'S00E'+special.pad(2) : sxe} - {t.replaceAll('?', 'ae').replaceAll('?', 'ue').replaceAll('?', 'oe')}

[MOVE] Rename [/media/disc/Downloads/B S11E13 D M i M G D W x-SOF/B S11E13 D M i M G D W x-SOF/Bones.S11E13.Das.Monster.im.Moerder.GERMAN.DUBBED.WebHDRiP.x264-SOF/sof-bones-s11e13.mkv] to [/media/disc/Serien/Bones/Staffel 11/Bones - 11x13 -.mkv]
Processed 1 files

I saw that he replaced "ä,ö,ü" with "?" but how can do then?

[rednoah]

1.
Have you read the FAQ yet?

Q: I'm running FileBot on a Linux machine and non-ASCII characters get all messed up. Why do unicode characters not work?
On some machines the locale is not set up. You'll need to tell Java what charset filenames are encoded with by setting the environment variable LANG. If you get an InvalidPathException about unmappable characters then it could very well be because LANG is not set up correctly.

Code: Select all

export LANG="en_US.UTF-8"
export LC_ALL="en_US.UTF-8"

2
What's your sysinfo and sysenv output?

[JamesBlond007]

Here's the log from the console:

Code: Select all

 export LANG="en_US.UTF-8"
root@MyServer:~# export LC_ALL="en_US.UTF-8"
-bash: Warnung: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

while installing sysinfo there came a lot of warnings. Here it is:

Code: Select all

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = "en_US.UTF-8",
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Thats the Output from sysinfo:

Code: Select all

~# sysinfo

(Sysinfo:15871): Gtk-WARNING **: Locale not supported by C library.
        Using the fallback 'C' locale.

(Sysinfo:15871): Gtk-WARNING **: cannot open display:

i can try to install sysenv but maybe you don't need cause it won't work the fast.

My System is a MicroServer Gen8 from HP.
It's running OpenMediaVault - Version 2.2.13 (StoneBurner) with Kernel Linux 3.16.0-0.bpo.4-amd64

The Support-Information from the Backend show this:

Code: Select all

================================================================================
= OS/Debian information
================================================================================
Distributor ID: debian
Description: Debian GNU/Linux 7 (wheezy)
Release: 7.11
Codename: wheezy

================================================================================
= openmediavault information
================================================================================
Release: 2.2.13
Codename: Stone burner

================================================================================
= Date
================================================================================
Thu Jan 12 18:54:47 CET 2017

================================================================================
= Locale
================================================================================
de_DE.UTF-8

[rednoah]

1.
Try google for some of those error messages. Java may defaults to ASCII as filesystem encoding on badly configured systems.


2.
I meant

Code: Select all

filebot -script fn:sysinfo

and

Code: Select all

filebot -script fn:sysenv

[JamesBlond007]

Code: Select all

filebot -script fn:sysinfo
FileBot 4.7 (r3923)
JNA Native: 4.0.1
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 native library libmediainfo.so: Unable to load library 'zen': Native library (linux-x86-64/libzen.so) not found in resource path ([file:/usr/share/filebot/FileBot.jar])
7-Zip-JBinding: net.sf.sevenzipjbinding.SevenZipNativeInitializationException: Failed to load 7z-JBinding: /usr/share/filebot/lib7-Zip-JBinding.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found (required by /usr/share/filebot/lib7-Zip-JBinding.so)
Chromaprint: [/usr/share/filebot/fpcalc, -version] failed
Extended Attributes: OK
Script Bundle: 2016-11-07 (r458)
Groovy: 2.4.6
JRE: Java(TM) SE Runtime Environment 1.8.0_111 (headless)
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 2 Core / 844 MB Max Memory / 13 MB Used Memory
OS: Linux (amd64)
Package: DEB
Data: /root/.filebot
uname: Linux MyMicroserver 3.16.0-0.bpo.4-amd64 #1 SMP Debian 3.16.36-1+deb8u2~bpo70+1 (2016-10-19) x86_64 GNU/Linux

------------------- UPDATE AVAILABLE: FileBot 4.7.7 (r4678) --------------------

Done ?(?????)?

and

Code: Select all

filebot -script fn:sysenv
# Environment Variables #
_: /usr/bin/filebot
MAIL: /var/mail/root
LOGNAME: root
SHLVL: 1
LD_LIBRARY_PATH: /usr/share/filebot
LC_ALL: en_US.UTF-8
HOME: /root
SSH_CONNECTION: myip 45212 192.168.2.113 22
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SSH_TTY: /dev/pts/3
SSH_CLIENT: myip 45212 22
XFILESEARCHPATH: /usr/dt/app-defaults/%L/Dt
SHELL: /bin/bash
TERM: xterm
NLSPATH: /usr/dt/lib/nls/msg/%L/%N.cat
LANG: en_US.UTF-8
PWD: /root
USER: root


# Java System Properties #
path.separator: :
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
java.io.tmpdir: /root/.filebot/temp
user.country: US
user.home: /root
useExtendedFileAttributes: true
org.apache.commons.logging.Log: org.apache.commons.logging.impl.NoOpLog
java.vm.vendor: Oracle Corporation
user.language: en
java.vendor: Oracle Corporation
java.vendor.url.bug: http://bugreport.sun.com/bugreport/
swing.crossplatformlaf: javax.swing.plaf.nimbus.NimbusLookAndFeel
application.rename.history: true
sun.net.client.defaultReadTimeout: 60000
java.specification.name: Java Platform API Specification
jna.nosys: true
os.name: Linux
java.runtime.name: Java(TM) SE Runtime Environment
application.deployment: deb
file.separator: /
sun.net.client.defaultConnectTimeout: 10000
java.vm.specification.version: 1.8
sun.boot.class.path: /usr/lib/jvm/java-8-oracle/jre/lib/resources.jar:/usr/lib/jvm/java-8-oracle/jre/lib/rt.jar:/usr/lib/jvm/java-8-oracle/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jsse.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jce.jar:/usr/lib/jvm/java-8-oracle/jre/lib/charsets.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jfr.jar:/usr/lib/jvm/java-8-oracle/jre/classes
java.version: 1.8.0_111
http.agent: FileBot 4.7
java.library.path: /usr/share/filebot:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
unixfs: false
useGVFS: false
awt.toolkit: sun.awt.X11.XToolkit
java.vm.specification.name: Java Virtual Machine Specification
os.version: 3.16.0-0.bpo.4-amd64
useCreationDate: true
file.encoding: ANSI_X3.4-1968
java.awt.printerjob: sun.print.PSPrinterJob
grape.root: /root/.filebot/grape
java.class.version: 52.0
java.ext.dirs: /usr/lib/jvm/java-8-oracle/jre/lib/ext:/usr/java/packages/lib/ext
java.specification.version: 1.8
net.filebot.AcoustID.fpcalc: /usr/share/filebot/fpcalc
java.vendor.url: http://java.oracle.com/
sun.os.patch.level: unknown
sun.java.launcher: SUN_STANDARD
os.arch: amd64
user.dir: /root
line.separator:

user.timezone:
sun.cpu.endian: little
java.vm.specification.vendor: Oracle Corporation
java.home: /usr/lib/jvm/java-8-oracle/jre
java.net.useSystemProxies: false
sun.management.compiler: HotSpot 64-Bit Tiered Compilers
sun.arch.data.model: 64
java.endorsed.dirs: /usr/lib/jvm/java-8-oracle/jre/lib/endorsed
file.encoding.pkg: sun.io
java.specification.vendor: Oracle Corporation
sun.boot.library.path: /usr/lib/jvm/java-8-oracle/jre/lib/amd64
sun.jnu.encoding: ANSI_X3.4-1968
java.runtime.version: 1.8.0_111-b14
java.vm.info: mixed mode
java.vm.version: 25.111-b14
sun.io.unicode.encoding: UnicodeLittle
java.awt.graphicsenv: sun.awt.X11GraphicsEnvironment
application.dir: /root/.filebot
java.class.path: /usr/share/filebot/FileBot.jar
sun.java.command: /usr/share/filebot/FileBot.jar -script fn:sysenv
sun.cpu.isalist:
user.name: root


# Arguments #
No such property: argumentArray for class: net.filebot.cli.ArgumentBean
groovy.lang.MissingPropertyException: No such property: argumentArray for class: net.filebot.cli.ArgumentBean
        at Script1.run(Script1.groovy:17)
        at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:62)
        at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:72)
        at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:114)
        at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
        at net.filebot.Main.main(Main.java:120)
Failure (?_?)

I tried out a bit and this looks good i think. Maybe it'll work now. But at changing i got many errors...

Code: Select all

LC_ALL: en_US.UTF-8

EDIT: Nope. It won't.

[rednoah]

Your environment is set to ANSI filesystem encoding:

Code: Select all

sun.jnu.encoding: ANSI_X3.4-1968

It should be set to UTF-8 filesystem encoding:

Code: Select all

$ filebot -script fn:sysenv | grep sun.jnu.encoding
sun.jnu.encoding: UTF-8

[JamesBlond007]

Code: Select all

~# filebot -script fn:sysenv | grep sun.jnu.encoding
sun.jnu.encoding: ANSI_X3.4-1968
No such property: argumentArray for class: net.filebot.cli.ArgumentBean
groovy.lang.MissingPropertyException: No such property: argumentArray for class: net.filebot.cli.ArgumentBean
        at Script1.run(Script1.groovy:17)
        at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:62)
        at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:72)
        at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:114)
        at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
        at net.filebot.Main.main(Main.java:120)

And now you say Google is my friend or do you have a fast solution?

[rednoah]

1.
You're using an old version of FileBot.

Code: Select all

FileBot 4.7 (r3923) => No such property: argumentArray for class: net.filebot.cli.ArgumentBean

2.
You will find this to be a generic Linux issue that many people have encountered in countless ways. I can only tell you that you need to set the locale correctly.

e.g.
https://perlgeek.de/en/article/set-up-a ... nvironment
http://askubuntu.com/questions/412495/s ... cale-utf-8
...

I cannot possibly provide support for every single flavour of Linux, thus Google is your friend.

[JamesBlond007]

So. Tried a bit.
Updated filebot. Done a few things from a few Threads but don't ask me what exactly. Could only get en_US run a few mins until reboot but it won't functioning
Now the result is:

Code: Select all

# filebot -script fn:sysenv | grep sun.jnu.encoding
sun.jnu.encoding: UTF-8

and

Code: Select all

# locale
LANG=de_DE.UTF-8
LANGUAGE=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=de_DE.UTF-8

I think de_DE.utf-8 is also okay?
But filebot won't work well with the EventScripter Script.
Maybe it's interesting that when i start my shellscript with filebot it works but not the jdownloader script.
The Logs without renameAll. When adding this then the Title is only missing in result:

Code: Select all

Malformed input or input contains unmappable characters: /media/disc/Serien/Bones/Staffel 10/Bones - 10x12 - Ein Raum mit Büchern und ein Körper ohne Seele.mkv
Processing 1 unsorted files
Illegal parameter: map
java.lang.IllegalArgumentException: Illegal parameter: map
	at net.filebot.cli.ScriptShellBaseClass.lambda$getArgumentBean$1(ScriptShellBaseClass.java:470)
	at net.filebot.cli.ScriptShellBaseClass.getArgumentBean(ScriptShellBaseClass.java:464)
	at net.filebot.cli.ScriptShellBaseClass.rename(ScriptShellBaseClass.java:346)
	at Script1.run(Script1.groovy:491)
	at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:64)
	at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:74)
	at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:113)
	at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:30)
	at net.filebot.Main.main(Main.java:120)
Caused by: java.lang.NoSuchFieldException: map
	at net.filebot.cli.ScriptShellBaseClass.lambda$getArgumentBean$1(ScriptShellBaseClass.java:466)
	... 8 more

EDIT: I'm really thinking to do a completley reinstall from the server but not with OMV. What's your advice for a server OS?
Want to do: Trakt (coming soon)->Flexget (coming soon)->Jdownloader->Filebot->Plex and maybe later will use as cloud with ownCloud. What do you thinking of FreeNAS? Or you think a headless Ubuntu or Linux is better?

[rednoah]

1.
This means that it's working:

Code: Select all

# filebot -script fn:sysenv | grep sun.jnu.encoding
sun.jnu.encoding: UTF-8

However, running this in your own shell means nothing, because the environment may be different when your script is called by another process. Next step is to confirm that LC_ALL is set correctly even when the script is called by another process.


2.
I'd go with Ubuntu because it's more popular (unless you have a very good reason for using FreeNAS).