[bash] SABnzbd script not working anymore - bad substitution

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
redoXiD
Posts: 11
Joined: 14 Apr 2022, 21:52

[bash] SABnzbd script not working anymore - bad substitution

Post by redoXiD »

Hi there,

I was running a Filebot (r9302) script in SABnzbd on a QNAP NAS with no problems until a few days ago. I don't exactly know what happened, but somehow the script now results in an error:

Code: Select all

/share/Download/SABnzbd Scripts/filebot-movies.sh: line 8: "{n.colon(' - ')} ({y}) { "{edition-${tags.first()}}" } - {vf}{' CD'+pi}{any{subt}{f.subtitle ? fn =~ /forced/ ? '.ger.forced' : '.ger' : null}}": bad substitution
I can confirm that my format works, I've already tested it on windows before applying it to the script. Here is my full post processing script, though obviously only the Filebot part matters:

Code: Select all

#!/bin/bash
cd "$1"
ls -A -R -g -p --group-directories-first > filelist.txt
sleep 1
find . \( -path "*/Subs/*" -o -path "*/subs/*" -o -path "*/SUBS/*" -o -path "*/Trailers/*" \) -print -exec mv {} ./ \; && find . -type d -empty -delete
sleep 2

filebot -rename -r "$1" --db TheMovieDB -non-strict --lang German \
   --conflict override -non-strict \
   --file-filter "none{ ext =~ /jpg|png|nfo|txt|dts|dtshd|mka|thd|ac3|eac3|ass/ }{ fn.match(/sample|trailer/) }{ f =~ /Extras|Featurettes/ }" \
   --format "{n.colon(' - ')} ({y}) { "{edition-${tags.first()}}" } - {vf}{' CD'+pi}{any{subt}{f.subtitle ? fn =~ /forced/ ? '.ger.forced' : '.ger' : null}}"

echo "Done. Refreshing library..."

sleep 4

curl -s "http://192.168.0.151:32400/library/sections/1/refresh?X-Plex-Token=RQ5PPsQF6s6Jfbqvjfuo"

sleep 2

echo Done
Is there anything in my script that could cause this error? I'd be thankful for any kind of help!
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: SABnzbd script not working anymore - bad substitution

Post by rednoah »

This is a bash error message telling you that your bash script is syntactically incorrect:

Code: Select all

*.sh: line N: LINE: bad substitution

:!: You have added a $ expression to your "..." quoted argument value, and so bash is interpreting (i.e. substituting) the ${...} in "..." as bash code which fortunately fails with an error, instead of silently evaluating to blank:

Code: Select all

# echo "${tags.first()}"
zsh: bad substitution

Code: Select all

# echo "${tags}"

Code: Select all

# echo "\${tags.first()}"
${tags.first()}

Code: Select all

# echo '${tags.first()}'
${tags.first()}
:idea: Please read Cmdline and Argument Passing for details.



:arrow: You can pass complex formats on the command-line, but correctly escaping complex formats is difficult. You'll want to side-step this class of problem by reading your complex format from an external *.groovy text file:

Code: Select all

--format "/path/to/MyFormat.groovy"
:idea: Please read the FAQ and How to Request Help.
redoXiD
Posts: 11
Joined: 14 Apr 2022, 21:52

Re: SABnzbd script not working anymore - bad substitution

Post by redoXiD »

Thank you for your answer! I've already tried to use a backslash \ to escape the $ but I acutally don't really know where to put it as { "{edition-\${tags.first()}}" } doesn't seem to work.

I also tried to use a groovy file but unfortunatly with no success either. According to a few code snippets I found it should look something like this:

Code: Select all

rename(file:files, format:'Movies/{n.colon(' - ')} ({y}) { "{edition-${tags.first()}}" } - {vf}{' CD'+pi}{any{subt}{f.subtitle ? fn =~ /forced/ ? '.ger.forced' : '.ger' : null}}', db:'TheMovieDB', lang:'German')
Is the usage correct? And if not, what should it look like? I definitely underestimated how complicated it would be to integrate the Plex Editions feature into my custom format... xD
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: SABnzbd script not working anymore - bad substitution

Post by rednoah »

rednoah wrote: 22 Oct 2022, 11:38 :arrow: You can pass complex formats on the command-line, but correctly escaping complex formats is difficult. You'll want to side-step this class of problem by reading your complex format from an external *.groovy text file:

Code: Select all

--format "/path/to/MyFormat.groovy"
Do this.


e.g. copy & paste your format verbatim from the Format Editor into a new text file named MyFormat.groovy and then pass the file path to that as --format option value:

Code: Select all

filebot -rename -r "$1" --db TheMovieDB -non-strict --lang German \
   --conflict override -non-strict \
   --file-filter "none{ ext =~ /jpg|png|nfo|txt|dts|dtshd|mka|thd|ac3|eac3|ass/ }{ fn.match(/sample|trailer/) }{ f =~ /Extras|Featurettes/ }" \
   --format /path/to/MyFormat.groovy
MyFormat.groovy file contains your format, literally, verbatim, just that:

Code: Select all

$ cat /path/to/MyFormat.groovy
{n.colon(' - ')} ({y}) { "{edition-${tags.first()}}" } - {vf}{' CD'+pi}{any{subt}{f.subtitle ? fn =~ /forced/ ? '.ger.forced' : '.ger' : null}}
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: SABnzbd script not working anymore - bad substitution

Post by rednoah »

redoXiD wrote: 22 Oct 2022, 23:48 I've already tried to use a backslash \ to escape the $ but I acutally don't really know where to put it as { "{edition-\${tags.first()}}" } doesn't
You did not read Cmdline and Argument Passing did you? :lol: I really recommend reading that. It's really short and real useful and really covers all the pitfalls you - and everyone else - inevitably run into sooner or later. :lol:


Compare & Contrast:

Code: Select all

echo " - { "{edition-${tags.first()}}" } - "
-sh: bad substitution

Code: Select all

echo " - { "{edition-\${tags.first()}}" } - "
-sh: syntax error near unexpected token `('

Code: Select all

echo " - { \"{edition-\${tags.first()}}\" } - "
 - { "{edition-${tags.first()}}" } - 
:arrow: At this point we learn that "..." is a really bad idea if we just want to pass along some literal value. We also learn that we can't use " in "..." quoted text (unless escaped with \") for obvious reasons.


With bash you can always use '...' instead:

Code: Select all

echo ' - { "{edition-${tags.first()}}" } - '
 - { "{edition-${tags.first()}}" } -
We of course can't use ' in '...' quoted text for above mentioned obvious reasons, so we need to end ' start " print ' end " restart ' just to print this:

Code: Select all

$ echo '{n.colon('"'"' - '"'"')}'
{n.colon(' - ')}

Now you have learned how the command-line works, and why we don't pass complex arguments via the command-line. Read this post thoroughly. Read it twice. Try with echo on the command-line yourself. Understand how it works. You'll probably still stumble of the same kind of problem a few times before you really learn. But maybe I could save you a few hours of ripping your hair out.
:idea: Please read the FAQ and How to Request Help.
redoXiD
Posts: 11
Joined: 14 Apr 2022, 21:52

Re: [bash] SABnzbd script not working anymore - bad substitution

Post by redoXiD »

rednoah wrote: 23 Oct 2022, 07:30 You did not read Cmdline and Argument Passing did you?
I actually did that, though apparently I misunderstood the article as I was trying to escape the $ instead of the quotes. It makes much more sense now... :D
rednoah wrote: 23 Oct 2022, 06:57 MyFormat.groovy file contains your format, literally, verbatim, just that:

$ cat /path/to/MyFormat.groovy
{n.colon(' - ')} ({y}) { "{edition-${tags.first()}}" } - {vf}{' CD'+pi}{any{subt}{f.subtitle ? fn =~ /forced/ ? '.ger.forced' : '.ger' : null}}
Now it works! That was actually the first thing I tried, the "rename(file ...)" etc. stuff was just because Filebot didn't rename the files at all using this method, even though there were no errors in the log file (so I thought something was wrong with the groovy file). However, it turned out that the problem was the beta version (4.9.8), as not even the simplest rename commands via CLI were working (successfully executed but no files were renamed). Now I have downgraded my version to 4.9.6 stable and everything works fine using the groovy method.

Thanks again for the really great help and the patience to explain everything in such detail over and over again!
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [bash] SABnzbd script not working anymore - bad substitution

Post by rednoah »

redoXiD wrote: 23 Oct 2022, 10:31 However, it turned out that the problem was the beta version (4.9.8), as not even the simplest rename commands via CLI were working (successfully executed but no files were renamed). Now I have downgraded my version to 4.9.6 stable and everything works fine using the groovy method.
What did the console output say when it didn't work?


:idea: Read Shell Script Debugging for Beginners if you're not collecting console output.
:idea: Please read the FAQ and How to Request Help.
redoXiD
Posts: 11
Joined: 14 Apr 2022, 21:52

Re: [bash] SABnzbd script not working anymore - bad substitution

Post by redoXiD »

rednoah wrote: 23 Oct 2022, 10:45 What did the console output say when it didn't work?
That's my console output using version 4.9.8:

Code: Select all

[~] # filebot -rename /share/Download/complete/War.of.the.Worlds.Anhihilation.German.2021.AC3.BDRiP.x264-SAVASTANOS/ --format "/share/Download/SABnzbd Scripts/movie-format.groovy" --db TheMovieDB --lang German
* Consider using --apply refresh to refresh file services and media library
Rename movies using [TheMovieDB]
[~] #
Looks actually very inconspicuous except for the missing listing of the new file names (as in 4.9.6).
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [bash] SABnzbd script not working anymore - bad substitution

Post by rednoah »

:?: Did you log-in via SSH to run the command above?

Console output like this shouldn't be possible. There will always be some kind of final status message on the last line. Even if the process crashes there should be some sort of console output. That suggests that we're missing something.



:?: What does filebot -script fn:sysinfo say?
:idea: Please read the FAQ and How to Request Help.
redoXiD
Posts: 11
Joined: 14 Apr 2022, 21:52

Re: [bash] SABnzbd script not working anymore - bad substitution

Post by redoXiD »

Yes, I used SSH to run the command. Here is the script output:

Code: Select all

[~] # filebot -script fn:sysinfo
FileBot 4.9.8 (r9302)
JNA Native: 6.1.2
MediaInfo: 19.04
Tools: 7z/16.02 unrar/5.30 ffprobe/3.3.6
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2022-08-15 (r836)
Groovy: 3.0.9
JRE: OpenJDK Runtime Environment 18.0.2.1
JVM: OpenJDK 64-Bit Server VM
CPU/MEM: 2 Core / 987 MB Max Memory / 41 MB Used Memory
OS: Linux (amd64)
HW: Linux NAS475B39 5.10.60-qnap #1 SMP Sat Oct 1 05:13:09 CST 2022 x86_64 GNU/Linux
CPU/MEM: Intel(R) Celeron(R) J4025 CPU @ 2.00GHz [MemTotal: 3.9 GB | MemFree: 493 MB | MemAvailable: 746 MB | SwapCached: 441 MB | SwapTotal: 25 GB | SwapFree: 21 GB]
STORAGE: ext3 [/mnt/HDA_ROOT] @ 327 MB | ext4 [/share/CACHEDEV1_DATA] @ 1.8 TB | ext4 [/mnt/ext] @ 10 MB
USER: admin
DATA: /opt/filebot/data/0
Package: QPKG
License: FileBot License PX21730371 (Valid-Until: 2070-12-28)
Done ヾ(@⌒ー⌒@)ノ
I also tried to run it on my Raspberry Pi 4 running Kali, but the output of the renaming command is the same as on my QNAP NAS. On my main system it's actually not possible to run the beta as there is not native x64 installer and the portable version crashes immediately and results in a Windows logout event.
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [bash] SABnzbd script not working anymore - bad substitution

Post by rednoah »

That is most peculiar. Sounds like corrupted code. I can't seem to reproduce the issue on any of my QNAP test devices though. I'll keep an eye out for similar reports to see if we can narrow down on the issue.
:idea: Please read the FAQ and How to Request Help.
Post Reply