Hint needed for calling bash script upon torrent completion

Support for macOS users and the Mac App Store package
Post Reply
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Hint needed for calling bash script upon torrent completion

Post by jprokos »

I have looked around the site for a hint on how to do this. I know it must exist.

Until now I have had all of my filebot commands inside the qBittorrent "run external program on torrent completion" window.

It's quite long and I thought I would just try to call a shell script instead. I realized that parameters aren't getting passed to the shell script from qbt.

Code: Select all

"ut_dir=%R" "ut_kind=multi" "ut_title=%N" "ut_label=%L"
:?: So, if calling a script with:

Code: Select all

/Users/john/Documents/Shell_Scripts/MyFileBot_Script.sh
in qbt what is the method to get those parameters over to the shell script?

Thanks for a simple answer.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

It looks like I an do this in qbittorrent?

Code: Select all

/Users/john/Documents/Shell_Scripts/MyFileBot_Script.sh  "%N" "%L" "%G" "%F" "%R" "%D" "%C" "%Z" "%T" "%I"
Now inside my shell script how do I define those?

Right now the shell script looks like this:

Code: Select all

filebot -script fn:amc --output "/Volumes" \
--action duplicate \
--conflict index  -non-strict \
--log-file amc.log \
--def excludeList="/Users/john/.filebot/amc.excludes" \
--def N="%N" \
--def L="%L" \
--def G=%"G" \
--def F="%F" \
--def R="%R" \
--def D="%D" \
--def C="%C" \
--def Z="%Z" \
--def T="%T" \
--def I="%I" \
--def pushover=xxx  \
--def unsorted=y \
--def music=y  \
--def skipExtract=y \
--def seriesFormat="/SeedDrive/PlexServer_2/{plex.derive{' {thetvdb-'}{id}{'}'}{' - ['+allOf{tags}{vf}{vs}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=R" "ut_kind=multi" "ut_title=N" "ut_label=L" \
--def movieFormat="/PlexMedia/PlexServer_1/{plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{if ('Documentary' in genres)'[doc]'}{info:video[0].displayAspectRatioString.colon('"∶"').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=R" "ut_kind=multi" "ut_title=N" "ut_label=L"; \
/bin/bash /Users/john/Documents/Shell_Scripts/Remove_non-English_subtitles.sh;

amc.log errors look like this

Code: Select all

Parameter ut_dir =
Parameter: ut_kind = multi
Parameter: ut_title = 
Parameter: ut_label = 
Parameter: movieFormat = as above
Invalid usage: no input
Abort (x_x)
I get it, my script isn't passing along the parameter variables, but I haven't figured how.
User avatar
rednoah
The Source
Posts: 19427
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Hint needed for calling bash script upon torrent completion

Post by rednoah »

You're passing information along as positional arguments. Therefore, those positional arguments will be available in the shell script as $1 .. $9.


e.g.

Code: Select all

./test.sh Hello World
$1 = Hello
$2 = World

:arrow: https://www.youtube.com/watch?v=BjBBfmD ... O_&index=4
:idea: Please read the FAQ and How to Request Help.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

Thanks Red!

Uhh, I know what 420 means but not "402". Hope it's not the mark of the devil. :evil:

I used your hint to solve the issue! I will elaborate here. Hope this helps others on macOS Catalina using qBittorrent 4.3.3

I replaced my filebot code at "Run external program on torrent completion" with a shell script by adding the path to the script and a list of the parameters (which qBittorrent passes to the script when the torrent is completed - cool)

Code: Select all

/Users/john/Documents/Shell_Scripts/MyFileBot_Script.sh  "%N" "%L" "%G" "%F" "%R" "%D" "%C" "%Z" "%T" "%I"
qBittorrent passes supported parameters as positional arguments.
${1} = %N: Torrent name
${2} = %L: Category
${3} = %G: Tags (separated by comma)
${4} = %F: Content path (same as root path for multi file torrent)
${5} = %R: Root path (first torrent subdirectory path)
${6} = %D: Save path
${7} = %C: Number of files
${8} = %Z: Torrent size (bytes)
${9} = %T: Current tracker
${10} = %I: Info hash

A few things tripped me up.
1. Having more than 9 arguments in a shell script caused an issue.
We need to use {} for the 10th argument or it will be treated as the value of $1 and then a 0 following, not $10. So curly brackets on all arguments is advised in the shell script.

:!: 2. When there is no value for a torrent's parameter, for me it was "%G" for tags, qBittorrent doesn't reliably pass "null" to the argument — it skips over it, which causes the rest of the arguments' values to be off by 1 position. I still have not figured out how to solve this one.

:?: When using --def seriesFormat and --def movieFormat what is the purpose of "ut_kind=multi"? Or maybe what is its purpose in all cases is the better question. Should we change that to the kind of file it is "movie" or "series" when we are defining?


For this script to work I had to modify my paths because they were failing as originally entered. I needed to add a "/" before {plex.derive.... when using parameter "%F" aka "ut_dir"=${4}

Code: Select all

--def seriesFormat="/Volumes/SeedDrive/PlexServer_2/{plex.derive{' {thetvdb-'}{id}{'}'}{' - ['+allOf{tags}{vf}{vs}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=${4}" "ut_kind=multi" "ut_title=${1}" "ut_label=${2}"

Code: Select all

--def movieFormat="/Volumes/PlexMedia/PlexServer_1/plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{if ('Documentary' in genres)'[doc]'}{info:video[0].displayAspectRatioString.colon('"∶"').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=${4}" "ut_kind=multi" "ut_title=${1}" "ut_label=${2}"
And I kept --output "/Volumes/PlexMedia/PlexServer_1"
User avatar
rednoah
The Source
Posts: 19427
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Hint needed for calling bash script upon torrent completion

Post by rednoah »

1.
Empty positional arguments should work though:

Code: Select all

./test.sh "" "X"
$1 = 
$2 = X

2.
If you're using qBT then you can just hard-code --def ut_kind=multi and leave it at that. The option is only required for interpreting uT argument values.
:idea: Please read the FAQ and How to Request Help.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

rednoah wrote: 18 Feb 2021, 04:17 1.
Empty positional arguments should work though:

Code: Select all

./test.sh "" "X"
$1 = 
$2 = X
2.
If you're using qBT then you can just hard-code --def ut_kind=multi and leave it at that. The option is only required for interpreting uT argument values.
I opened an issue because I didn't get a response in any qbt forum.
If I enter "G" in the third position, which is not a qbt parameter, then it gets sent over as positional argument with the value of G.
If I enter "%G" like the rest of the parameters and tag the torrent, then the tag value is assigned to the positional argument and the other parameters have their correct positional values.
If I enter "%G" like the rest of the parameters and don't tag the torrent, then apparently no value is assigned to that positional argument and each parameter's value is shifted towards the 3rd position leaving ${10} empty.

:?: I looked around the web for a way to re-assign a value to a parameter if it's initially empty but not sure how. Saw this but couldn't wrap my head around it.

I added this to the beginning of my script to check the value of the third parameter:

Code: Select all

if [ -z "$3"]; then
  echo "\$3 is null" > ~/Desktop/nullresult.txt
else
  echo "\$3 is not null" > ~/Desktop/notnullresult.txt
fi; \
The result was

Code: Select all

$3 is not null
So, I am at a loss for why the values are being transposed in the amc.log

:?: Can I rename the ut argument values or is filebot expecting them to be named the way you've named them, i.e. ut_title, ut_label?
User avatar
rednoah
The Source
Posts: 19427
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Hint needed for calling bash script upon torrent completion

Post by rednoah »

Well, I'd start by checking what exact command qBT is trying to execute in cases that don't work as expected. There should be a log somewhere. Only qBT knows how %variable replacement logic works exactly, and if there's any special handling for "%variables" syntax.


As for empty positional arguments, that's fundamentally not a problem. You can use the sysenv script to mirror the argument array that is passed in to the console output for debugging:

Code: Select all

filebot -script fn:sysenv "1" "" "3"
...
args[0] = -script
args[1] = fn:sysenv
args[2] = 1
args[3] =
args[4] = 3
:!: Note that test -z checks for undefined variables as well as defined variables that are empty.
:idea: Please read the FAQ and How to Request Help.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

rednoah wrote: 20 Feb 2021, 05:08 Well, I'd start by checking what exact command qBT is trying to execute in cases that don't work as expected. There should be a log somewhere. Only qBT knows how %variable replacement logic works exactly, and if there's any special handling for "%variables" syntax.


As for empty positional arguments, that's fundamentally not a problem. You can use the sysenv script to mirror the argument array that is passed in to the console output for debugging:

Code: Select all

filebot -script fn:sysenv "1" "" "3"
...
args[0] = -script
args[1] = fn:sysenv
args[2] = 1
args[3] =
args[4] = 3
:!: Note that test -z checks for undefined variables as well as defined variables that are empty.
There are no logs that I have that contain information about how those parameters were transferred. It's a black hole.

:?: I am not sure how to use the commands you wrote here. I should put that into a shell script and add that shell script to the run program field in qbt and output to a text file? How would I check all 10 of the arguments then?

When I enter that command in Terminal I get a lot more information than you show here. But I do get this at the end:

Code: Select all

# Arguments #
args[0] = -script
args[1] = fn:sysenv
args[2] = 1
args[3] = 
args[4] = 3
[2021-02-20 00:39:33.055] Done ヾ(@⌒ー⌒@)ノ
The hard part of testing this is, is having to download a torrent each time to active the process of running the external program. It's taking forever.
Last edited by jprokos on 20 Feb 2021, 08:03, edited 1 time in total.
User avatar
rednoah
The Source
Posts: 19427
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Hint needed for calling bash script upon torrent completion

Post by rednoah »

Do whatever you're doing. Change -script fn:amc to -script fn:sysenv. Now instead of running the amc script, your running the sysenv script, with the same arguments, to mirror the argument array (and environment variables) to the --log-file so you can see what is and isn't passed along and in what order.
:idea: Please read the FAQ and How to Request Help.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

Thanks for the explanation. I did as you said.

This is what I have in run external program in qbt:

Code: Select all

/Users/john/Documents/Shell_Scripts/qBttorrent_e-program.sh "%N" "%L" "%G" "%F" "%R" "%D" "%C" "%Z" "%T" "%I"
This is my altered bash script:

Code: Select all

#!/bin/sh
/usr/local/bin/filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" \
--action copy \
--conflict index -non-strict \
--log-file amc.log \
--def "ut_kind"="multi" \
--def "ut_title"=${1} \
--def "ut_label"=${2} \
--def "%G_tags"=${3} \
--def "ut_dir"=${4} \
--def "%R_rpath"=${5} \
--def "%D_spath"=${6} \
--def "%C_files"=${7} \
--def "%Z_bytes"=${8} \
--def "%T_track"=${9} \
--def "%I_info"=${10} \
--def excludeList="/Users/john/.filebot/amc.txt" \
--def pushover=1234 \
--def unsorted=y \
--def music=y \
--def skipExtract=y \
--def seriesFormat="/Volumes/SeedDrive/PlexServer_2/{plex.derive{' {thetvdb-'}{id}{'}'}{' - ['+allOf{tags}{vf}{vs}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=${4}" "ut_title=${1}" "ut_label=${2}" \
--def movieFormat="/Volumes/PlexMedia/PlexServer_1/{plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{if ('Documentary' in genres)'[doc]'}{info:video[0].displayAspectRatioString.colon('"∶"').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=${4}" "ut_title=${1}" "ut_label=${2}"; \
/bin/bash /Users/john/Documents/Shell_Scripts/Remove_non-English_subtitles.sh; \
This is the result inside amc.log: I don't think it did what you thought it would. In any case, again, %G is a path and not a null value. It appears to be passed from qbt as a path.

:?: Is there something else i need to look at. Is filebot caching something that I can clear?
User avatar
rednoah
The Source
Posts: 19427
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Hint needed for calling bash script upon torrent completion

Post by rednoah »

You'll want to call the sysenv script from qBT so that you can see the arguments that are passed along:

Code: Select all

filebot -script fn:sysenv --log-file sysenv.log "%N" "%L" "%G" "%F" "%R" "%D" "%C" "%Z" "%T" "%I"
:idea: Please read the FAQ and How to Request Help.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

rednoah wrote: 20 Feb 2021, 10:22 You'll want to call the sysenv script from qBT so that you can see the arguments that are passed along:

Code: Select all

filebot -script fn:sysenv --log-file sysenv.log "%N" "%L" "%G" "%F" "%R" "%D" "%C" "%Z" "%T" "%I"
That gave us stuff... but not everything that we asked for — "%G" "%F" "%R" "%D" are missing and in that order. In this case I did not tag the torrent (a value for %G)

Code: Select all

Sat Feb 20 11:26:19
iMac191:~ john$ tail /Users/john/.filebot/logs/sysenv.log 
[2021-02-20 11:15:19.404] Using persistent disk cache: /Users/john/.filebot/cache/0
[2021-02-20 11:15:19.464] File does not exist: Painting.with.John.S01E05.480p.x264-mSD[eztv.re].mkv
[2021-02-20 11:15:19.464] File does not exist: TV Shows
[2021-02-20 11:15:19.465] File does not exist: 1
[2021-02-20 11:15:19.465] File does not exist: 136825866
[2021-02-20 11:15:19.465] File does not exist: udp://tracker.opentrackr.org:1337/announce
[2021-02-20 11:15:19.465] File does not exist: 249162f92be34bd5aa9cb309fac9ec37d25cf74a
[2021-02-20 11:15:20.164] Done ヾ(@⌒ー⌒@)ノ
In this case I did tag the torrent (a value for %G) "%F" "%R" "%D" are missing and in that order.

Code: Select all

Sat Feb 20 11:36:02
iMac191:~ john$ tail /Users/john/.filebot/logs/sysenv.log 
[2021-02-20 11:53:01.219] Using persistent disk cache: /Users/john/.filebot/cache/0
[2021-02-20 11:53:01.282] File does not exist: WandaVision.S01E07.WEBRip.x264-ION10[eztv.re].mp4
[2021-02-20 11:53:01.283] File does not exist: TV Shows
[2021-02-20 11:53:01.283] File does not exist: TestTag
[2021-02-20 11:53:01.283] File does not exist: 1
[2021-02-20 11:53:01.283] File does not exist: 340247456
[2021-02-20 11:53:01.284] File does not exist: udp://tracker.opentrackr.org:1337/announce
[2021-02-20 11:53:01.284] File does not exist: fd850cbd0aebb56a63048a045ceb084540f9d2ce
[2021-02-20 11:53:02.032] Done ヾ(@⌒ー⌒@)ノ
:!: Before that worked, by calling from qBT, my script stopped getting called altogether. This was after I mistakenly entered the commands you gave in the script.Even after changing it back to the original - it was broken. Nothing was getting passed to filebot after torrent completions.

:?: The other thing I noticed in the amc.log from a different attempt was this.

Code: Select all

[2021-02-20 04:26:37.553] CLONE: Inappropriate ioctl for device / Invalid cross-device link
Can we fix this? Its logic chose [DUPLICATE].
User avatar
rednoah
The Source
Posts: 19427
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Hint needed for calling bash script upon torrent completion

Post by rednoah »

I see. I guess filebot -script fn:sysenv prints to the console output directly, and thus doesn't use the log file.


:idea: In this case, we need to use standard IO redirection to redirect the output:

Code: Select all

filebot -script fn:sysenv 1 2 3 >> /path/to/console-output.txt 2>&1


:arrow: Well, without qBT logging this is getting tricky. You'll want to contact the qBT people on how to find debug logs that can tell you what exactly is executed.
:idea: Please read the FAQ and How to Request Help.
jprokos
402
Posts: 135
Joined: 09 Oct 2016, 02:01

Re: Hint needed for calling bash script upon torrent completion

Post by jprokos »

Thanks Red. I will give that a shot tomorrow. I did open an "issue" on qBT's github page regarding this.

I am posting what is working now for qBittorrent 4.3.3 "run external program" shell script.

:idea: One site I found helpful for checking validity of shell scripts: https://www.shellcheck.net. They also have an executable you can install to check shell scripts.

There seemed to be some issues with apple security features on my system preventing the script from running. Since I recently upgraded to Catalina I decided to just reset them all using:

Code: Select all

tccutil reset All
This corrected my issues with Privacy & Security panel in Preferences.

On to qBT:

Since parameter %G seems to be messing up the other parameters when it's empty I decided to move it to the last, or 10th, parameter position for now.

Some notes.
It's best to quote the arguments in the script as I have done below and use curly brackets to avoid whitespace.
We must use the format that FileBot is expecting for the values. i.e. ut_kind, ut_title, ut_dir. They can't be renamed or the script will fail.
Those ut parameters have to be at the end of the shell script... I think they have to be at the end of --def seriesFormat and --def movieFormat as well? - not sure on that.

qBT "run external program" commands.

Code: Select all

/Users/username/Documents/Shell_Scripts/qBttorrent_e-program.sh "%N" "%L" "%F" "%R" "%D" "%C" "%Z" "%T" "%I" "%G"
shell script formatting

Code: Select all

#!/bin/dash
#
filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" \
--action copy \
--conflict index -non-strict \
--log-file amc.log \
--def "ut_kind"="multi" \
--def "ut_title"="${1}" \
--def "ut_label"="${2}" \
--def "ut_tags"="${10}" \
--def "ut_dir"="${3}" \
--def "ut_rpath"="${4}" \
--def "ut_spath"="${5}" \
--def "ut_files"="${6}" \
--def "ut_bytes"="${7}" \
--def "ut_track"="${8}" \
--def "ut_info"="${9}" \
--def excludeList="/Users/john/.filebot/amc_excludes.txt" \
--def pushover=xxxx \
--def unsorted=y \
--def music=y \
--def skipExtract=y \
--def seriesFormat="/Volumes/SeedDrive/PlexServer_2/{plex.derive{' {thetvdb-'}{id}{'}'}{' - ['+allOf{tags}{vf}{vs}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir"="${3}" "ut_title"="${1}" "ut_label"="${2}" \
--def movieFormat="/Volumes/PlexMedia/PlexServer_1/{plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{if ('Documentary' in genres)'[doc]'}{info:video[0].displayAspectRatioString.colon('":"').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir"="${3}" "ut_title"="${1}" "ut_label"="${2}"; \
Post Reply