[FileBot Node] DSM 6.2.4 authentication workaround

[rednoah]

⭑⭑ UPDATE ⭑⭑

⭑⭑ FileBot packages for Synology DSM 6.2.4 are in beta and available for testing ⭑⭑


Packages for DSM 6.2.4:

• Java Installer (java-installer_*.spk)
• FileBot (FileBot_*.spk)
• FileBot Node (filebot-node_*.spk)

Add Package Source:

• Package Center ➔ Settings ➔ Package Sources ➔ Add ➔ Name: FileBot Node

  Code: Select all

  https://raw.githubusercontent.com/filebot/filebot-node/master/syno-dsm6.json

⭑⭑ UPDATE ⭑⭑




DSM 6.2.4 breaks DSM authentication for FileBot Node, thus the client-side Web UI won't be able to communicate with the server-side node service, i.e. nothing happens when I click on Execute or any other button.



Option A for docker capable devices:

If you are familiar with docker and have an x86_64 device, then you can just use the generic filebot-node docker container instead:
https://hub.docker.com/r/rednoah/filebot/

e.g. Run a custom filebot-node service at http://<IP>:5452/filebot/

Code: Select all

docker run --rm -it -v /volume1:/volume1 -v data:/data -p 5452:5452 -e FILEBOT_NODE_AUTH=BASIC -e FILEBOT_NODE_AUTH_USER=alice -e FILEBOT_NODE_AUTH_PASS=wxy87rFb

Option B for FileBot Node 0.3.1 and lower:

If you're familiar with SSH, then you can configure the FileBot Node server-side service to use HTTP BASIC authentication instead of Synology authentication.

Edit /var/packages/filebot-node/target/start

Code: Select all

sudo vi /var/packages/filebot-node/target/start

and configure FILEBOT_NODE_AUTH from

Code: Select all

export FILEBOT_NODE_AUTH=SYNO

to use HTTP BASIC authentication with a custom username and password

Code: Select all

export FILEBOT_NODE_AUTH=BASIC
export FILEBOT_NODE_AUTH_USER=alice
export FILEBOT_NODE_AUTH_PASS=wxy87rFb

Restart the FileBot Node server-side service by restarting your NAS.

You can now open http://<IP>:5000/webman/3rdparty/filebot-node/index.html directly without having to log in via Synology DSM first.

[aglyons]

I tried Option B.

This does not work. Yes, Filebot Node is accessible but Node, for some reason cannot communicate with Filebot.

When I start FB on a pre 6.2.4 DSM, Node displays the FB version showing that it can call the service. on 6.2.4 DSM version, that does not happen and calling the script does nothing.

*** Pre 6.2.4 - Displays FB Version info****




**** Post 6.2.4 - No FB version info displayed****




**** Post 6.2.4 following Option B - No FB version info displayed*****

[colpepe]

Hello. I am having similar issues with option B. Here is my full dump from '/var/packages/filebot-node/target/start':

Code: Select all

#!/bin/sh

export FILEBOT_NODE_HOST="127.0.0.1" # bind to local nginx reverse proxy
export FILEBOT_NODE_AUTH="BASIC"
export FILEBOT_NODE_AUTH_USER=MyUser
export FILEBOT_NODE_AUTH_PASS=MyPa$$

export FILEBOT_NODE_HTTP="YES"
export FILEBOT_NODE_HTTP_PORT="5452"

export USER="admin"                                                                                           >
export JAVA_OPTS=`awk '/MemTotal:/ { xmx = ($2*0.8)/1024; if (xmx < 1024) { printf "-Xmx%dm", xmx }; exit}' /p>

export FILEBOT_NODE_DATA="/usr/local/filebot-node/data"
export FILEBOT_TASK_CMD="/usr/local/filebot-node/task"

export FILEBOT_CMD="filebot"
export FILEBOT_CMD_CWD="/volume1"
export FILEBOT_CMD_UID=`id -u $USER`
export FILEBOT_CMD_GID=`cat /etc/group | grep 'administrators' | cut -d: -f3`                 # cannot use `id>

export FILEBOT_NODE_CLIENT="client"

# sanity check
if [ -z "$FILEBOT_CMD_UID" ]; then
        echo "id -u $USER must not be empty"
        exit 1
fi

# debug output
if ! ls -l "$(which node)"; then
        echo "Node.js is not installed. Please install Node.js in the Package Center."
fi

# set working dir
cd "/usr/local/filebot-node"

# --optimize_for_size (Enables optimizations which favor memory size over execution speed.)
node --optimize_for_size "server/app.js"

It loads up ok when ran, no errors in pkgctl logs, but terminal is blank and the buttons do not a thing.


Edit: I get the same thing when I access from 'http://<IP>:5000/webman/3rdparty/filebot-node/index.html' just a bigger window.

Thanks in advance!

[rednoah]

1.
Did you restart the FileBot Node service after you mode your code changes?


2.
If you continue to have trouble, please open the Developer Tools in your Browser so you can see each HTTP request and the corresponding error message:

[davemccreath]

For rednoah

I too have the same issue with FileBot post the 6.2.4 update last night.

I'm not that savvy when it comes to SSH or Docker, is it worth waiting for an update for FileBot to fix the issue or are you focussed on the forthcoming DSM 7 release (finger's crossed).

Regards

Dave

[rednoah]

DSM 6.2.4 seems to be rolling out to more devices since yesterday. I'll start working on a fix for DSM 6.2.4 once my DS213J auto-updates.

[colpepe]

On my Node page I get the following errors:

Firstly,

Code: Select all

app.js?_dc=20200824200454:1 Uncaught RangeError: Maximum call stack size exceeded
    at String.replace (<anonymous>)
    at Object.encodeString (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)

Followed by a lot of,

Code: Select all

tasks:1 Failed to load resource: the server responded with a status of 401 (Unauthorized)

Thanks Noah!

[colpepe]

Alright I will take back what I said Noah. I am up and working at present.

1. Killed Filebot Node service
2. Updated '/var/packages/filebot-node/target/start' with local device admin creds
3. Started Filebot Node service back up
4. Logged into web page; text is now showing in terminal window
5. Imported license back in, dry run good, execute works!

Not sure what I did wrong yesterday but your SSH workaround worked just fine for me today! Thanks for your hard work and your great product!

[aglyons]

I restarted Filebot-Node and saw the same thing.

I adjusted the changes to the start file, mainly wrapped the cars in quotes as that's how they are in the original. I missed that BASIC was in caps so fixed that. Upon loading the page I was prompted for web auth U and P but only once. Subsiquent loads didn't ask for credentials.

Now, while I did manage to get Node talking with Filebot as there is output in the CLI window, it appears that Filebot cannot access the DIR's configured.

Plot thickens.

Code: Select all

filebot -script 'fn:amc' '/Sync/Seedbox/2-To Process' --output /Sync/Seedbox/3-Processed --action duplicate -non-strict --order Airdate --conflict auto --lang en --def 'unsorted=y' 'skipExtract=y' 'seriesFormat=TV/{n}/{n} - {s00e00} - {t}' 'movieFormat=Movies/{n} {y}' 'excludeList=.excludes' --apply refresh -no-probe -no-index --log all --log-file '/usr/local/filebot-node/data/filebot.log'

------------------------------------------

OpenJDK 64-Bit Server VM warning: Option --illegal-access is deprecated and will be removed in a future release.
File does not exist: /Sync/Seedbox/2-To Process
Run script [fn:amc] at [Wed Apr 07 15:49:09 COT 2021]
Parameter: unsorted = y
Parameter: skipExtract = y
Parameter: seriesFormat = TV/{n}/{n} - {s00e00} - {t}
Parameter: movieFormat = Movies/{n} {y}
Parameter: excludeList = .excludes
Invalid usage: output folder must exist and must be a directory: /Sync/Seedbox/3-Processed
Abort (×_×)

------------------------------------------

[Process error]

🔺 Exit Code: 4

[rednoah]

These errors would suggest some kind of generic file system / user permission issue somewhat unrelated to FileBot Node or FileBot specifically:

Code: Select all

File does not exist: /Sync/Seedbox/2-To Process
...
Invalid usage: output folder must exist and must be a directory: /Sync/Seedbox/3-Processed
...

EDIT:

This is likely related to DSM 6.2.4 encouraging you to disable the admin user. FileBot Node runs filebot as admin so strange things may happen if admin is disabled.

[rednoah]

FileBot Node for Synology DSM 6.2.4 is available for testing: filebot-node-0.4.0-noarch.spk


You may need to uninstall older versions of FileBot Node, before manually installing the latest beta package. Upgrading the package may result in broken permissions caused by previously created user data.


FileBot Node is no longer integrated into the DSM nginx service. You therefore cannot use Quick Connect to access the FileBot Node WebUI.



EDIT:

You may or may not need to disable CSRF authentication token feature for Synology authentication integration to work:

[aglyons]

Hey renoah, thanks for putting some time into fixing the debacle Synology has handed us all.

I also use MODs to trigger filebot so I can have a one-click desktop shortcut. Perhaps you guys can chat for a bit and figure out how to incorporate what he's been doing with what you're doing. His approach basically triggers the same script that Node creates for the scheduled tasks. While great, any changes to Node creates a new schedule and breaks MODs which requires editing a text file via SSH in order to get it to run the right script.

https://github.com/vletroye/SynoPackage ... t%20Runner

[colpepe]

@rednoah - my syno 'admin' account is disabled and FBN runs fine in my case, just for the record.

@Aglyons - That is the first I heard about the MODS tools, thanks for that. Is there any reason you wouldn't just schedule filebot to run a task every 5-10 mins so you don't have to hassle with running it manually? I am just curious of your usage scenario.

[Dmx]

Hi rednoah,

thanks for still putting effort in DSM 6.2.4, I think its good since DSM 7 is still some time away. I saw that you uploaded a testversion and I want to give you feedback on it. This is v. 0.4.0 from your previous post. I had the latest online available version installed and updated it manually via Syno Package Center. A new window pops up after upgrading saying you need to give the System Internal Filebot User Read/Write Permissions on your working shares. This is very good advice at this point as otherwise it wouldn't work (see next paragraph)!

Filebot Node opens in a new Browser tab from now on (not like before in the Syno GUI), so I could confirm that authentication works properly again without changing anything manually. I tried a Dry run in Node before giving the Filebot User Read/Write access just to test and as expected it gave me an error due to missing permissions. Gave permission to the user and rebooted (the reboot was necessary at least for me, otherwise I would still get the same Permission error). Creating Tasks and Scheduling them seems to also work properly with this version.

Only thing I noticed which I'm not sure of is that a warning appears if you do something in the Filebot Node GUI saying you shouldn't run filebot.sh as root. I believe it wasn't like this in the old version, only if you really ran filebot.sh as root via SSH or your scheduler for example.

Example from opening the Filebot Node GUI:

Code: Select all

$ filebot -version
/volume1/@appstore/filebot/filebot.sh must NOT run as root
FileBot 4.9.3 (r8338) / OpenJDK Runtime Environment 14.0.1 / Linux 4.4.59+ (amd64)

Additional Sidenote:
- I didn't need to disable CSRF protection

[darko]

Thank you for this update, @rednoah! Like many others, I was stuck with DSM 6.2.4 and a broken FileBot Node installation. I now have FileBot Node 0.4.0 working smoothly with DSM 6.2.4.

Here are my notes with respect to getting 0.4.0 working:

• I needed to change permissions on /var/packages/filebot-node/target/*, as I was getting an access denied error when trying to set my license and when writing to log files.
• I also needed to do the same for .excludes files, which FileBot was not able to write to at first.
• For new users, I think the new UI and process around scheduling tasks will be a bit counter-intuitive compared to previous versions. If this is temporary/beta-only, disregard.

Thanks again for getting this out quickly!

[rednoah]

Only thing I noticed which I'm not sure of is that a warning appears if you do something in the Filebot Node GUI saying you shouldn't run filebot.sh as root. I believe it wasn't like this in the old version, only if you really ran filebot.sh as root via SSH or your scheduler for example.

The filebot must NOT run as root warning should go away once you create a custom admin user and disable the default admin user.

I needed to change permissions on /var/packages/filebot-node/target/*, as I was getting an access denied error when trying to set my license and when writing to log files.

Executed via FileBot Node or via SSH? The filebot process executed by FileBot Node will run as $SYNOPKG_USERNAME:users but if you run it via SSH or Task Scheduler, then you can run it as different users with different permissions which may lead to permission conflicts.

For new users, I think the new UI and process around scheduling tasks will be a bit counter-intuitive compared to previous versions. If this is temporary/beta-only, disregard.

FileBot Node can no longer integrate into DSM because we can no longer modify DSM internal files. The Scheduled Tasks feature is now the same across all platforms, i.e. it'll give you a command but you'll have to paste it into Task Scheduler yourself.

[darko]

Executed via FileBot Node or via SSH? The filebot process executed by FileBot Node will run as $SYNOPKG_USERNAME:users but if you run it via SSH or Task Scheduler, then you can run it as different users with different permissions which may lead to permission conflicts.

This was when executing FileBot via the FileBot Node GUI to test everything before I created a scheduled task.

Thanks for the specifics around what user/group is used in these cases; I'll be able to go back and tighten down permissions a bit.

Regarding what you know so far about the changes to DSM, do you think this change is permanent, or will DSM 7 restore some flexibility? I ask because this change seems to have garnered a lot of negative press, and I'm wondering if Synology is considering taking a different approach in the future.

[rednoah]

DSM 7.0 is all about implementing good security practices. Forcing each package to run as its own user / group with limited permissions is easy and just makes sense. Synology won't (and shouldn't) go back on that. Running everything as root was never a good idea in the first place.

[darko]

I agree with you there, but it's regrettable that FileBot Node can no longer leverage the DSM UI. Or maybe FileBot Node was using it in some non-standard ways. Regardless, it feels like they made a change with good intentions that has had some negative consequences for package authors.

On the other hand, much of the value of FileBot is in the set-it-and-forget-it paradigm it affords, and once things are running smoothly I typically don't need to interact with it much.

[Dmx]

Only thing I noticed which I'm not sure of is that a warning appears if you do something in the Filebot Node GUI saying you shouldn't run filebot.sh as root. I believe it wasn't like this in the old version, only if you really ran filebot.sh as root via SSH or your scheduler for example.

The filebot must NOT run as root warning should go away once you create a custom admin user and disable the default admin user.

Hi,
ah, I see. Thanks for the explanation. At least I know why this is appearing now. I'm not going to disable the default admin use though, so it doesn't really matter in the end
Thanks again for the new version.

[davemccreath]

rednoah. Many thanks for the update but for me it failed.

I set up a custom administrator account several months ago and disabled the old admin account.

I manually installed the filebot-node-0.4.0-noarch.spk file and followed the on-screen instructions post the install allowing filebot Read/Write access to both the folder items are downloaded into and the folder that filebot moves them to after renaming.

For me, when executing filebot it opens in a new window with my IP address as the URL and a port number in the 54** series. The window says This site can't be reached, (IP Address) refused to connect.

Tried a reinstall and also reversing the file permissions all to no avail (with reboots of the system after each change).

Regards

Dave

[rednoah]

What does the log say?
viewtopic.php?t=11030

[davemccreath]

rednoah

I've managed to SSH into the NAS but being a bit of a noob I've no idea how to open the file and I'm not so sure that it is there. Below is a dump from putty showing the contents of the directory.

[rednoah]

find / -name filebot-node.log will find the log for you:

Code: Select all

find / -name filebot-node.log

[davemccreath]

Ahh, found it under /var/log/packages/filebot-node.log

Stopped the FileBot Node process at 1658hrs via DSM and restarted a couple of seconds later.