[FileBot Node] DSM 6.2.4 authentication workaround

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

[FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

⭑⭑ UPDATE ⭑⭑
rednoah wrote: 22 May 2021, 05:04 ⭑⭑ FileBot packages for Synology DSM 6.2.4 are in beta and available for testing ⭑⭑


Packages for DSM 6.2.4:
Add Package Source:
  • Package Center ➔ Settings ➔ Package Sources ➔ Add ➔ Name: FileBot Node

    Code: Select all

    https://raw.githubusercontent.com/filebot/filebot-node/master/syno-dsm6.json
⭑⭑ UPDATE ⭑⭑




DSM 6.2.4 breaks DSM authentication for FileBot Node, thus the client-side Web UI won't be able to communicate with the server-side node service, i.e. nothing happens when I click on Execute or any other button.



Option A for docker capable devices:

If you are familiar with docker and have an x86_64 device, then you can just use the generic filebot-node docker container instead:
https://hub.docker.com/r/rednoah/filebot/

e.g. Run a custom filebot-node service at http://<IP>:5452/filebot/

Code: Select all

docker run --rm -it -v /volume1:/volume1 -v data:/data -p 5452:5452 -e FILEBOT_NODE_AUTH=BASIC -e FILEBOT_NODE_AUTH_USER=alice -e FILEBOT_NODE_AUTH_PASS=wxy87rFb


Option B for FileBot Node 0.3.1 and lower:

If you're familiar with SSH, then you can configure the FileBot Node server-side service to use HTTP BASIC authentication instead of Synology authentication.

Edit /var/packages/filebot-node/target/start

Code: Select all

sudo vi /var/packages/filebot-node/target/start
and configure FILEBOT_NODE_AUTH from

Code: Select all

export FILEBOT_NODE_AUTH=SYNO
to use HTTP BASIC authentication with a custom username and password

Code: Select all

export FILEBOT_NODE_AUTH=BASIC
export FILEBOT_NODE_AUTH_USER=alice
export FILEBOT_NODE_AUTH_PASS=wxy87rFb
:arrow: Restart the FileBot Node server-side service by restarting your NAS.

:arrow: You can now open http://<IP>:5000/webman/3rdparty/filebot-node/index.html directly without having to log in via Synology DSM first.
:idea: Please read the FAQ and How to Request Help.
aglyons
Posts: 19
Joined: 29 Dec 2020, 18:42

Re: [FileBot Node] DSM 6.2.4 workaround

Post by aglyons »

I tried Option B.

This does not work. Yes, Filebot Node is accessible but Node, for some reason cannot communicate with Filebot.

When I start FB on a pre 6.2.4 DSM, Node displays the FB version showing that it can call the service. on 6.2.4 DSM version, that does not happen and calling the script does nothing.

*** Pre 6.2.4 - Displays FB Version info****
Image



**** Post 6.2.4 - No FB version info displayed****
Image



**** Post 6.2.4 following Option B - No FB version info displayed*****
Image
colpepe
Posts: 4
Joined: 06 Apr 2021, 19:36

Re: [FileBot Node] DSM 6.2.4 workaround

Post by colpepe »

Hello. I am having similar issues with option B. Here is my full dump from '/var/packages/filebot-node/target/start':

Code: Select all

#!/bin/sh

export FILEBOT_NODE_HOST="127.0.0.1" # bind to local nginx reverse proxy
export FILEBOT_NODE_AUTH="BASIC"
export FILEBOT_NODE_AUTH_USER=MyUser
export FILEBOT_NODE_AUTH_PASS=MyPa$$

export FILEBOT_NODE_HTTP="YES"
export FILEBOT_NODE_HTTP_PORT="5452"

export USER="admin"                                                                                           >
export JAVA_OPTS=`awk '/MemTotal:/ { xmx = ($2*0.8)/1024; if (xmx < 1024) { printf "-Xmx%dm", xmx }; exit}' /p>

export FILEBOT_NODE_DATA="/usr/local/filebot-node/data"
export FILEBOT_TASK_CMD="/usr/local/filebot-node/task"

export FILEBOT_CMD="filebot"
export FILEBOT_CMD_CWD="/volume1"
export FILEBOT_CMD_UID=`id -u $USER`
export FILEBOT_CMD_GID=`cat /etc/group | grep 'administrators' | cut -d: -f3`                 # cannot use `id>

export FILEBOT_NODE_CLIENT="client"

# sanity check
if [ -z "$FILEBOT_CMD_UID" ]; then
        echo "id -u $USER must not be empty"
        exit 1
fi

# debug output
if ! ls -l "$(which node)"; then
        echo "Node.js is not installed. Please install Node.js in the Package Center."
fi

# set working dir
cd "/usr/local/filebot-node"

# --optimize_for_size (Enables optimizations which favor memory size over execution speed.)
node --optimize_for_size "server/app.js"
It loads up ok when ran, no errors in pkgctl logs, but terminal is blank and the buttons do not a thing.
Image

Edit: I get the same thing when I access from 'http://<IP>:5000/webman/3rdparty/filebot-node/index.html' just a bigger window.

Thanks in advance!
Last edited by colpepe on 06 Apr 2021, 19:49, edited 1 time in total.
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 workaround

Post by rednoah »

1.
Did you restart the FileBot Node service after you mode your code changes?


2.
If you continue to have trouble, please open the Developer Tools in your Browser so you can see each HTTP request and the corresponding error message:
Image
:idea: Please read the FAQ and How to Request Help.
davemccreath
Posts: 31
Joined: 08 Oct 2019, 12:03

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by davemccreath »

For rednoah

I too have the same issue with FileBot post the 6.2.4 update last night.

I'm not that savvy when it comes to SSH or Docker, is it worth waiting for an update for FileBot to fix the issue or are you focussed on the forthcoming DSM 7 release (finger's crossed).

Regards

Dave
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

DSM 6.2.4 seems to be rolling out to more devices since yesterday. I'll start working on a fix for DSM 6.2.4 once my DS213J auto-updates.
:idea: Please read the FAQ and How to Request Help.
colpepe
Posts: 4
Joined: 06 Apr 2021, 19:36

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by colpepe »

On my Node page I get the following errors:

Firstly,

Code: Select all

app.js?_dc=20200824200454:1 Uncaught RangeError: Maximum call stack size exceeded
    at String.replace (<anonymous>)
    at Object.encodeString (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
    at Object.doEncode [as encodeValue] (app.js?_dc=20200824200454:1)
    at encodeObject (app.js?_dc=20200824200454:1)
Followed by a lot of,

Code: Select all

tasks:1 Failed to load resource: the server responded with a status of 401 (Unauthorized)
Thanks Noah!
colpepe
Posts: 4
Joined: 06 Apr 2021, 19:36

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by colpepe »

Alright I will take back what I said Noah. I am up and working at present.

1. Killed Filebot Node service
2. Updated '/var/packages/filebot-node/target/start' with local device admin creds
3. Started Filebot Node service back up
4. Logged into web page; text is now showing in terminal window
5. Imported license back in, dry run good, execute works!

Not sure what I did wrong yesterday but your SSH workaround worked just fine for me today! Thanks for your hard work and your great product!
aglyons
Posts: 19
Joined: 29 Dec 2020, 18:42

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by aglyons »

I restarted Filebot-Node and saw the same thing.

I adjusted the changes to the start file, mainly wrapped the cars in quotes as that's how they are in the original. I missed that BASIC was in caps so fixed that. Upon loading the page I was prompted for web auth U and P but only once. Subsiquent loads didn't ask for credentials.

Now, while I did manage to get Node talking with Filebot as there is output in the CLI window, it appears that Filebot cannot access the DIR's configured.

Plot thickens.


Code: Select all

filebot -script 'fn:amc' '/Sync/Seedbox/2-To Process' --output /Sync/Seedbox/3-Processed --action duplicate -non-strict --order Airdate --conflict auto --lang en --def 'unsorted=y' 'skipExtract=y' 'seriesFormat=TV/{n}/{n} - {s00e00} - {t}' 'movieFormat=Movies/{n} {y}' 'excludeList=.excludes' --apply refresh -no-probe -no-index --log all --log-file '/usr/local/filebot-node/data/filebot.log'

------------------------------------------

OpenJDK 64-Bit Server VM warning: Option --illegal-access is deprecated and will be removed in a future release.
File does not exist: /Sync/Seedbox/2-To Process
Run script [fn:amc] at [Wed Apr 07 15:49:09 COT 2021]
Parameter: unsorted = y
Parameter: skipExtract = y
Parameter: seriesFormat = TV/{n}/{n} - {s00e00} - {t}
Parameter: movieFormat = Movies/{n} {y}
Parameter: excludeList = .excludes
Invalid usage: output folder must exist and must be a directory: /Sync/Seedbox/3-Processed
Abort (×_×)

------------------------------------------

[Process error]

🔺 Exit Code: 4
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

These errors would suggest some kind of generic file system / user permission issue somewhat unrelated to FileBot Node or FileBot specifically:

Code: Select all

File does not exist: /Sync/Seedbox/2-To Process
...
Invalid usage: output folder must exist and must be a directory: /Sync/Seedbox/3-Processed
...


EDIT:

This is likely related to DSM 6.2.4 encouraging you to disable the admin user. FileBot Node runs filebot as admin so strange things may happen if admin is disabled.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

FileBot Node for Synology DSM 6.2.4 is available for testing: filebot-node-0.4.0-noarch.spk


:!: You may need to uninstall older versions of FileBot Node, before manually installing the latest beta package. Upgrading the package may result in broken permissions caused by previously created user data.


:!: FileBot Node is no longer integrated into the DSM nginx service. You therefore cannot use Quick Connect to access the FileBot Node WebUI.



EDIT:

You may or may not need to disable CSRF authentication token feature for Synology authentication integration to work:
Image
:idea: Please read the FAQ and How to Request Help.
aglyons
Posts: 19
Joined: 29 Dec 2020, 18:42

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by aglyons »

Hey renoah, thanks for putting some time into fixing the debacle Synology has handed us all.

I also use MODs to trigger filebot so I can have a one-click desktop shortcut. Perhaps you guys can chat for a bit and figure out how to incorporate what he's been doing with what you're doing. His approach basically triggers the same script that Node creates for the scheduled tasks. While great, any changes to Node creates a new schedule and breaks MODs which requires editing a text file via SSH in order to get it to run the right script.

https://github.com/vletroye/SynoPackage ... t%20Runner
colpepe
Posts: 4
Joined: 06 Apr 2021, 19:36

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by colpepe »

@rednoah - my syno 'admin' account is disabled and FBN runs fine in my case, just for the record.

@Aglyons - That is the first I heard about the MODS tools, thanks for that. Is there any reason you wouldn't just schedule filebot to run a task every 5-10 mins so you don't have to hassle with running it manually? I am just curious of your usage scenario.
Dmx
Posts: 7
Joined: 26 Feb 2021, 18:55

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by Dmx »

Hi rednoah,

thanks for still putting effort in DSM 6.2.4, I think its good since DSM 7 is still some time away. I saw that you uploaded a testversion and I want to give you feedback on it. This is v. 0.4.0 from your previous post. I had the latest online available version installed and updated it manually via Syno Package Center. A new window pops up after upgrading saying you need to give the System Internal Filebot User Read/Write Permissions on your working shares. This is very good advice at this point as otherwise it wouldn't work (see next paragraph)!

Filebot Node opens in a new Browser tab from now on (not like before in the Syno GUI), so I could confirm that authentication works properly again without changing anything manually. I tried a Dry run in Node before giving the Filebot User Read/Write access just to test and as expected it gave me an error due to missing permissions. Gave permission to the user and rebooted (the reboot was necessary at least for me, otherwise I would still get the same Permission error). Creating Tasks and Scheduling them seems to also work properly with this version.

Only thing I noticed which I'm not sure of is that a warning appears if you do something in the Filebot Node GUI saying you shouldn't run filebot.sh as root. I believe it wasn't like this in the old version, only if you really ran filebot.sh as root via SSH or your scheduler for example.

Example from opening the Filebot Node GUI:

Code: Select all

$ filebot -version
/volume1/@appstore/filebot/filebot.sh must NOT run as root
FileBot 4.9.3 (r8338) / OpenJDK Runtime Environment 14.0.1 / Linux 4.4.59+ (amd64)
Additional Sidenote:
- I didn't need to disable CSRF protection
darko
Posts: 3
Joined: 08 Apr 2021, 07:22

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by darko »

Thank you for this update, @rednoah! Like many others, I was stuck with DSM 6.2.4 and a broken FileBot Node installation. I now have FileBot Node 0.4.0 working smoothly with DSM 6.2.4.

Here are my notes with respect to getting 0.4.0 working:

• I needed to change permissions on /var/packages/filebot-node/target/*, as I was getting an access denied error when trying to set my license and when writing to log files.
• I also needed to do the same for .excludes files, which FileBot was not able to write to at first.
• For new users, I think the new UI and process around scheduling tasks will be a bit counter-intuitive compared to previous versions. If this is temporary/beta-only, disregard.

Thanks again for getting this out quickly!
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

Dmx wrote: 08 Apr 2021, 18:05 Only thing I noticed which I'm not sure of is that a warning appears if you do something in the Filebot Node GUI saying you shouldn't run filebot.sh as root. I believe it wasn't like this in the old version, only if you really ran filebot.sh as root via SSH or your scheduler for example.
The filebot must NOT run as root warning should go away once you create a custom admin user and disable the default admin user.

darko wrote: 08 Apr 2021, 23:03 I needed to change permissions on /var/packages/filebot-node/target/*, as I was getting an access denied error when trying to set my license and when writing to log files.
Executed via FileBot Node or via SSH? The filebot process executed by FileBot Node will run as $SYNOPKG_USERNAME:users but if you run it via SSH or Task Scheduler, then you can run it as different users with different permissions which may lead to permission conflicts.

darko wrote: 08 Apr 2021, 23:03 For new users, I think the new UI and process around scheduling tasks will be a bit counter-intuitive compared to previous versions. If this is temporary/beta-only, disregard.
FileBot Node can no longer integrate into DSM because we can no longer modify DSM internal files. The Scheduled Tasks feature is now the same across all platforms, i.e. it'll give you a command but you'll have to paste it into Task Scheduler yourself.
:idea: Please read the FAQ and How to Request Help.
darko
Posts: 3
Joined: 08 Apr 2021, 07:22

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by darko »

Executed via FileBot Node or via SSH? The filebot process executed by FileBot Node will run as $SYNOPKG_USERNAME:users but if you run it via SSH or Task Scheduler, then you can run it as different users with different permissions which may lead to permission conflicts.
This was when executing FileBot via the FileBot Node GUI to test everything before I created a scheduled task.

Thanks for the specifics around what user/group is used in these cases; I'll be able to go back and tighten down permissions a bit.

Regarding what you know so far about the changes to DSM, do you think this change is permanent, or will DSM 7 restore some flexibility? I ask because this change seems to have garnered a lot of negative press, and I'm wondering if Synology is considering taking a different approach in the future.
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

DSM 7.0 is all about implementing good security practices. Forcing each package to run as its own user / group with limited permissions is easy and just makes sense. Synology won't (and shouldn't) go back on that. Running everything as root was never a good idea in the first place.
:idea: Please read the FAQ and How to Request Help.
darko
Posts: 3
Joined: 08 Apr 2021, 07:22

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by darko »

I agree with you there, but it's regrettable that FileBot Node can no longer leverage the DSM UI. Or maybe FileBot Node was using it in some non-standard ways. Regardless, it feels like they made a change with good intentions that has had some negative consequences for package authors.

On the other hand, much of the value of FileBot is in the set-it-and-forget-it paradigm it affords, and once things are running smoothly I typically don't need to interact with it much. :D
Dmx
Posts: 7
Joined: 26 Feb 2021, 18:55

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by Dmx »

rednoah wrote: 09 Apr 2021, 02:54
Dmx wrote: 08 Apr 2021, 18:05 Only thing I noticed which I'm not sure of is that a warning appears if you do something in the Filebot Node GUI saying you shouldn't run filebot.sh as root. I believe it wasn't like this in the old version, only if you really ran filebot.sh as root via SSH or your scheduler for example.
The filebot must NOT run as root warning should go away once you create a custom admin user and disable the default admin user.
Hi,
ah, I see. Thanks for the explanation. At least I know why this is appearing now. I'm not going to disable the default admin use though, so it doesn't really matter in the end 👍
Thanks again for the new version.
davemccreath
Posts: 31
Joined: 08 Oct 2019, 12:03

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by davemccreath »

rednoah. Many thanks for the update but for me it failed.

I set up a custom administrator account several months ago and disabled the old admin account.

I manually installed the filebot-node-0.4.0-noarch.spk file and followed the on-screen instructions post the install allowing filebot Read/Write access to both the folder items are downloaded into and the folder that filebot moves them to after renaming.

For me, when executing filebot it opens in a new window with my IP address as the URL and a port number in the 54** series. The window says This site can't be reached, (IP Address) refused to connect.

Tried a reinstall and also reversing the file permissions all to no avail (with reboots of the system after each change).

Regards

Dave
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

What does the log say?
viewtopic.php?t=11030
:idea: Please read the FAQ and How to Request Help.
davemccreath
Posts: 31
Joined: 08 Oct 2019, 12:03

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by davemccreath »

rednoah

I've managed to SSH into the NAS but being a bit of a noob I've no idea how to open the file and I'm not so sure that it is there. Below is a dump from putty showing the contents of the directory.

Image
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by rednoah »

find / -name filebot-node.log will find the log for you:

Code: Select all

find / -name filebot-node.log
:idea: Please read the FAQ and How to Request Help.
davemccreath
Posts: 31
Joined: 08 Oct 2019, 12:03

Re: [FileBot Node] DSM 6.2.4 authentication workaround

Post by davemccreath »

Ahh, found it under /var/log/packages/filebot-node.log

Stopped the FileBot Node process at 1658hrs via DSM and restarted a couple of seconds later.

Image
Post Reply