filebot-watcher docker: persist open subtitles configuration after container restart

[sabinder62]

Hi rednoah,

I have been using the filebot-watcher container for many months and it works perfectly. One thing I cannot figure out is how to get the opensubtitles configuration to persist after I restart the container.

I do this by running the command

Code: Select all

filebot -script fn:configure

in the container and it always confirms success. However I update all my containers and restart them on a bi-weekly basis and have to run the command again every time it starts back up.

Any idea on what I might be doing wrong?

[rednoah]

You are most likely not storing application data to a persistent volume.

Code: Select all

docker run --rm -it -v $PWD:/volume1 -v data:/data rednoah/filebot:watcher /volume1/input --output /volume1/output

-v data:/data ... mount persistent named volume "data" available inside the container as /data (from inside the container) so that the filebot instance running inside the container can remember things. If something is wrong here, or deleted, then filebot will not be able to remember things.



What bind mount paths are you using when launching the container?


What does the console output say when you run filebot -script fn:configure to write the configuration file? (EDIT: nevermind; we already figured out where preferences are stored; see below)



EDIT:

Those settings are probably written to $HOME/.java/.userPrefs but since $HOME is a variable that depends on what user you're running as, I have no idea where the data is actually stored and if it is indeed stored to /data somewhere. I'll look into adding configuration to the container to make this more explicit.

e.g. if you configure filebot like so:

Code: Select all

docker run --rm -it -v data:/data rednoah/filebot -script fn:configure --def osdbUser=USERNAME --def osdbPwd=PASSWORD

then those settings are stored to /data/filebot/.java/.userPrefs so you just need to make sure you don't lose or delete that information and configure your docker instance accordingly. This should typically not be an issue since all examples ensure that /data is mapped to a named volume and thus not lost on container shutdown.

[sabinder62]

Thanks very much for the rapid response as always.

I can confirm I mount the /data folder as a persistent volume as per your instructions and on the docker page readme so that isn't the issue.

Your suggestion looks good!

An alternative might be to pass them as environment variables? There is a 3rd party docker container that has implemented this for the full GUI:

https://github.com/jlesage/docker-filebot

I haven't tried it myself as I much prefer to use your official containers.

Look forward to whatever you decide to update and thanks as always for your hard work!

[rednoah]

Adding docker-specific environment variables is unfortunately not planned. Especially since we have not yet found out why settings are lost in your specific case. Moreover, both license key and user settings are stored to /data so you'd typically have trouble with either both or neither.


You'll want to check the /data folder and find the files that contain your OpenSubtitles configuration and then find out why those files are deleted after container restart:

Code: Select all

$ cat /data/filebot/.java/.userPrefs/net/filebot/login/prefs.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE map SYSTEM "http://java.sun.com/dtd/preferences.dtd">
<map MAP_XML_VERSION="1.0">
  <entry key="OpenSubtitles" value="..."/>
</map>

The correct solution is to call filebot -script fn:configure correctly via docker run --rm -it -v data:/data rednoah/filebot <filebot options> commands to permanently set these settings within the docker context, so this should just work:

Code: Select all

docker run --rm -it -v data:/data rednoah/filebot -script fn:configure --def osdbUser=USERNAME --def osdbPwd=PASSWORD

Notably, you can use the GUI (inside a docker container) to set your OpenSubtitles login details for all docker containers that share the same /data folder, so this should just work:

Code: Select all

docker run --rm -it -v data:/data -p 5454:5454 rednoah/filebot:xpra

EDIT:

I've updated the FAQ:
https://github.com/filebot/filebot-dock ... in-details

[sabinder62]

I think I see the issue, at least how I was doing it.

I was logging into the docker container command line with:

Code: Select all

docker exec -it filebot-watcher /bin/bash

Then running filebot -script fn:configure

This gives the following output (after adding username and password):

Code: Select all

Set OpenSubtitles login details
Checking... OK

limit_check_by: user_id
user_id: 9255112
global_24h_download_limit: 200
client_ip: 137.220.92.90
client_24h_download_count: 0
client_download_quota: 200
client_24h_download_limit: 200
Done ヾ(＠⌒ー⌒＠)ノ

however, if I navigate to /data/filebot/.java/.userPrefs/net/filebot/login/prefs.xml within the container, the file is essentially empty so it isn't saving the details where expected:

Code: Select all

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE map SYSTEM "http://java.sun.com/dtd/preferences.dtd">
<map MAP_XML_VERSION="1.0"/>

Sorry slightly noob question, I have used docker compose to run filebot-watcher rather than a direct run command as you suggest. What would I edit to make sure this works? I have tried using && or | operators as suggested here: https://www.baeldung.com/ops/docker-com ... e-commands but can't make it work. Here is an excerpt of my compose file with the filebot-watcher section (I use it with .env to pass through variables you can see below):

Code: Select all

version: "2.3"
services:
    filebotTV:
        container_name: filebot-watcher
        image: rednoah/filebot:watcher
        network_mode: host
        environment:
            - PUID=${PUID}
            - PGID=${PGID}
            - TZ=${TZ}
        volumes:
            - ${CONFIG}/FileBotTV:/data
            - ${DATA3}:/data3
        command: /data3/torrents/completeTV --output /data3/torrents/processedTV --action hardlink --conflict auto -non-strict --log-file "/data3/torrents/processedTV/amc.log" --def excludeList="/data3/torrents/processedTV/amc.txt" clean=y unsorted=n artwork=n subtitles=en deleteAfterExtract=n storeReport=/data3/torrents/processedTV/reports reportError=y pushover=XXXX "seriesFormat=/data3/media/{plex.id} [{vf}, {vs}, {vc}, {bitrate}]"
        restart: unless-stopped

[rednoah]

Looks like you are already using the docker command-line tool:

Code: Select all

docker exec -it filebot-watcher /bin/bash

This command will give you a root shell. However, the normal entry point will switch user to your PUID user and then run filebot as that. $HOME is thus different depending on the context, and so running filebot -script fn:configure as root has no effect on the configuration for the user that is actually running filebot commands.


So I'd just use the docker command-line tool to set the settings:

Code: Select all

docker run --rm -it -v data:/data rednoah/filebot -script fn:configure --def osdbUser=USERNAME --def osdbPwd=PASSWORD

Note that all example code uses:

Code: Select all

-v data:/data

But you do this in your compose file:

Code: Select all

volumes:
            - ${CONFIG}/FileBotTV:/data

So you need to adjust your docker call accordingly:

Code: Select all

docker run --rm -it -v /path/to/FileBotTV:/data rednoah/filebot -script fn:configure --def osdbUser=USERNAME --def osdbPwd=PASSWORD

I cannot give you a copy & paste solution because ${CONFIG} is an unknown variable to me.

[sabinder62]

Understood, sorry showing my limited knowledge of docker there!

I have successfully run the command with the right path. Prefs.xml now has the values and the settings have persisted on a container restart.

Thanks very much for your help as always!

[sabinder62]

P.S. Wanted to donate for your support on this issue and my last one but just read that you disabled it. (I already purchased a lifetime licence)

Lots of gratitude to you, sir! I hope the project stays around forever.

[rednoah]

Understood, sorry showing my limited knowledge of docker there!

No worries. docker is complicated. Linux is complicated. It's systems on top of systems on top of systems. Nobody knows exactly what's going on at any particular point in time. If everything happens to just work then we don't care. If something doesn't work, then we have to systematically narrow down step by step until we figure it out. Can't know. Could be anything. That's just how it goes.

[sabinder62]

Well your constant patience is still very much appreciated! Thanks again