Feature: Filebot on Alpine

All your suggestions, requests and ideas for future development
Post Reply
shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Feature: Filebot on Alpine

Post by shadycuz » 18 Dec 2017, 13:06

I use a lot of the linuxserver.io docker images like deluge and plex. I would like to see if Filebot can run on that platform. @rednoah if you can get a hold of me I would like to sponsor this development.

User avatar
rednoah
The Source
Posts: 14114
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Feature: Filebot on Alpine

Post by rednoah » 18 Dec 2017, 13:25

FileBot and all of it's dependencies should work out of the box on Alpine.

The current docker images aren't based on Alpine, but it shouldn't be much work other than changing the docker conf and creating custom image.

@see https://hub.docker.com/r/rednoah/filebot/
:idea: Please read the FAQ and How to Request Help.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 27 Dec 2017, 17:52

Got this far @rednoah

Code: Select all

Step 15/32 : RUN FILEBOT_SHA256=892723dcec8fe5385ec6665db9960e7c1a88e459a60525c02afb7f1195a50523  && FILEBOT_PACKAGE=filebot_${FILEBOT_VERSION}_amd64.deb  && curl -L -O https://downloads.sourceforge.net/project/filebot/filebot/FileBot_$FILEBOT_VERSION/$FILEBOT_PACKAGE  && echo "$FILEBOT_SHA256 *$FILEBOT_PACKAGE" | sha256sum --check --strict  && dpkg -i $FILEBOT_PACKAGE  && rm $FILEBOT_PACKAGE
 ---> Running in 9966544c59b2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0   369    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 19 22.1M   19 4384k    0     0  4384k      0  0:00:05  0:00:01  0:00:04 4428k
 46 22.1M   46 10.3M    0     0  5296k      0  0:00:04  0:00:02  0:00:02 5301k
 67 22.1M   67 14.9M    0     0  5098k      0  0:00:04  0:00:03  0:00:01 5115k
 92 22.1M   92 20.5M    0     0  5272k      0  0:00:04  0:00:04 --:--:-- 5285k
100 22.1M  100 22.1M    0     0  5673k      0  0:00:04  0:00:04 --:--:-- 5312k
filebot_4.7.9_amd64.deb: OK
dpkg: error processing archive filebot_4.7.9_amd64.deb (--install):
 package architecture (amd64) does not match system (musl-linux-amd64)
Errors were encountered while processing:
 filebot_4.7.9_amd64.deb
The command '/bin/sh -c FILEBOT_SHA256=892723dcec8fe5385ec6665db9960e7c1a88e459a60525c02afb7f1195a50523  && FILEBOT_PACKAGE=filebot_${FILEBOT_VERSION}_amd64.deb  && curl -L -O https://downloads.sourceforge.net/project/filebot/filebot/FileBot_$FILEBOT_VERSION/$FILEBOT_PACKAGE  && echo "$FILEBOT_SHA256 *$FILEBOT_PACKAGE" | sha256sum --check --strict  && dpkg -i $FILEBOT_PACKAGE  && rm $FILEBOT_PACKAGE' returned a non-zero code: 1
ERROR: Job failed: error executing remote command: command terminated with non-zero exit code: Error executing in Docker Container: 1|
The issue is described here https://github.com/tianon/gosu/issues/24 , but I don't know how to workaround it?

User avatar
rednoah
The Source
Posts: 14114
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Feature: Filebot on Alpine

Post by rednoah » 27 Dec 2017, 18:44

This command-line is composed of many commands &&ed together. Which command is failing? (you'll have to run commands one by one and figure this one out)
:idea: Please read the FAQ and How to Request Help.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 27 Dec 2017, 19:08

Error msg...

Code: Select all

dpkg: error processing archive filebot_4.7.9_amd64.deb (--install):
 package architecture (amd64) does not match system (musl-linux-amd64)
Errors were encountered while processing:
 filebot_4.7.9_amd64.deb
I believe failing command is...

Code: Select all

dpkg -i $FILEBOT_PACKAGE


From the github issue I linked
A quick check revealed that dpkg --print-architecture on alpine:3.4 returns musl-linux-amd64 instead of amd64.
I'm on alpine 3.6 but getting the same thing. Not sure how to work around this.

User avatar
rednoah
The Source
Posts: 14114
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Feature: Filebot on Alpine

Post by rednoah » 27 Dec 2017, 19:16

I see. Is there a way to make dpkg install the package anyway? The core of FileBot is platform-independent so it'll work if Java works. Some native dependencies like libmediainfo may or may not work.
:idea: Please read the FAQ and How to Request Help.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 27 Dec 2017, 19:30

Yes that worked but I'm still tweaking other things so it will work. I will post back if I get a working docker file.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 27 Dec 2017, 20:21

Cool so I got the build to succeed

Code: Select all

$ docker exec deluge-${CI_JOB_ID} bash -c "filebot -version"
FileBot 4.7.9 (r4984) / OpenJDK Runtime Environment 1.8.0_131 / Linux 4.4.96-mainline-rev1 (amd64)
Dockerfile

Code: Select all

######### Setup Filebot
ENV FILEBOT_VERSION 4.7.9


WORKDIR /usr/share/filebot

# Add experimental repo for packages like chromaprint, inotify-tools etc.
RUN echo "http://nl.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories

# I changed all my repo's from 3.6 to edge, might not be needed
RUN sed -i -e 's/v3\.6/edge/g' /etc/apk/repositories

# not sure if this is needed
RUN apk update

# these need installed before installing filebot
RUN apk add --no-cache curl dpkg tar

#had to add this because the status file was missing
RUN mkdir -p /var/lib/dpkg/ && touch /var/lib/dpkg/status

RUN FILEBOT_SHA256=892723dcec8fe5385ec6665db9960e7c1a88e459a60525c02afb7f1195a50523 \
 && FILEBOT_PACKAGE=filebot_${FILEBOT_VERSION}_amd64.deb \
 && curl -L -O https://downloads.sourceforge.net/project/filebot/filebot/FileBot_$FILEBOT_VERSION/$FILEBOT_PACKAGE \
 && echo "$FILEBOT_SHA256 *$FILEBOT_PACKAGE" | sha256sum --check --strict \
 && dpkg --force-architecture -i $FILEBOT_PACKAGE \
 && rm $FILEBOT_PACKAGE

RUN apk add --no-cache \
    mediainfo \
    chromaprint \
    file \
    inotify-tools


ENV DOCKER_DATA /data
ENV DOCKER_VOLUME /volume1


ENV HOME $DOCKER_DATA
ENV JAVA_OPTS "-DuseGVFS=false -Djava.net.useSystemProxies=false -Dapplication.deployment=docker -Dapplication.dir=$DOCKER_DATA -Duser.home=$DOCKER_DATA -Djava.io.tmpdir=$DOCKER_DATA/tmp -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory -Dnet.filebot.util.prefs.file=$DOCKER_DATA/prefs.properties"

WORKDIR $DOCKER_DATA

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 09 Feb 2018, 16:42

@rednoah So the build does work, but the AMC script is not working.

Code: Select all

filebot -script fn:amc --output "/decrypted" --action copy -non-strict "/downloads" --log-file amc.log
Locking /data/logs/amc.log
Run script [fn:amc] at [Fri Feb 09 11:31:20 EST 2018]
Argument[0]: /downloads
Segmentation fault (core dumped)
[email protected]:/var/log/deluge$ ls
amc.log  core.444
when trying to less the core.444 my terminal froze. I will post the log file to pastebin or something and send ya a link.
Last edited by shadycuz on 09 Feb 2018, 19:31, edited 1 time in total.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 09 Feb 2018, 16:52

rednoah... I might have trouble getting this to you...

Code: Select all

du -hs core.444
302M    core.444

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 09 Feb 2018, 17:07

rednoah I will pm you a link to the core dump from gdrive.

User avatar
rednoah
The Source
Posts: 14114
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Feature: Filebot on Alpine

Post by rednoah » 09 Feb 2018, 18:32

Segmentation fault (core dumped) means that native code is crashing the java process. That means that it's either the JVM that's crashing, or any of the 3rd party lib*.so are crashing the process.

I don't see a FROM line in your docker file. That's strange. What image is your docker based on? I'd start by using the official OpenJDK for Alpine images and see if those work well for running FileBot.
:idea: Please read the FAQ and How to Request Help.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 09 Feb 2018, 18:46

rednoah wrote:
09 Feb 2018, 18:32
Segmentation fault (core dumped) means that native code is crashing the java process. That means that it's either the JVM that's crashing, or any of the 3rd party lib*.so are crashing the process.

I don't see a FROM line in your docker file. That's strange. What image is your docker based on? I'd start by using the official OpenJDK for Alpine images and see if those work well for running FileBot.
I'm using

Code: Select all

FROM lsiobase/alpine:3.6
. The java install part I got from the official openjdk alpine container. I see it's using alpine 3.7 and I'm using 3.6.

https://github.com/docker-library/openj ... Dockerfile

I will try again with some different setups. Thanks for pointing me in the right direction. I'm actually thinkings its the libraries and not the java, but I will do some testing.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 09 Feb 2018, 19:37

rednoah I changed from alpine3.6 to 3.7 and got a tiny bit further

Code: Select all

Run script [fn:amc] at [Fri Feb 09 14:31:07 EST 2018]
Argument[0]: /downloads
Ignore extra: /downloads/free.documentary/Sample
Segmentation fault (core dumped)

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 09 Feb 2018, 19:49

evidently it's possible... https://github.com/flo313/filebot-watch ... Dockerfile

edit: This is using alpine:3.4 and I don't think it could run the AMC script either. He doesn't install all the libs needed.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 17 Feb 2018, 20:00

@rednoah,

I have now changed my container to use your filebot container as my base. I then set up some other services.

Code: Select all

FROM rednoah/filebot

######### Setup S6-Overlay
ADD https://github.com/just-containers/s6-overlay/releases/download/v1.21.2.2/s6-overlay-amd64.tar.gz /tmp/
RUN tar xzf /tmp/s6-overlay-amd64.tar.gz -C /

######### Setup Deluge
RUN apt-get update && apt-get install -yq python-libtorrent \
	deluged deluge-web encfs unzip curl wget
I still can't run the AMC script, using this command.

Code: Select all

filebot -script fn:amc --output "/decrypted" --action copy -non-strict "/data/Downloads" --log-file amc.log
results in

Code: Select all

Locking /data/logs/amc.log
Run script [fn:amc] at [Sat Feb 17 14:55:15 EST 2018]
Argument[0]: /data/Downloads
Input: /data/Downloads/Stargate.Origins.S01E01.Episode.1.REPACK.1080p.WEB-DL.AAC2.0.H.264-AJP69/Stargate.Origins.S01E01.Episode.1.REPACK.1080p.WEB-DL.AAC2.0.H.264-AJP69.mkv
Group: [tvs:stargate origins] => [Stargate.Origins.S01E01.Episode.1.REPACK.1080p.WEB-DL.AAC2.0.H.264-AJP69.mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Stargate Origins, Stargate]
Killed
Would you like me to reopen this issue with a new post or in github?
Last edited by shadycuz on 17 Feb 2018, 20:21, edited 1 time in total.

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 17 Feb 2018, 20:05

Also this might be helpful

Code: Select all

FileBot 4.7.9 (r4984)
JNA Native: 5.1.0
MediaInfo: 0.7.93
7-Zip-JBinding: 9.20
Chromaprint: 1.4.2
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2018-02-04 (r514)
Groovy: 2.4.10
JRE: OpenJDK Runtime Environment 1.8.0_151
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 878 MB Max Memory / 57 MB Used Memory
OS: Linux (amd64)
Package: DOCKER
uname: Linux automated-media-59987c79bf-6jflh 4.4.96-mainline-rev1 #1 SMP Thu Nov 2 11:39:05 UTC 2017 x86_64 GNU/Linux
Done ヾ(@⌒ー⌒@)ノ

shadycuz
Posts: 19
Joined: 15 Sep 2016, 07:09

Re: Feature: Filebot on Alpine

Post by shadycuz » 17 Feb 2018, 20:48

OOM killed, Increased my limits and it worked *blush*

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests