Feature: Filebot on Alpine

[shadycuz]

I use a lot of the linuxserver.io docker images like deluge and plex. I would like to see if Filebot can run on that platform. @rednoah if you can get a hold of me I would like to sponsor this development.

[rednoah]

FileBot and all of it's dependencies should work out of the box on Alpine.

The current docker images aren't based on Alpine, but it shouldn't be much work other than changing the docker conf and creating custom image.

@see https://hub.docker.com/r/rednoah/filebot/

[shadycuz]

Got this far @rednoah

Code: Select all

Step 15/32 : RUN FILEBOT_SHA256=892723dcec8fe5385ec6665db9960e7c1a88e459a60525c02afb7f1195a50523  && FILEBOT_PACKAGE=filebot_${FILEBOT_VERSION}_amd64.deb  && curl -L -O https://downloads.sourceforge.net/project/filebot/filebot/FileBot_$FILEBOT_VERSION/$FILEBOT_PACKAGE  && echo "$FILEBOT_SHA256 *$FILEBOT_PACKAGE" | sha256sum --check --strict  && dpkg -i $FILEBOT_PACKAGE  && rm $FILEBOT_PACKAGE
 ---> Running in 9966544c59b2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0   369    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 19 22.1M   19 4384k    0     0  4384k      0  0:00:05  0:00:01  0:00:04 4428k
 46 22.1M   46 10.3M    0     0  5296k      0  0:00:04  0:00:02  0:00:02 5301k
 67 22.1M   67 14.9M    0     0  5098k      0  0:00:04  0:00:03  0:00:01 5115k
 92 22.1M   92 20.5M    0     0  5272k      0  0:00:04  0:00:04 --:--:-- 5285k
100 22.1M  100 22.1M    0     0  5673k      0  0:00:04  0:00:04 --:--:-- 5312k
filebot_4.7.9_amd64.deb: OK
dpkg: error processing archive filebot_4.7.9_amd64.deb (--install):
 package architecture (amd64) does not match system (musl-linux-amd64)
Errors were encountered while processing:
 filebot_4.7.9_amd64.deb
The command '/bin/sh -c FILEBOT_SHA256=892723dcec8fe5385ec6665db9960e7c1a88e459a60525c02afb7f1195a50523  && FILEBOT_PACKAGE=filebot_${FILEBOT_VERSION}_amd64.deb  && curl -L -O https://downloads.sourceforge.net/project/filebot/filebot/FileBot_$FILEBOT_VERSION/$FILEBOT_PACKAGE  && echo "$FILEBOT_SHA256 *$FILEBOT_PACKAGE" | sha256sum --check --strict  && dpkg -i $FILEBOT_PACKAGE  && rm $FILEBOT_PACKAGE' returned a non-zero code: 1
ERROR: Job failed: error executing remote command: command terminated with non-zero exit code: Error executing in Docker Container: 1|

The issue is described here https://github.com/tianon/gosu/issues/24 , but I don't know how to workaround it?

[rednoah]

This command-line is composed of many commands &&ed together. Which command is failing? (you'll have to run commands one by one and figure this one out)

[shadycuz]

Error msg...

Code: Select all

dpkg: error processing archive filebot_4.7.9_amd64.deb (--install):
 package architecture (amd64) does not match system (musl-linux-amd64)
Errors were encountered while processing:
 filebot_4.7.9_amd64.deb

I believe failing command is...

Code: Select all

dpkg -i $FILEBOT_PACKAGE

From the github issue I linked

A quick check revealed that dpkg --print-architecture on alpine:3.4 returns musl-linux-amd64 instead of amd64.

I'm on alpine 3.6 but getting the same thing. Not sure how to work around this.

[rednoah]

I see. Is there a way to make dpkg install the package anyway? The core of FileBot is platform-independent so it'll work if Java works. Some native dependencies like libmediainfo may or may not work.

[shadycuz]

Yes that worked but I'm still tweaking other things so it will work. I will post back if I get a working docker file.

[shadycuz]

Cool so I got the build to succeed

Code: Select all

$ docker exec deluge-${CI_JOB_ID} bash -c "filebot -version"
FileBot 4.7.9 (r4984) / OpenJDK Runtime Environment 1.8.0_131 / Linux 4.4.96-mainline-rev1 (amd64)

Dockerfile

Code: Select all

######### Setup Filebot
ENV FILEBOT_VERSION 4.7.9


WORKDIR /usr/share/filebot

# Add experimental repo for packages like chromaprint, inotify-tools etc.
RUN echo "http://nl.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories

# I changed all my repo's from 3.6 to edge, might not be needed
RUN sed -i -e 's/v3\.6/edge/g' /etc/apk/repositories

# not sure if this is needed
RUN apk update

# these need installed before installing filebot
RUN apk add --no-cache curl dpkg tar

#had to add this because the status file was missing
RUN mkdir -p /var/lib/dpkg/ && touch /var/lib/dpkg/status

RUN FILEBOT_SHA256=892723dcec8fe5385ec6665db9960e7c1a88e459a60525c02afb7f1195a50523 \
 && FILEBOT_PACKAGE=filebot_${FILEBOT_VERSION}_amd64.deb \
 && curl -L -O https://downloads.sourceforge.net/project/filebot/filebot/FileBot_$FILEBOT_VERSION/$FILEBOT_PACKAGE \
 && echo "$FILEBOT_SHA256 *$FILEBOT_PACKAGE" | sha256sum --check --strict \
 && dpkg --force-architecture -i $FILEBOT_PACKAGE \
 && rm $FILEBOT_PACKAGE

RUN apk add --no-cache \
    mediainfo \
    chromaprint \
    file \
    inotify-tools


ENV DOCKER_DATA /data
ENV DOCKER_VOLUME /volume1


ENV HOME $DOCKER_DATA
ENV JAVA_OPTS "-DuseGVFS=false -Djava.net.useSystemProxies=false -Dapplication.deployment=docker -Dapplication.dir=$DOCKER_DATA -Duser.home=$DOCKER_DATA -Djava.io.tmpdir=$DOCKER_DATA/tmp -Djava.util.prefs.PreferencesFactory=net.filebot.util.prefs.FilePreferencesFactory -Dnet.filebot.util.prefs.file=$DOCKER_DATA/prefs.properties"

WORKDIR $DOCKER_DATA

[shadycuz]

@rednoah So the build does work, but the AMC script is not working.

Code: Select all

filebot -script fn:amc --output "/decrypted" --action copy -non-strict "/downloads" --log-file amc.log
Locking /data/logs/amc.log
Run script [fn:amc] at [Fri Feb 09 11:31:20 EST 2018]
Argument[0]: /downloads
Segmentation fault (core dumped)
abc@automated-media-769f77c88-v4lg7:/var/log/deluge$ ls
amc.log  core.444

when trying to less the core.444 my terminal froze. I will post the log file to pastebin or something and send ya a link.

[shadycuz]

rednoah... I might have trouble getting this to you...

Code: Select all

du -hs core.444
302M    core.444

[shadycuz]

rednoah I will pm you a link to the core dump from gdrive.

[rednoah]

Segmentation fault (core dumped) means that native code is crashing the java process. That means that it's either the JVM that's crashing, or any of the 3rd party lib*.so are crashing the process.

I don't see a FROM line in your docker file. That's strange. What image is your docker based on? I'd start by using the official OpenJDK for Alpine images and see if those work well for running FileBot.

[shadycuz]

Segmentation fault (core dumped) means that native code is crashing the java process. That means that it's either the JVM that's crashing, or any of the 3rd party lib*.so are crashing the process.

I don't see a FROM line in your docker file. That's strange. What image is your docker based on? I'd start by using the official OpenJDK for Alpine images and see if those work well for running FileBot.

I'm using

Code: Select all

FROM lsiobase/alpine:3.6

. The java install part I got from the official openjdk alpine container. I see it's using alpine 3.7 and I'm using 3.6.

https://github.com/docker-library/openj ... Dockerfile

I will try again with some different setups. Thanks for pointing me in the right direction. I'm actually thinkings its the libraries and not the java, but I will do some testing.

[shadycuz]

rednoah I changed from alpine3.6 to 3.7 and got a tiny bit further

Code: Select all

Run script [fn:amc] at [Fri Feb 09 14:31:07 EST 2018]
Argument[0]: /downloads
Ignore extra: /downloads/free.documentary/Sample
Segmentation fault (core dumped)

[shadycuz]

evidently it's possible... https://github.com/flo313/filebot-watch ... Dockerfile

edit: This is using alpine:3.4 and I don't think it could run the AMC script either. He doesn't install all the libs needed.

[shadycuz]

@rednoah,

I have now changed my container to use your filebot container as my base. I then set up some other services.

Code: Select all

FROM rednoah/filebot

######### Setup S6-Overlay
ADD https://github.com/just-containers/s6-overlay/releases/download/v1.21.2.2/s6-overlay-amd64.tar.gz /tmp/
RUN tar xzf /tmp/s6-overlay-amd64.tar.gz -C /

######### Setup Deluge
RUN apt-get update && apt-get install -yq python-libtorrent \
	deluged deluge-web encfs unzip curl wget

I still can't run the AMC script, using this command.

Code: Select all

filebot -script fn:amc --output "/decrypted" --action copy -non-strict "/data/Downloads" --log-file amc.log

results in

Code: Select all

Locking /data/logs/amc.log
Run script [fn:amc] at [Sat Feb 17 14:55:15 EST 2018]
Argument[0]: /data/Downloads
Input: /data/Downloads/Stargate.Origins.S01E01.Episode.1.REPACK.1080p.WEB-DL.AAC2.0.H.264-AJP69/Stargate.Origins.S01E01.Episode.1.REPACK.1080p.WEB-DL.AAC2.0.H.264-AJP69.mkv
Group: [tvs:stargate origins] => [Stargate.Origins.S01E01.Episode.1.REPACK.1080p.WEB-DL.AAC2.0.H.264-AJP69.mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Stargate Origins, Stargate]
Killed

Would you like me to reopen this issue with a new post or in github?

[shadycuz]

Also this might be helpful

Code: Select all

FileBot 4.7.9 (r4984)
JNA Native: 5.1.0
MediaInfo: 0.7.93
7-Zip-JBinding: 9.20
Chromaprint: 1.4.2
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2018-02-04 (r514)
Groovy: 2.4.10
JRE: OpenJDK Runtime Environment 1.8.0_151
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 878 MB Max Memory / 57 MB Used Memory
OS: Linux (amd64)
Package: DOCKER
uname: Linux automated-media-59987c79bf-6jflh 4.4.96-mainline-rev1 #1 SMP Thu Nov 2 11:39:05 UTC 2017 x86_64 GNU/Linux
Done ヾ(＠⌒ー⌒＠)ノ

[shadycuz]

OOM killed, Increased my limits and it worked *blush*