filebot crashing(?) on Raspberry Pi 4

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

Hi,

I've used filebot successfully for a while now and it works for most files. But, every once and a while it will fail without any reason that I can see and I'd like to figure out what's going on.

Here's the filebot version info:

Code: Select all

pi@raspberrypi:~ $ filebot.sh -script 'fn:sysinfo'

FileBot 4.9.0 (r7255)
JNA Native: 6.1.0
MediaInfo: 19.04
p7zip: p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)
unrar: UNRAR 5.61 beta 1 freeware
FFprobe: 4.1.4-1+rpt7~deb10u1
Chromaprint: 1.4.3
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-03-16 (r625)
Groovy: 3.0.2
JRE: LibericaJDK Runtime Environment 12.0.1-BellSoft
JVM: 32-bit LibericaJDK Server VM
CPU/MEM: 4 Core / 519 MB Max Memory / 25 MB Used Memory
OS: Linux (arm)
HW: Linux raspberrypi 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux
STORAGE: ext4 [/] @ 25 GB | autofs [/media/MEDIA] @ 2.2 TB | cifs [/media/MEDIA] @ 2.2 TB
DATA: /home/pi/filebot/data/pi
Package: TAR
Activate License [P8129***] on [Mon Mar 23 10:51:59 EDT 2020]
License: FileBot License P8129*** (Valid-Until: 2020-06-07)
Done ヾ(@⌒ー⌒@)ノ
Here's the output from running filebot:

Code: Select all

/home/pi/filebot/filebot.sh -script fn:amc --output $MEDIA --log all --log-file /tmp/filebot.log \
    --action move --conflict auto -non-strict "$FILE_PATH" --def artwork=y \
    seriesFormat="$MEDIA/TV/{ny}/Season {s}\{n} - S{s.pad(2)}E{e.pad(2)} - {t}" \
    movieFormat="$MEDIA/DVD/{n} ({y})/{n} ({y})"

Run script [fn:amc] at [Mon Mar 23 10:52:36 EDT 2020]
Parameter: artwork = y
Parameter: seriesFormat = /media/MEDIA/TV/{ny}/Season {s}\{n} - S{s.pad(2)}E{e.pad(2)} - {t}
Parameter: movieFormat = /media/MEDIA/DVD/{n} ({y})/{n} ({y})
Argument[0]: /media/MEDIA/INCOMING
File does not exist: /media/MEDIA/INCOMING/<MOVIE1>.2020.1080p.WEB-DL.DD5.1.H264-FGT/<MOVIE1>.2020.1080p.WEB-DL.DD5.1.H264-FGT.mkv
Input: /media/MEDIA/INCOMING/<MOVIE2>.2020.720p.WEBRip.800MB.x264-GalaxyRG[TGx]/<MOVIE2>.2020.720p.WEBRip.800MB.x264-GalaxyRG.mkv
Input: /media/MEDIA/INCOMING/<MOVIE2>.2020.720p.WEBRip.800MB.x264-GalaxyRG[TGx]/<MOVIE2>.2020.720p.WEBRip.800MB.x264-GalaxyRG.srt
Input: /media/MEDIA/INCOMING/<MOVIE3>.2020.720p.webrip.hevc.x265.rmteam.mkv

Killed
The 3 movie files are 3Gig, 500M and 800M (in order)

The script hangs for maybe 1-2 minutes and then reports that "Killed" message and exists. I didn't abort the script.

I'm not sure if this issue is a problem with filebot or a problem with one of the media libraries or the raspberry PI 4.. just looking to get more information
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

:?: Is there any specific reproducible pattern? Does it always end up getting killed when processing that specific file?


This message is definitely not printed by FileBot code:

Code: Select all

Killed
:arrow: It would suggest that the OS is killing the filebot process for some reason.


:idea: If you think that libmediainfo might somehow be related, then you can use -no-probe to make FileBot not read files, and thus process files based on file path information only.
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

Thanks for the fast reply!!

I used the -no-probe option and same results (~2 minutes before the "Killed" message and the script exits). I do see an additional log entry showing up before the first 2 movies:

Code: Select all

Failed to read media characteristics: UnsupportedOperationException: MediaCharacteristicsParser::open
File does not exist: /media/MEDIA/INCOMING/<MOVIE1>.2020.1080p.WEB-DL.DD5.1.H264-FGT/<MOVIE1>.2020.1080p.WEB-DL.DD5.1.H264-FGT.mkv
Failed to read media characteristics: UnsupportedOperationException: MediaCharacteristicsParser::open
Input: /media/MEDIA/INCOMING/<MOVIE2>.2020.720p.WEBRip.800MB.x264-GalaxyRG[TGx]/<MOVIE2>.2020.720p.WEBRip.800MB.x264-GalaxyRG.mkv
This log entry seems really odd to me.. It's a valid file but why would it say it doesn't exist? I'm not passing this file in as an argument - just the entire "/media/MEDIA/INCOMING/" path.

Code: Select all

File does not exist: /media/MEDIA/INCOMING/<MOVIE1>.2020.1080p.WEB-DL.DD5.1.H264-FGT/<MOVIE1>.2020.1080p.WEB-DL.DD5.1.H264-FGT.mkv
I've updated my Raspberry PI 4 OS and of course filebot.. not sure what else I can try. But, I figured if this was happening to me than it would likely be happening to others so I wanted to try and figure it out.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

>> Is there any specific reproducible pattern? Does it always end up getting killed when processing that specific file?

To this question, when it does happen like in this case, the filebot script will always fail. I usually manually fix the issue by manually moving/renaming the file and Kodi will correctly scrape it.

In the past I tried renaming the files & folders, thinking maybe there was some invalid character. But, that didn't do anything.

I did just manually move the first MOVIE1 file which was 3.5 Gig.. thinking maybe it was the root issue. But, running the script again on the next 2 still timed-out the same way.

I'll also restart my Raspberry Pi but it already reboots automatically every night.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

I also found an older post that I created which doesn't appear to be the same issue but still wanted to point it out: viewtopic.php?f=8&t=11310&p=47175&hilit ... i+4#p47175

In that case, there was a huge file 6.4 Gig and it wasn't showing up when using the mediainfo command

Code: Select all

filebot.sh -mediainfo -r . --format "{f} | {f.isFile()} | {f.isDirectory()}"
I ran the same command here (without the larger 3Gig file since I already moved it) and it works fine.. printing out true for the 2 .mkv files.

I know that thread ended with a question about my NAS and I guess I can't rule that out.. but, I can move/rename files just fine from the Raspberry PI and haven't seen any other issues with my Synology NAS.

Again, most files are handled just fine too.. this only happens once and a while so I was hoping to debug as much as possible to help both myself and others in the future if possible
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

Sorry for all of the back-to-back posts... last one for now.

Since the other thread I started had a question about my NAS - I decided to try and reproduce this all locally.

I copied 1 movie locally and ran the command:

Code: Select all

/home/pi/filebot/filebot.sh -script fn:amc --output /home/pi/MEDIA --log all --log-file /tmp/filebot.log  --action move --conflict auto -non-strict /home/pi/Downloads --def artwork=y seriesFormat="/home/pi/TV/{ny}/Season {s}\{n} - S{s.pad(2)}E{e.pad(2)} - {t}" movieFormat="/home/pi/DVD/{n} ({y})/{n} ({y})"
This time filebot doesn't exit out but instead just hangs for what's been 10+ minutes. There is a new log entry this time -- one about DiskStorageFactory..

There's also an entry about the cache being invalid but that might be expected since I had to hard reboot my PI the last time I ran this script.

Code: Select all

Run script [fn:amc] at [Mon Mar 23 13:17:35 EDT 2020]
Parameter: artwork = y
Parameter: seriesFormat = /home/pi/TV/{ny}/Season {s}\{n} - S{s.pad(2)}E{e.pad(2)} - {t}
Parameter: movieFormat = /home/pi/DVD/{n} ({y})/{n} ({y})
Argument[0]: /home/pi/Downloads
Mar 23, 2020 1:17:35 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file /home/pi/filebot/data/pi/cache/0/filebot_1.data is out of date, probably due to an unclean shutdown. Deleting index file /home/pi/filebot/data/pi/cache/0/filebot_1.index
Activate License [P8129***] on [Mon Mar 23 13:17:36 EDT 2020]
Input: /home/pi/Downloads/<MOVIE3>.2020.720p.webrip.hevc.x265.rmteam.mkv
Mar 23, 2020 1:17:51 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file /home/pi/filebot/data/pi/cache/0/data_0.data is out of date, probably due to an unclean shutdown. Deleting index file /home/pi/filebot/data/pi/cache/0/data_0.index
Any thoughts or ideas I can try?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

If -no-probe is working, then you will always get this message when FileBot tries to read some media file:

Code: Select all

Failed to read media characteristics: UnsupportedOperationException: MediaCharacteristicsParser::open

Indeed, this message means nothing. You will always get this warning if FileBot was killed at a bad time, now when restarted finds some of it's application data files in an inconsistent state, which is then automatically resolved by repopulating the cache from scratch:

Code: Select all

Mar 23, 2020 1:17:51 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file /home/pi/filebot/data/pi/cache/0/data_0.data is out of date, probably due to an unclean shutdown. Deleting index file /home/pi/filebot/data/pi/cache/0/data_0.index
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

just quick update: I've stopped trying to get filebot to work with these movies and manually moved them into place.

I do think there's an issue here.. now that I took my NAS completely out of the picture I feel like it has to be with the Raspberry PI 4 or filebot (or one of the underlying libraries of course).

I will try to keep these files around and can re-test with newer versions to see if anything fixes it and will try to update this thread if anything seems to work.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

So you've narrowed down the problem to a specific file? It always doesn't work for this specific file? But it does work for other files?

Can you provide this file somehow? e.g. PM me Google Drive link?
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

I did think I had it narrowed down to a specific file.. or at least it would always fail with the 3 movies I was trying to scrape. Since then I moved these files but it's still happening - this time on a different movie. Strangely, I've never had an issue scraping TV shows.. not sure if that's just a coincidence or it has to do with the file size that's being scraped. Also, just to clarify, most movies have also worked in the past using the same command. This just seems to happen once in a while and I was hoping to debug it for the next time.

Also, I don't think filebot is the issue.. more likely a library that it's using or something with my PI OS.

Question: what is filebot doing after printing this line?
"Input: /path/to/movie/abc.mkv"

In all failure cases, this is where the script hangs. Is there some additional logging that I could enable to see what's happening after this point? I do have "--log all" in my command.

Also, not sure if this is important or not but I noticed in another post you said this:

Code: Select all

 Make sure to use the OracleJDK and not the OpenJDK if you're on ARM architecture. The OpenJDK doesn't use JIT so it's slower by a magnitude or two
Is that still true today? I noticed I'm using the OpenJDK version:

Code: Select all

pi@raspberrypi:~ $ java --version
openjdk 12.0.1-BellSoft 2019-04-16
LibericaJDK Runtime Environment (build 12.0.1-BellSoft+12)
LibericaJDK Server VM (build 12.0.1-BellSoft+12, mixed mode)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

OpenJDK is the way to go starting with Java 9 an up. OpenJDK 14 is the latest, so it's worth a try.

You can add additional debug logging like so:
viewtopic.php?t=3913

The next step after Input: /file logging would be auto-detection, which is very CPU and memory intensive, so if the OS is somehow enforcing limits by killing memory hungry processes, that would be the moment:
https://github.com/filebot/scripts/blob ... roovy#L277

:idea: You could try to use --def ut_label=Movie to force movie mode and skip auto-detection.
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

cool thanks! I'll try all of these steps

I did find that extra logging and enabled it. sorry, i didn't want to include the full movie name but I did notice 1 thing in common between this current movie that's failing and the previous one that also failed.

I downloaded that moviedb.txt.xz file and looked and found both movie names in this list. However, both movies in the list were older movies (ie: same name but different year so not the same movie per-ce). Again, could just be coincidence though.

Code: Select all

[2020-03-26 10:14:55.117] Input: /media/MEDIA/INCOMING/MOVIE3.HDRip.XviD.AC3-EVO[TGx]/MOVIE3.HDRip.XviD.AC3-EVO.avi
[2020-03-26 10:14:55.479] Fetch resource: https://api.filebot.net/7255/data/moviedb.txt.xz
[2020-03-26 10:14:57.829] [GET https://api.filebot.net/7255/data/moviedb.txt.xz] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a17b20fc23e80d-LAX
Server: cloudflare
Connection: keep-alive
Last-Modified: Thu, 26 Mar 2020 05:44:19 GMT
Date: Thu, 26 Mar 2020 14:14:57 GMT
Accept-Ranges: bytes
CF-Cache-Status: HIT
Cache-Control: max-age=2678400
ETag: "1d9edc-5a1bb79e86147"
Vary: User-Agent, Accept-Encoding
Set-Cookie:
Expires: Sat, 28 Mar 2020 07:56:12 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 1941212
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/x-xz
[2020-03-26 10:15:00.071] Received 1.9 MB
[8d9b2f1513d5ca1ce43d45355bd*****]

(SCRIPT HANGS HERE FOREVER)
>> which is very CPU and memory intensive
I could see that being a problem for the PI.. I've only got the 2 Gig version too
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

The content of the data file wouldn't affect processing. Especially not the OS deciding to kill our process.


:?: Do you have a SWAP partition? i.e. unlimited but slow RAM? Because if not, and something really hits the physical memory limit, then the OS might start cleaning house, which it would never do on a normal Desktop PC setup.


Along those lines, try running with -Xmx384 and -Xms384, to make sure memory neither grows nor shrinks at runtime:

Code: Select all

export JAVA_OPTS="-Xmx384 -Xms384"
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

>> try running with -Xmx384 and -Xms384

I upgraded java to "openjdk 14 2020-03-16" and set the min/max size (the above commands didn't work until I realized I needed to add "M" after each one! I'm using:

Code: Select all

export JAVA_OPTS="-Xms512M -Xmx512M"
filebot still hangs and is eventually "Killed" (not sure by which process).

I even added

Code: Select all

--def ut_label=Movie
to the command and got the same result too (full logs minus movie name below).

>> Do you have a SWAP partition?
I'm not exactly sure how to check or disable this..

Code: Select all

pi@raspberrypi:/media/MEDIA/INCOMING $ run-filebot
[2020-03-27 10:29:40.814] Locking /tmp/filebot.log
[2020-03-27 10:29:40.847] Using persistent disk cache: /home/pi/filebot/data/pi/cache/0
[2020-03-27 10:29:42.319] Fetch resource: https://api.filebot.net/7255/script/fn.jar.xz
[2020-03-27 10:29:44.373] [GET https://api.filebot.net/7255/script/fn.jar.xz] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a9ce278e316caa-SJC
Server: cloudflare
Connection: keep-alive
Last-Modified: Mon, 16 Mar 2020 05:05:18 GMT
Date: Fri, 27 Mar 2020 14:29:44 GMT
Accept-Ranges: bytes
CF-Cache-Status: HIT
Cache-Control: max-age=2678400
ETag: "6528-5a0f1c3ead6ab"
Vary: User-Agent, Accept-Encoding
Set-Cookie:
Expires: Sun, 29 Mar 2020 14:20:26 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 25896
Age: 558
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/x-xz
[2020-03-27 10:29:44.454] Received 25 KB
[1f2dab4881da2dfab9be27d2c3a5d735]

[2020-03-27 10:30:03.895] Run script [fn:amc] at [Fri Mar 27 10:30:03 EDT 2020]
[2020-03-27 10:30:04.040] Parameter: artwork = y
[2020-03-27 10:30:04.042] Parameter: ut_label = Movie
[2020-03-27 10:30:04.043] Parameter: seriesFormat = /media/MEDIA/TV/{ny}/Season {s}\{n} - S{s.pad(2)}E{e.pad(2)} - {t}
[2020-03-27 10:30:04.044] Parameter: movieFormat = /media/MEDIA/DVD/{n} ({y})/{n} ({y})
[2020-03-27 10:30:04.073] Argument[0]: /media/MEDIA/INCOMING
[2020-03-27 10:30:04.351] Activate License [P8129***] on [Fri Mar 27 10:30:04 EDT 2020]
[2020-03-27 10:30:05.279] [POST https://license.filebot.net/verify/P8129***] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a9cea8891d6d16-SJC
Server: cloudflare
Connection: keep-alive
Date: Fri, 27 Mar 2020 14:30:05 GMT
CF-Cache-Status: DYNAMIC
Cache-Control: max-age=172800
Cf-Railgun: direct (waiting for pending WAN connection)
Vary: User-Agent
Expires: Sun, 29 Mar 2020 14:30:05 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 33
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/json
[2020-03-27 10:30:05.283] Received 33 bytes
{"status": 200, "message": "OK"}


[2020-03-27 10:30:18.023] Execute [ffprobe, -show_streams, -show_format, -print_format, json, -v, error, /media/MEDIA/INCOMING/<MOVIE4>.2020.HDRip.XviD.AC3-EVO[TGx]/<MOVIE4>.2020.HDRip.XviD.AC3-EVO.avi]
{
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "High",
            "codec_type": "video",
            "codec_time_base": "1001/48000",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
            "width": 720,
            "height": 300,
            "coded_width": 720,
            "coded_height": 304,
            "has_b_frames": 2,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "12:5",
            "pix_fmt": "yuv420p",
            "level": 41,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "field_order": "progressive",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "r_frame_rate": "24000/1001",
            "avg_frame_rate": "24000/1001",
            "time_base": "1/1000",
            "start_pts": 83,
            "start_time": "0.083000",
            "bits_per_raw_sample": "8",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            }
        },
        {
            "index": 1,
            "codec_name": "ac3",
            "codec_long_name": "ATSC A/52A (AC-3)",
            "codec_type": "audio",
            "codec_time_base": "1/48000",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
            "sample_fmt": "fltp",
            "sample_rate": "48000",
            "channels": 6,
            "channel_layout": "5.1(side)",
            "bits_per_sample": 0,
            "dmix_mode": "-1",
            "ltrt_cmixlev": "-1.000000",
            "ltrt_surmixlev": "-1.000000",
            "loro_cmixlev": "-1.000000",
            "loro_surmixlev": "-1.000000",
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/1000",
            "start_pts": 78,
            "start_time": "0.078000",
            "bit_rate": "384000",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {
                "language": "eng",
                "title": "Surround"
            }
        }
    ],
    "format": {
        "filename": "/media/MEDIA/INCOMING/<MOVIE4>.2020.HDRip.XviD.AC3-EVO[TGx]/<MOVIE4>.2020.HDRip.XviD.AC3-EVO.avi",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "matroska,webm",
        "format_long_name": "Matroska / WebM",
        "start_time": "0.078000",
        "duration": "6503.822000",
        "size": "1563968836",
        "bit_rate": "1923753",
        "probe_score": 100,
        "tags": {
            "CREATION_TIME": "2020-03-24T04:13:07Z",
            "ENCODER": "Lavf55.12.0"
        }
    }
}

[2020-03-27 10:30:18.122] Input: /media/MEDIA/INCOMING/<MOVIE4>.2020.HDRip.XviD.AC3-EVO[TGx]/<MOVIE4>.2020.HDRip.XviD.AC3-EVO.avi
[2020-03-27 10:30:18.242] Group: {Movie=true} => [<MOVIE4>.2020.HDRip.XviD.AC3-EVO.avi]
[2020-03-27 10:30:18.388] Rename movies using [TheMovieDB]
[2020-03-27 10:30:18.414] Auto-detect movie from context [/media/MEDIA/INCOMING/<MOVIE4>.2020.HDRip.XviD.AC3-EVO[TGx]/<MOVIE4>.2020.HDRip.XviD.AC3-EVO.avi]
[2020-03-27 10:30:18.520] Fetch resource: https://api.filebot.net/7255/data/query-blacklist.txt.xz
[2020-03-27 10:30:18.907] [GET https://api.filebot.net/7255/data/query-blacklist.txt.xz] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a9cf001deded1f-SJC
Server: cloudflare
Connection: keep-alive
Last-Modified: Thu, 26 Mar 2020 05:44:19 GMT
Date: Fri, 27 Mar 2020 14:30:18 GMT
Accept-Ranges: bytes
CF-Cache-Status: HIT
Cache-Control: max-age=2678400
ETag: "d7c-5a1bb79e89027"
Vary: User-Agent, Accept-Encoding
Expires: Sun, 29 Mar 2020 14:19:15 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 3452
Age: 663
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/x-xz
[2020-03-27 10:30:18.910] Received 3 KB
[0d1aa2ce726a246aba9a246d1c895d3c]

[2020-03-27 10:30:19.027] Fetch resource: https://api.filebot.net/7255/data/release-groups.txt.xz
[2020-03-27 10:30:19.130] [GET https://api.filebot.net/7255/data/release-groups.txt.xz] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a9cf018873ed1f-SJC
Server: cloudflare
Connection: keep-alive
Last-Modified: Thu, 26 Mar 2020 05:44:19 GMT
Date: Fri, 27 Mar 2020 14:30:19 GMT
Accept-Ranges: bytes
CF-Cache-Status: HIT
Cache-Control: max-age=2678400
ETag: "40f0-5a1bb79e84207"
Vary: User-Agent, Accept-Encoding
Expires: Sun, 29 Mar 2020 14:19:16 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 16624
Age: 663
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/x-xz
[2020-03-27 10:30:19.141] Received 16 KB
[e59bdab83479ab02c91f666f122b7b04]

[2020-03-27 10:30:20.194] Fetch resource: https://api.filebot.net/7255/data/media-sources.txt.xz
[2020-03-27 10:30:20.294] [GET https://api.filebot.net/7255/data/media-sources.txt.xz] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a9cf08ddffed1f-SJC
Server: cloudflare
Connection: keep-alive
Last-Modified: Sat, 16 Nov 2019 19:47:02 GMT
Date: Fri, 27 Mar 2020 14:30:20 GMT
Accept-Ranges: bytes
CF-Cache-Status: HIT
Cache-Control: max-age=2678400
ETag: "1e0-5977bfa7ee2d7"
Vary: User-Agent, Accept-Encoding
Expires: Sun, 29 Mar 2020 14:19:17 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 480
Age: 663
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/x-xz
[2020-03-27 10:30:20.296] Received 480 bytes
[a92b9c1d112ff5b2d6057a5cae5e31f7]

[2020-03-27 10:30:20.452] Fetch resource: https://api.filebot.net/7255/data/moviedb.txt.xz
[2020-03-27 10:30:20.561] [GET https://api.filebot.net/7255/data/moviedb.txt.xz] => [200 OK]
HTTP/1.1 200 OK
CF-RAY: 57a9cf0a78bbed1f-SJC
Server: cloudflare
Connection: keep-alive
Last-Modified: Thu, 26 Mar 2020 05:44:19 GMT
Date: Fri, 27 Mar 2020 14:30:20 GMT
Accept-Ranges: bytes
CF-Cache-Status: HIT
Cache-Control: max-age=2678400
ETag: "1d9edc-5a1bb79e86147"
Vary: User-Agent, Accept-Encoding
Expires: Sun, 29 Mar 2020 13:10:00 GMT
alt-svc: h3-27=":443"; ma=86400, h3-25=":443"; ma=86400, h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Content-Length: 1941212
Age: 663
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Content-Type: application/x-xz
[2020-03-27 10:30:21.334] Received 1.9 MB
[8d9b2f1513d5ca1ce43d45355bdb3702]

Killed
I can DM you the movie name but it's happened to a few different movies now so I have a feeling it's not the movie but more like the environment or file size..

Let me know if there's anything else I can try!
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

Since the OS is killing the process, it's highly unlikely that the name of the file matters. The content could matter if libmediainfo somehow triggers the OS into killing the process by using to much memory, same as the java process.

IDK. I'm out of ideas. If you Google around a little bit maybe you'll think of something. All the threads I've read indicate it's running out of memory. Just FileBot shouldn't be an issue, but if there's lots of stuff eating lots of memory, then it'll run out for sure.
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

that's ok - I'll keep playing with it and if I figure something out I'll post -- in case there's someone else noticing this too.

thanks for the quick help!
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

Here's something interesting.. I have filebot installed on my Synology NAS as well and it's working fine for the same files that have failed on my Raspberry PI.

The Synology is a lot slower and has less memory so I'd much rather run filebot on the Raspberry PI. I did notice that the filebot command on the NAS included

Code: Select all

    export JAVA_OPTS="-Xmx256m"
I tried using the same setting on the Raspberry PI but it failed with an OutOfMemoryError. Any idea why this setting would work on 1 computer and not another? The files themselves are stored on the NAS so that's 1 big difference between the 2. Does filebot try to analyze the contents of a file or just look at the filename? Just wondering if working with files stored on another network could require more memory

Synology NAS:

Code: Select all

/volume1/MEDIA/INCOMING$ filebot -script 'fn:sysinfo'
FileBot 4.9.0 (r7234)
JNA Native: 6.1.0
MediaInfo: 19.04
p7zip: p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)
unrar: UNRAR 5.21 freeware
FFprobe: java.io.IOException: Cannot run program "/volume1/@appstore/MediaServer/bin/ffprobe": error=2, No such file or directory
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-03-16 (r625)
Groovy: 3.0.2
JRE: OpenJDK Runtime Environment 14
JVM: 32-bit OpenJDK Client VM
CPU/MEM: 2 Core / 259 MB Max Memory / 22 MB Used Memory
OS: Linux (arm)
HW: Linux SHARE 3.2.40 #24922 SMP Mon Aug 19 12:08:20 CST 2019 armv7l GNU/Linux synology_armada375_ds215j
STORAGE: ext4 [/] @ 1.4 GB | ext4 [/volume1] @ 2.2 TB | ext4 [/volume2] @ 217 GB
DATA: /volume1/@appstore/filebot/data/jp4500
Package: SPK
License: FileBot License P8129*** (Valid-Until: 2020-06-07)
Done ヾ(@⌒ー⌒@)ノ
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: filebot crashing(?) on Raspberry Pi 4

Post by rednoah »

Please try the latest FileBot 4.9.1 pre-release, which has some memory optimizations and should work with -Xmx256m because that's what I've configured for my development environment for the last few days:
viewtopic.php?t=1609


:idea: 32-bit (i.e. armv7l) vs 64-bit (i.e. aarch64) will make a significant difference, as each pointer is twice the size, and might put things over the edge if it's already borderline enough not enough. Client VM vs Server VM can make a difference as well. The latter makes the code run faster, but likely requires more memory as well.
:idea: Please read the FAQ and How to Request Help.
jp4500
Posts: 33
Joined: 23 Aug 2015, 01:55

Re: filebot crashing(?) on Raspberry Pi 4

Post by jp4500 »

Great news -- that worked!

filebot was hanging for a specific movie over and over - I'd have to reboot the pi just to get it to respond after each test.

I made 2 changes: using filebot 4.9.1 and added the "-no-index" option which was recommended when running the sysinfo command (also using export JAVA_OPTS="-Xmx256m" as well).

I tested the movie that was hanging as well as several others and all worked.

thanks for all of the help!
Post Reply