GC overhead limit exceeded

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
doscrash
Posts: 8
Joined: 22 Feb 2015, 08:54

GC overhead limit exceeded

Post by doscrash »

Hi, I'm trying to run Filebot for Synology on a folder with thousands of new media files. However, it doesn't work and gives me this error:

Code: Select all

GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at net.filebot.media.MediaDetection.matchMovieName(MediaDetection.java:887)
        at net.filebot.media.MediaDetection.isMovie(MediaDetection.java:137)
        at net.filebot.media.AutoDetection.isMovie(AutoDetection.java:91)
        at net.filebot.media.AutoDetection.detectGroup(AutoDetection.java:191)
        at net.filebot.media.AutoDetection.group(AutoDetection.java:162)
        at net.filebot.media.AutoDetection$group.call(Unknown Source)
        at Script1.group(Script1.groovy:331)
        at Script1.run(Script1.groovy:337)
        at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:64)
        at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:74)
        at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:163)
        at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:37)
        at net.filebot.Main.main(Main.java:132)

Error (o_O)
I'm running

Code: Select all

FileBot 4.8.5 (r6224) / Java(TM) SE Runtime Environment 1.8.0_121 (headless)
Does anyone know what the issue is and how I can fix it?
User avatar
rednoah
The Source
Posts: 22987
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: GC overhead limit exceeded

Post by rednoah »

What does filebot -script fn:sysinfo say?


:?: Have you tried upgrading to Java 13 yet? Assuming you have plenty of memory to spare, then it's a runtime issue, so a new runtime might work better.
:idea: Please read the FAQ and How to Request Help.
doscrash
Posts: 8
Joined: 22 Feb 2015, 08:54

Re: GC overhead limit exceeded

Post by doscrash »

Thanks for replying so quickly!

I definitely have more than enough memory -- I upgraded my Synology from 4GB to 16GB!

It says this:

Code: Select all

FileBot 4.8.5 (r6224)
JNA Native: 5.2.2
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Unable to load library 'mediainfo': Native library (linux-x86-64/libmediainfo.so) not found in resource path ([file:/volume1/@appstore/filebot/jar/filebot.jar])
7-Zip-JBinding: 9.20
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2019-05-15 (r565)
Groovy: 2.5.6
JRE: Java(TM) SE Runtime Environment 1.8.0_121
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 4 Core / 3 GB Max Memory / 18 MB Used Memory
OS: Linux (amd64)
HW: Linux wayneflix 3.10.105 #24922 SMP Wed Jul 3 16:37:24 CST 2019 x86_64 GNU/Linux synology_avoton_1515+
DATA: /volume1/@appstore/filebot/data/[redacted]
Package: SPK
License: FileBot License P8606361 (Valid-Until: 2020-07-20)
I only see the ability to install Java8 in my install, though I do see something from you in my Community package center called "Java Installer" -- is that it?
User avatar
rednoah
The Source
Posts: 22987
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: GC overhead limit exceeded

Post by rednoah »

My Java Installer package will install the latest OpenJDK 13. If Java 8 doesn't work, Java 13 is the way to go. If you upgrade to Java 13, then you will also need to upgrade to FileBot 4.9.0 (available via latest beta) for compatibility reasons.


:idea: There's no particular reason think that upgrading to the latest JDK will make the problem go away, but it might. Definitely worth a try.
:idea: Please read the FAQ and How to Request Help.
doscrash
Posts: 8
Joined: 22 Feb 2015, 08:54

Re: GC overhead limit exceeded

Post by doscrash »

Darn, I upgraded to OpenJDK13 and Filebot 4.9.0 but still get a similar error. Do you have any other suggestions? Is there just too many files in the folder? It seems to work on other directories with less files to scan.

Code: Select all

Resource not found: https://api.thetvdb.com/search/series?imdbId=tt7280898 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4080956 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt2392830 [404 Not Found]
Failed to read media characteristics: NumberFormatException: empty String
Failed to read media characteristics: NumberFormatException: empty String
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5634960 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5952332 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6719408 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6719416 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt8269124 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4392770 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt7364220 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt9011954 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt9182528 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt8760684 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4971408 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5816374 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt1980929 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4669264 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4443658 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt0822727 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt0873451 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt0873451 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt3208026 [404 Not Found]
Ignore invalid series: null [343874]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt7579776 [404 Not Found]
Ignore invalid series: null [343874]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt7579782 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5826046 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5748610 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5173032 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt3860916 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6508716 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6415222 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5294198 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt5270948 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4819510 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6144762 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4441098 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6381862 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt0118880 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4457344 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6090710 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt6243334 [404 Not Found]
Resource not found: https://api.thetvdb.com/search/series?imdbId=tt4619986 [404 Not Found]
Ignore invalid series: null [318506]
Java heap space
java.lang.OutOfMemoryError: Java heap space
        at com.ibm.icu.impl.ICUBinary.getByteBufferFromInputStreamAndCloseStream(ICUBinary.java:729)
        at com.ibm.icu.impl.ICUBinary.getData(ICUBinary.java:500)
        at com.ibm.icu.impl.ICUBinary.getData(ICUBinary.java:438)
        at com.ibm.icu.impl.ICUResourceBundleReader$ReaderCache.createInstance(ICUResourceBundleReader.java:189)
        at com.ibm.icu.impl.ICUResourceBundleReader$ReaderCache.createInstance(ICUResourceBundleReader.java:178)
        at com.ibm.icu.impl.SoftCache.getInstance(SoftCache.java:65)
        at com.ibm.icu.impl.ICUResourceBundleReader.getReader(ICUResourceBundleReader.java:233)
        at com.ibm.icu.impl.ICUResourceBundle.createBundle(ICUResourceBundle.java:1308)
        at com.ibm.icu.impl.ICUResourceBundle$4.load(ICUResourceBundle.java:1181)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:92)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:89)
        at com.ibm.icu.impl.SoftCache.getInstance(SoftCache.java:65)
        at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUResourceBundle.java:1168)
        at com.ibm.icu.impl.ICUResourceBundle.access$600(ICUResourceBundle.java:35)
        at com.ibm.icu.impl.ICUResourceBundle$4.load(ICUResourceBundle.java:1229)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:92)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:89)
        at com.ibm.icu.impl.SoftCache.getInstance(SoftCache.java:65)
        at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUResourceBundle.java:1168)
        at com.ibm.icu.impl.ICUResourceBundle.access$600(ICUResourceBundle.java:35)
        at com.ibm.icu.impl.ICUResourceBundle$4.load(ICUResourceBundle.java:1205)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:92)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:89)
        at com.ibm.icu.impl.SoftCache.getInstance(SoftCache.java:65)
        at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUResourceBundle.java:1168)
        at com.ibm.icu.impl.ICUResourceBundle.access$600(ICUResourceBundle.java:35)
        at com.ibm.icu.impl.ICUResourceBundle$4.load(ICUResourceBundle.java:1211)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:92)
        at com.ibm.icu.impl.ICUResourceBundle$1.createInstance(ICUResourceBundle.java:89)

Error (o_O)
User avatar
rednoah
The Source
Posts: 22987
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: GC overhead limit exceeded

Post by rednoah »

I suppose. I just don't see how 3 GB of memory isn't enough for any amount of files.


:idea: If you could run find so I can get a snapshot of all your files, then I could have a look. Probably a large text file. Zip it up and PM me a Google Drive link.


:idea: If it's less than 10k files, then it really shouldn't be an issue. Perhaps there's one specific unusual file or circumstance that somehow screws things up. Try processing files in smaller batches. And see if you find the one specific batch that still errors out with OutOfMemoryError, and then narrow down the set of files until you find the cause.


:idea: My educated guess at this point, some unusual file, like an *.srt or an *.nfo file, but like 2-3 GB large, which FileBot might handle like any other 2-3 KB text file, and completely screw up memory management... or a strange and rare Java bug specific to your CPU architecture optimizations.
:idea: Please read the FAQ and How to Request Help.
Post Reply