Filebot Freezing during Data Match

[randy]

I am running Filebot on Manjaro Linux, installed from the Arch AUR repository.
Filebot version is 4.7.2.
The Java environment is jre8-openjdk version 8.u112, with java-openjfx version 8.u76.
All other java applications on my system run without issue.

Installation went smoothly, the application launches fine. I click on Rename tab. I am able to add files into the Original Files listbox, and I then set the episode name format. I click on Match button, click on TheTVDB, a dialog box appears for TV show names that require selection, during this process of selecting the different the show's name, this dialog box freezes, completely hanging the application.

I presume this is a modal dialog box, so I have no access to the main application window beneath. The only option is to send a KILL signal to the process. I do not like having to do that. Closing the application does nothing, other signals sent to the process has no effect, and via my process monitor it looks like the java runtime environment has hung.

I have experimented with very small file sets (ie 1-5) and gradually increased in size. I would say 95% of the time the application hangs after prompting me to select an ambiguous TV show name. It has only succeeded on very small file sets (ie < 5), and very rarely then.

If the data matching succeeds and the new file names are generated in the New Names list box then renaming has worked 100% of the time. Data fetching is the issue here, not the physical renaming of the file once the new name has been generated.

I don't know much about troubleshooting java apps and the java runtime environment, so what other info is required to help in troubleshooting this issue? What would cause an application to hang retrieving data from the internet?

Thanks in advance.

[randy]

Within ~/.java/.userPrefs/net/filebot/ui/episodelist I have a directory with the name

'_!'\''w!}@"u!'\''c!d@"h!'\''c!~@!u!'\''}!}@"2!'\''8!cg"p!(@!~@"z'

I have a similarly weird directory name in ~/.java/.userPrefs/net/filebot/ui/rename

Is this right? Could this be causing issues?

Does Filebot produce application logs to help diagnose this? If so where?

EDIT:
Within ~/.java/.userPrefs there are two files.

user.lock.[username]
userRootModFile.[username]

Are these semaphore like lock files? Could this be the cause of some sort of deadlock, or some sort of race condition?

Should I remove them after each application freeze?

[randy]

I found the error log in ~/.config/filebot

I ran the application from the console to view any error message generated as filebot was running.

I used a large directory to generate test data, and it generated a lot of errors.

Code: Select all

$ filebot
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on
Nov 24, 2016 9:16:48 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file ~/.config/filebot/cache/0/data_0.data is out of date, probably due to an unclean shutdown. Deleting index file ~/.config/filebot/cache/0/data_0.index
Nov 24, 2016 9:16:49 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file ~/.config/filebot/cache/0/thetvdb_1.data is out of date, probably due to an unclean shutdown. Deleting index file ~/.config/filebot/cache/0/thetvdb_1.index
Nov 24, 2016 9:16:51 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file ~/.config/filebot/cache/0/thetvdb_search_en_3.data is out of date, probably due to an unclean shutdown. Deleting index file ~/.config/filebot/cache/0/thetvdb_search_en_3.index
Fetch failed: java.net.SocketTimeoutException: connect timed out [http://thetvdb.com/api/GetSeries.php?seriesname=Incorporated&language=en]
Fetch failed: java.net.SocketTimeoutException: connect timed out [http://thetvdb.com/api/GetSeries.php?seriesname=No+Tomorrow&language=en]

... Many more of these failed java.net.SocketTimeoutException lookups ...
... This didn't hang to app though, at least not yet ...
... Then the following once I started selecting series in the dialog box ...

Nov 24, 2016 9:17:16 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file ~/.config/filebot/cache/0/selection_thetvdb_0.data is out of date, probably due to an unclean shutdown. Deleting index file ~/.config/filebot/cache/0/selection_thetvdb_0.index
Nov 24, 2016 9:17:20 PM net.sf.ehcache.store.disk.DiskStorageFactory <init>
WARNING: The index for data file ~/.config/filebot/cache/0/thetvdb_data_0_en_3.data is out of date, probably due to an unclean shutdown. Deleting index file ~/.config/filebot/cache/0/thetvdb_data_0_en_3.index
** Cursor after hang ***

For each file the series lookup failed with

Fetch failed: java.net.SocketTimeoutException: connect timed out [http://thetvdb.com/api/GetSeries.php?se ... anguage=en]

It generated a lot of these, almost one per file attempting to be renamed.

Filebot then started prompting me to select the show series name manually via the dialog box, of which I was able to complete a couple. The CPU then spiked very high, laptop fan went crazy, then the freeze.

After the freeze, no CPU activity, no error message to say there was a crash, nothing.

The error.log file also contained an error message for each failed lookup.

Code: Select all

$ cat error.log
Nov 24, 2016 9:17:02 PM net.filebot.CachedResource lambda$get$1
WARNING: Fetch failed: java.net.SocketTimeoutException: connect timed out [http://thetvdb.com/api/GetSeries.php?seriesname=Incorporated&language=en]
Nov 24, 2016 9:17:02 PM net.filebot.CachedResource lambda$get$1
WARNING: Fetch failed: java.net.SocketTimeoutException: connect timed out [http://thetvdb.com/api/GetSeries.php?seriesname=No+Tomorrow&language=en]

I only included a couple of the messages, there where many more identical to this, except the seriesname was obviously different. There were no other error message types in this file.

How long is the timeout value for these lookups? They failed very quickly and started prompting me after a handful of seconds.

Interestingly, the very rapid CPU spike I described above generated these errors in my system logs

Code: Select all

Nov 24 21:17:03 manjaro kernel: CPU6: Package temperature above threshold, cpu clock throttled
Nov 24 21:17:03 manjaro kernel: CPU2: Package temperature above threshold, cpu clock throttled
Nov 24 21:17:03 manjaro kernel: CPU7: Package temperature above threshold, cpu clock throttled 
Nov 24 21:17:03 manjaro kernel: CPU3: Package temperature above threshold, cpu clock throttled
Nov 24 21:17:03 manjaro kernel: CPU2: Core temperature above threshold, cpu clock throttled 
Nov 24 21:17:03 manjaro kernel: CPU1: Package temperature above threshold, cpu clock throttled
Nov 24 21:17:03 manjaro kernel: CPU5: Package temperature above threshold, cpu clock throttled
Nov 24 21:17:03 manjaro kernel: CPU4: Package temperature above threshold, cpu clock throttled 
Nov 24 21:17:03 manjaro kernel: CPU0: Package temperature above threshold, cpu clock throttled
Nov 24 21:17:03 manjaro kernel: CPU6: Core temperature above threshold, cpu clock throttled

WTF is going on?

This is a recent MSI gaming machine with a lot of grunt, how the hell could renaming 100 files kill it?

Something is not right here.

[randy]

I tried again, this time using a different data source. This time I used TVMaze instead of TheTVDB. This time my CPU didn't spike at all and filebot didn't hang. Why would one data source cause CPU spike and application freeze and another doesn't? It is like TheTVDB caused some sort or runaway, cascading infinite loop or something.

Lookup errors were generated, but much less than with TheTVDB. Examples... same errors

Fetch failed: java.net.SocketTimeoutException: connect timed out [http://api.tvmaze.com/search/shows?q=People+of+Earth]
Fetch failed: java.net.SocketTimeoutException: connect timed out [http://api.tvmaze.com/search/shows?q=Survivor]
Fetch failed: java.net.SocketTimeoutException: connect timed out [http://api.tvmaze.com/search/shows?q=ma ... th+secrets]

Anyway, I manually selected all the non resolved series via the modal dialog box, no CPU spike, then encountered this error when the new file names were attempted to be generated. This is the text output from console, stderr I presume.

Code: Select all

SocketTimeoutException: connect timed out
java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: connect timed out
        at net.filebot.ui.rename.RenamePanel$AutoCompleteAction$1.done(RenamePanel.java:843)
Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: connect timed out
        at net.filebot.ui.rename.EpisodeListMatcher.match(EpisodeListMatcher.java:114)
        at net.filebot.ui.rename.RenamePanel$AutoCompleteAction$1.doInBackground(RenamePanel.java:828)
        at net.filebot.ui.rename.RenamePanel$AutoCompleteAction$1.doInBackground(RenamePanel.java:824)
Caused by: java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: connect timed out
        at net.filebot.ui.rename.EpisodeListMatcher.fetchEpisodeSet(EpisodeListMatcher.java:217)
        at net.filebot.ui.rename.EpisodeListMatcher.matchEpisodeSet(EpisodeListMatcher.java:152)
        at net.filebot.ui.rename.EpisodeListMatcher.lambda$null$3(EpisodeListMatcher.java:106)
Caused by: java.net.SocketTimeoutException: connect timed out
        at net.filebot.web.WebRequest.fetch(WebRequest.java:143)
        at net.filebot.CachedResource.lambda$fetchIfModified$8(CachedResource.java:186)
        at net.filebot.CachedResource.lambda$null$0(CachedResource.java:83)
        at net.filebot.CachedResource.retry(CachedResource.java:112)
        at net.filebot.CachedResource.lambda$get$1(CachedResource.java:83)
        at net.filebot.Cache.computeIf(Cache.java:85)
        at net.filebot.CachedResource.get(CachedResource.java:78)
        at net.filebot.web.TVMazeClient.request(TVMazeClient.java:105)
        at net.filebot.web.TVMazeClient.fetchSearchResult(TVMazeClient.java:50)
        at net.filebot.web.AbstractEpisodeListProvider.lambda$search$0(AbstractEpisodeListProvider.java:22)
        at net.filebot.Cache.computeIf(Cache.java:85)
        at net.filebot.Cache$TypedCache.computeIf(Cache.java:166)
        at net.filebot.Cache.computeIfAbsent(Cache.java:91)
        at net.filebot.Cache$TypedCache.computeIfAbsent(Cache.java:171)
        at net.filebot.web.AbstractEpisodeListProvider.search(AbstractEpisodeListProvider.java:21)
        at net.filebot.ui.rename.EpisodeListMatcher.lambda$null$6(EpisodeListMatcher.java:202)
Caused by: java.net.SocketTimeoutException: connect timed out
        at net.filebot.web.WebRequest.fetch(WebRequest.java:140)
        ... 15 more

Is there a limit to the number of lookup requests?

Just a thought, but I am using a VPN, are VPN connections being refused by these web data sources?

[randy]

I checked connectivity with a url that supposedly had previously generated a

java.net.SocketTimeoutException: connect timed out

error.

Code: Select all

$ curl http://api.tvmaze.com/search/shows?q=The+Big+Bang+Theory
[{"score":29.423094,"show":{"id":66,"url":"http://www.tvmaze.com/shows/66/the-big-bang-theory","name":"The Big Bang Theory","type":"Scripted","language":"English","genres":["Comedy"],"status":"Running","runtime":30,"premiered":"2007-09-24","schedule":{"time":"20:00","days":["Thursday"]},"rating":{"average":8.4},"weight":19,"network":{"id":2,"name":"CBS","country":{"name":"United States","code":"US","timezone":"America/New_York"}},"webChannel":null,"externals":{"tvrage":8511,"thetvdb":80379,"imdb":"tt0898266"},"image":{"medium":"http://tvmazecdn.com/uploads/images/medium_portrait/58/145601.jpg","original":"http://tvmazecdn.com/uploads/images/original_untouched/58/145601.jpg"},"summary":"<p><strong>The Big Bang Theory</strong> is a comedy about brilliant physicists, Leonard and Sheldon, who are the kind of \"beautiful minds\" that understand how the universe works. But none of that genius helps them interact with people, especially women. All this begins to change when a free-spirited beauty named Penny moves in next door. Sheldon, Leonard's roommate, is quite content spending his nights playing Klingon Boggle with their socially dysfunctional friends, fellow Cal Tech scientists Wolowitz and Koothrappali. However, Leonard sees in Penny a whole new universe of possibilities... including love.</p>","updated":1479980890,"_links":{"self":{"href":"http://api.tvmaze.com/shows/66"},"previousepisode":{"href":"http://api.tvmaze.com/episodes/979245"},"nextepisode":{"href":"http://api.tvmaze.com/episodes/991390"}}}}]

It seems to work fine.

I also check connectivity with a TheTVDB url that failed previously.

Code: Select all

$ curl http://thetvdb.com/api/GetSeries.php?seriesname=Queen+Victorias+Children&language=en
[1] 20447
[glenn@manjaro-msi filebot]$ <?xml version="1.0" encoding="UTF-8" ?>
<Data>
<Series>
<seriesid>265378</seriesid>
<language>en</language>
<SeriesName>Queen Victoria's Children</SeriesName>
<banner>graphical/265378-g3.jpg</banner>
<Overview>Queen Victoria’s Children is a documentary series which spans over sixty years as it explores the reign of Victoria through her personal relationships with her husband and her nine children. It features manipulation, conflict, intimidation, emotional blackmail and several attempts by her children to escape from her clutches.</Overview>
<FirstAired>2013-01-01</FirstAired>
<Network>BBC Two</Network>
<id>265378</id>
</Series>
</Data>
[1]+  Done  curl http://thetvdb.com/api/GetSeries.php?seriesname=Queen+Victorias+Children

This works fine too.

No connectivity issues, so what is causing the timeouts?

[rednoah]

TheTVDB was down at the time:

Code: Select all

java.net.SocketTimeoutException: connect timed out

There is nothing you can do about that. It's most likely a issue on their side. Either the server being down, or otherwise unavailable, for example when there's too many connections, some might get dropped and go unanswered.

Try again a few hours later, and it'll probably work just fine again. Let me know if it doesn't resolve itself by tomorrow. If you get the same sort of issues with multiple data sources, then the issue is likely on your side of the internet (localhost, provider, VPN, etc).


PS: There's some developer options for debug logging. It'll create huge amounts of logs though, and might not be useful if it's just waiting for a HTTP response.