Filebot randomly failing to download subtitles

[migat]

When I try to download subtitles, I randomly get the following error:

https://i.imgur.com/gsqxRFU.png

If I close and re-open Filebot, the exact same subtitle will download. After a few successful downloads, that error pops up for every subtitle I try to download until I re-open the application. I am running the latest version of Java. Windows 7 x64.

[rednoah]

Looks like a general server-side issue. What does the log say?

[migat]

I don't see a filebot folder in the app data folder. Can you point me in the right direction please? It seems strange that if I close and re-open it fixes the issue, albeit temporarily.

[rednoah]

1.
What does filebot -script fn:sysinfo say?

The sysinfo output will tell you where the logs are.


2.
What does filebot -script fn:logs say?

The logs output will print the logs wherever they may be.

[migat]

I wasn't able to reproduce the issue for quite some time, and it has reared it's ugly head again. Perhaps I am reading this wrong, but it appears that the subtitle "doesn't exist", then there is an API timeout? Again, this issue can be remedied by closing and re-opening the client. Does this perhaps cause a "fresh" API call that then fixes it, albeit briefly? It doesnt fail to return a search result, it fails to download it once selected.

I kept getting a 503 when trying to paste my logs, hopefully this time it works...logs can be found here - https://pastebin.com/56gMQ0QE

[rednoah]

OpenSubtitles is sometimes down:

Code: Select all

Server returned HTTP response code: 503 for URL: https://api.opensubtitles.org/xml-rpc

As for the FileNotFoundException errors, that just means that the server returned a 404 Not Found error code at the time for the given URL, and even though it may work now, it just didn't work then, probably due to server-side issues or bugs.

[migat]

But why does closing and re-opening the client immediately resolve the issue? And why does it show the list of subtitles, but then fail to download them? I can understand if it is a server side issue, but how can it be down if I am able to search and get a list back, but then unable to download it, which then works perfectly if I restart the client for the exact same piece of media?

[rednoah]

But why does closing and re-opening the client immediately resolve the issue?

If have no technical explanation as to why that might be the case. My educated guess it that it doesn't, though trying again at a later point might make a difference.


The search results and subtitle listings may be cached for up to ~24 hours.


If you'd like to debug this, then you can just put a HTTP debug proxy (e.g. Fiddler) in the middle and observe the request/response dance. That will also tell you which actions actually require network requests, and which requests might be satisfied locally from the cache.

[migat]

I just had this fail for every episode of that 70's show, then succeed after closing and re-opening the application...it only had an issue with one episode because it couldn't find a subtitle for it. That's 209 files that correctly searched but failed to download in a row, followed by 209 files that correctly searched and downloaded after closing and reopening the app. This seems like too big of a data set to be a coincidence.

[migat]

But why does closing and re-opening the client immediately resolve the issue?

If have no technical explanation as to why that might be the case. My educated guess it that it doesn't, though trying again at a later point might make a difference.

Once the issue starts it does not go away until I relaunch the program.

[migat]

But why does closing and re-opening the client immediately resolve the issue?

If you'd like to debug this, then you can just put a HTTP debug proxy (e.g. Fiddler) in the middle and observe the request/response dance. That will also tell you which actions actually require network requests, and which requests might be satisfied locally from the cache.

Will a packet capture suffice? It will give you all the same info as a debug proxy and more in case it would be needed at a later time.

I think I might have a valid explanation of what is happening...how often does authentication against opensubtitles happen? Does it only happen when you launch the program? This could be a login timeout happening which would explain why relaunching the application fixes the issue. It could also easily be fixed with a re-auth every x minutes or every time a search is conducted. The former seems less aggressive on their infrastructure.

[rednoah]

Well, a HTTP debug proxy will make it easier for you to observe what is going on. If you can reliably reproduce the issue, then you should be able to track the difference in the HTTP request / response sequence. You can use packet capture of course, but that'll just make life unnecessarily difficult for yourself. The issue is almost certainly visible on the HTTP layer, and not the underlying TCP/IP layer, so we'll want to look at the former.


I would start debugging by finding the answers to these questions:

What's the request / response sequence for a working case?

What's the request / response sequence for a broken case?

What is the difference?

[migat]

Well, a HTTP debug proxy will make it easier for you to observe what is going on. If you can reliably reproduce the issue, then you should be able to track the difference in the HTTP request / response sequence. You can use packet capture of course, but that'll just make life unnecessarily difficult for yourself. The issue is almost certainly visible on the HTTP layer, and not the underlying TCP/IP layer, so we'll want to look at the former.


I would start debugging by finding the answers to these questions:

What's the request / response sequence for a working case?

What's the request / response sequence for a broken case?

What is the difference?

Im going to have to wait a day or so...maxed out my requests last night, heh

[migat]

I was able to capture the issue I believe in the following fiddler session. Any way you can take a look at this for me?

https://mega.nz/file/8SZU3Lpa#KQbQROWS8 ... 7HR5K2XZZU

[rednoah]

Unfortunately, I'm not currently using Windows so I can't use Fiddler. Please paste the HTTP request/response raw text on pastebin so I can have a look. One paste per request/response would be nice, using request/response number as paste title for easy sorting.

[migat]

They recently released fiddler everywhere which allows for easy sharing across all platforms via a pretty slick UI. https://www.telerik.com/download/fiddler-everywhere want to give that a shot or do you need me to extract the headers for you one by one? I run quite a bit of services on this box (sonarr/radarr/plex/filebot) so sorting through it one request and response and a time and copy pasting the headers will likely take quite a bit of time, whereas you can sort it all quickly in the UI. If I am going to do it one packet at a time I might as well go with my original idea of a packet capture which will contain every possible bit of header info in case it is needed later (I am also much more familiar with wireshark than I am fiddler, my background is more in VOIP than it is in http, but I can get around somewhat.)

[migat]

Well, a HTTP debug proxy will make it easier for you to observe what is going on. If you can reliably reproduce the issue, then you should be able to track the difference in the HTTP request / response sequence. You can use packet capture of course, but that'll just make life unnecessarily difficult for yourself. The issue is almost certainly visible on the HTTP layer, and not the underlying TCP/IP layer, so we'll want to look at the former.


I would start debugging by finding the answers to these questions:

What's the request / response sequence for a working case?

What's the request / response sequence for a broken case?

What is the difference?

Filebot Subtitle Download Failure - https://pastebin.com/BVU9F60J
Filebot Single Subtitle Success After Application Reboot - https://pastebin.com/TFk4vVD0
Filebot Multiple Subtitle Success After Application Reboot - https://pastebin.com/4t0XnpJd

[rednoah]

Well, that's not what HTTP request / response pairs look like... Looks like you forgot to configure Fiddler for HTTPS debugging?
https://docs.telerik.com/fiddler/Config ... cryptHTTPS


We'll want to look at the request headers / request content and the response header / response content:



You'll also want to configure things so that only FileBot requests go through the debugging proxy, as to filter out all other stuff right away. I'd configure Fiddler to not act as system proxy on startup, and then configure FileBot to ignore system settings and use a specific proxy instead:

Code: Select all

filebot -script fn:properties --def http.proxyHost=localhost http.proxyPort=8888