FileBot

Posted: **22 Mar 2021, 11:07**

Hi,

If I submit too many files at once(I am using Filebot Node on a synology NAS) the opensubtitles API returns http response 429. In the response it says too many responses please wait 2 seconds. Can we make it so this exception is caught and retried after the period of time specified in the response? If I have more requests later on they start working as the 2 seconds have passed.

Thanks,

James

Full stack trace below:

HTTP response code: 429 for URL: https://api.opensubtitles.org/xml-rpc

Code: Select all

Get [English] subtitles for 121 files
Looking up subtitles by hash via OpenSubtitles
Server returned HTTP response code: 429 for URL: https://api.opensubtitles.org/xml-rpc
redstone.xmlrpc.XmlRpcException: The response could not be parsed.
	at redstone.xmlrpc.XmlRpcClient.handleResponse(Unknown Source)
	at redstone.xmlrpc.XmlRpcClient.endCall(Unknown Source)
	at redstone.xmlrpc.XmlRpcClient.invoke(Unknown Source)
	at net.filebot.web.OpenSubtitlesXmlRpc.invoke(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient$OpenSubtitlesXmlRpcWithRetryAndFloodLimit.lambda$invoke$0(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient$OpenSubtitlesXmlRpcWithRetryAndFloodLimit.retry(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient$OpenSubtitlesXmlRpcWithRetryAndFloodLimit.invoke(Unknown Source)
	at net.filebot.web.OpenSubtitlesXmlRpc.searchSubtitles(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.lambda$getSubtitleList$2(Unknown Source)
	at net.filebot.Cache.computeIf(Unknown Source)
	at net.filebot.Cache$TypedCache.computeIf(Unknown Source)
	at net.filebot.Cache.computeIfAbsent(Unknown Source)
	at net.filebot.Cache$TypedCache.computeIfAbsent(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleList(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleList(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleListByHash(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleList(Unknown Source)
	at net.filebot.subtitle.SubtitleUtilities.lookupSubtitlesByHash(Unknown Source)
	at net.filebot.cli.CmdlineOperations.getSubtitles(Unknown Source)
	at net.filebot.cli.CmdlineOperations.getMissingSubtitles(Unknown Source)
	at net.filebot.cli.ScriptShellBaseClass.getMissingSubtitles(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at Script1$_run_closure65$_closure106.doCall(Script1.groovy:366)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at Script1$_run_closure65.doCall(Script1.groovy:365)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at Script1.run(Script1.groovy:362)
	at net.filebot.cli.ScriptShell.evaluate(Unknown Source)
	at net.filebot.cli.ScriptShell.runScript(Unknown Source)
	at net.filebot.cli.ArgumentProcessor.runScript(Unknown Source)
	at net.filebot.cli.ArgumentProcessor.run(Unknown Source)
	at net.filebot.Main.main(Unknown Source)
Caused by: java.io.IOException: Server returned HTTP response code: 429 for URL: https://api.opensubtitles.org/xml-rpc
	... 38 more

Fetch failed: Try again in 2 seconds (2 more) => redstone.xmlrpc.XmlRpcFault: Service Unavailable

Posted: **22 Mar 2021, 11:17**

I'm not aware of this limitation yet. If I knew the limits then I could add a request limiter on the client-side.

EDIT:

I found the limits, but since they've been enforced since 2017 maybe normal operation is already below that limit:
https://forum.opensubtitles.org/viewtopic.php?t=16072

If you're using a shared outgoing IP (e.g. VPN) then the request limit may be exhausted by other users sharing the same IP.

EDIT 2:

FileBot r8370 adds additional rate limiting and increased retry wait time.

Posted: **27 Mar 2021, 11:41**

I'm using filebot on my synology NAS there doesn't seem to be this functionality. I have version 4.9.3

I have been using a vpn so maybe that is the problem. Moving to a dedicated IP vpn next week so will see if it improves

FileBot

OpenSubtitles HTTP Response 429

OpenSubtitles HTTP Response 429

Re: OpenSubtitles HTTP Response 429

Re: OpenSubtitles HTTP Response 429