OpenSubtitles HTTP Response 429

All your suggestions, requests and ideas for future development
Post Reply
Acejames87
Posts: 4
Joined: 21 Mar 2021, 19:00

OpenSubtitles HTTP Response 429

Post by Acejames87 »

Hi,

If I submit too many files at once(I am using Filebot Node on a synology NAS) the opensubtitles API returns http response 429. In the response it says too many responses please wait 2 seconds. Can we make it so this exception is caught and retried after the period of time specified in the response? If I have more requests later on they start working as the 2 seconds have passed.

Thanks,

James

Full stack trace below:

HTTP response code: 429 for URL: https://api.opensubtitles.org/xml-rpc

Code: Select all

Get [English] subtitles for 121 files
Looking up subtitles by hash via OpenSubtitles
Server returned HTTP response code: 429 for URL: https://api.opensubtitles.org/xml-rpc
redstone.xmlrpc.XmlRpcException: The response could not be parsed.
	at redstone.xmlrpc.XmlRpcClient.handleResponse(Unknown Source)
	at redstone.xmlrpc.XmlRpcClient.endCall(Unknown Source)
	at redstone.xmlrpc.XmlRpcClient.invoke(Unknown Source)
	at net.filebot.web.OpenSubtitlesXmlRpc.invoke(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient$OpenSubtitlesXmlRpcWithRetryAndFloodLimit.lambda$invoke$0(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient$OpenSubtitlesXmlRpcWithRetryAndFloodLimit.retry(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient$OpenSubtitlesXmlRpcWithRetryAndFloodLimit.invoke(Unknown Source)
	at net.filebot.web.OpenSubtitlesXmlRpc.searchSubtitles(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.lambda$getSubtitleList$2(Unknown Source)
	at net.filebot.Cache.computeIf(Unknown Source)
	at net.filebot.Cache$TypedCache.computeIf(Unknown Source)
	at net.filebot.Cache.computeIfAbsent(Unknown Source)
	at net.filebot.Cache$TypedCache.computeIfAbsent(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleList(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleList(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleListByHash(Unknown Source)
	at net.filebot.web.OpenSubtitlesClient.getSubtitleList(Unknown Source)
	at net.filebot.subtitle.SubtitleUtilities.lookupSubtitlesByHash(Unknown Source)
	at net.filebot.cli.CmdlineOperations.getSubtitles(Unknown Source)
	at net.filebot.cli.CmdlineOperations.getMissingSubtitles(Unknown Source)
	at net.filebot.cli.ScriptShellBaseClass.getMissingSubtitles(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at Script1$_run_closure65$_closure106.doCall(Script1.groovy:366)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at Script1$_run_closure65.doCall(Script1.groovy:365)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at Script1.run(Script1.groovy:362)
	at net.filebot.cli.ScriptShell.evaluate(Unknown Source)
	at net.filebot.cli.ScriptShell.runScript(Unknown Source)
	at net.filebot.cli.ArgumentProcessor.runScript(Unknown Source)
	at net.filebot.cli.ArgumentProcessor.run(Unknown Source)
	at net.filebot.Main.main(Unknown Source)
Caused by: java.io.IOException: Server returned HTTP response code: 429 for URL: https://api.opensubtitles.org/xml-rpc
	... 38 more

Fetch failed: Try again in 2 seconds (2 more) => redstone.xmlrpc.XmlRpcFault: Service Unavailable
User avatar
rednoah
The Source
Posts: 22898
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: OpenSubtitles HTTP Response 429

Post by rednoah »

I'm not aware of this limitation yet. If I knew the limits then I could add a request limiter on the client-side.



EDIT:

I found the limits, but since they've been enforced since 2017 maybe normal operation is already below that limit:
https://forum.opensubtitles.org/viewtopic.php?t=16072

:!: If you're using a shared outgoing IP (e.g. VPN) then the request limit may be exhausted by other users sharing the same IP.



EDIT 2:

FileBot r8370 adds additional rate limiting and increased retry wait time.
:idea: Please read the FAQ and How to Request Help.
Acejames87
Posts: 4
Joined: 21 Mar 2021, 19:00

Re: OpenSubtitles HTTP Response 429

Post by Acejames87 »

I'm using filebot on my synology NAS there doesn't seem to be this functionality. I have version 4.9.3

I have been using a vpn so maybe that is the problem. Moving to a dedicated IP vpn next week so will see if it improves
Post Reply