[req] Retry on internet failure

[iampaulh]

Sometimes when processing a file via the AMC script, TMDB or TVDB fail to respond for whatever reason. It could be saturated network at my end, maybe the wifi dropped, broadband down, etc.

When this happens, naturally processing is stopped and finished files are not moved/renamed. This means having to go in and run the script again manually.

I was hoping if you could add a retry option for when the network connection fails. For example, retry 5 times, with an incremental delay. The first retry is 5 minutes later, the next is 15 minutes after the last retry, the next is 30 minutes after the last retry and so on.

[rednoah]

If you're gonna modify the script you can probably use my retry(...) helper and then just wrap the rename(...) parts that with retry{ ... } blocks. If it works well for you can merge it into my master amc later.

e.g.

Code: Select all

retry(3, 5000) {
	if (new Random().nextInt(5) != 0) throw new Exception("random != 0")
}

[iampaulh]

I'm not too sure how to do that! I'd happily sponsor though!

This is the sort of error I get when the network connection is down. It would be great if it could try again later!

Code: Select all

Jul 18, 2013 8:29:24 AM net.sourceforge.filebot.web.CachedResource get
WARNING: java.net.SocketTimeoutException: connect timed out
Parameter: xbmc = localhost
Parameter: ut_dir = D:\torrents\Big_Buck_Bunny_1080p_surround_frostclick
Parameter: ut_file = Big_Buck_Bunny_1080p_surround_frostclick.mkv
Parameter: ut_title =  Big_Buck_Bunny_1080p_surround_frostclick.mkv
Parameter: ut_label = 
Parameter: ut_state = 5
Input: D:\torrents\Big_Buck_Bunny_1080p_surround_frostclick\ Big_Buck_Bunny_1080p_surround_frostclick.mkv
Jul 18, 2013 8:29:37 AM net.sourceforge.filebot.web.CachedResource get
WARNING: java.net.SocketTimeoutException: connect timed out
Jul 18, 2013 8:29:52 AM net.sourceforge.filebot.media.MediaDetection detectSeriesNames
WARNING: Failed to lookup info by id: connect timed out
java.net.SocketTimeoutException: connect timed out
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at net.sourceforge.filebot.web.WebRequest.getReader(Unknown Source)
	at net.sourceforge.filebot.web.WebRequest.getDocument(Unknown Source)
	at net.sourceforge.filebot.web.WebRequest.getDocument(Unknown Source)
	at net.sourceforge.filebot.web.TheTVDBClient.lookupByIMDbID(Unknown Source)
	at net.sourceforge.filebot.media.MediaDetection.lookupSeriesNameByInfoFile(Unknown Source)
	at net.sourceforge.filebot.media.MediaDetection.detectSeriesNames(Unknown Source)
	at net.sourceforge.filebot.media.MediaDetection$detectSeriesNames.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
	at Script2.detectSeriesName(Script2.groovy:232)
	at Script2.detectSeriesName(Script2.groovy)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1085)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:952)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:909)
	at groovy.lang.Closure.call(Closure.java:411)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.callGlobal(GroovyScriptEngineImpl.java:411)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.access$000(GroovyScriptEngineImpl.java:90)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:332)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:321)
	at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:44)
	at groovy.lang.Script.invokeMethod(Script.java:78)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeOnDelegationObjects(ClosureMetaClass.java:407)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:346)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:909)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
	at Script3$_run_closure31.doCall(Script3.groovy:135)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:909)
	at groovy.lang.Closure.call(Closure.java:411)
	at groovy.lang.Closure.call(Closure.java:427)
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.groupBy(DefaultGroovyMethods.java:3534)
	at org.codehaus.groovy.runtime.dgm$267.invoke(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:271)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at Script3.run(Script3.groovy:121)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:346)
	at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:146)
	at javax.script.AbstractScriptEngine.eval(Unknown Source)
	at net.sourceforge.filebot.cli.ScriptShell.evaluate(Unknown Source)
	at net.sourceforge.filebot.cli.ScriptShell.runScript(Unknown Source)
	at net.sourceforge.filebot.cli.ArgumentProcessor.process(Unknown Source)
	at net.sourceforge.filebot.Main.main(Unknown Source)
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
	at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
	at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
	at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
	at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
	at java.net.PlainSocketImpl.connect(Unknown Source)
	at java.net.SocksSocketImpl.connect(Unknown Source)
	at java.net.Socket.connect(Unknown Source)
	at sun.net.NetworkClient.doConnect(Unknown Source)
	at sun.net.www.http.HttpClient.openServer(Unknown Source)
	at sun.net.www.http.HttpClient.openServer(Unknown Source)
	at sun.net.www.http.HttpClient.<init>(Unknown Source)
	at sun.net.www.http.HttpClient.New(Unknown Source)
	at sun.net.www.http.HttpClient.New(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source)
	at java.net.URLConnection.getContentType(Unknown Source)
	... 61 more

Jul 18, 2013 8:30:02 AM net.sourceforge.filebot.web.CachedResource get
WARNING: java.net.SocketTimeoutException: connect timed out
Jul 18, 2013 8:30:18 AM net.sourceforge.filebot.web.CachedResource get
WARNING: java.net.SocketTimeoutException: connect timed out
Jul 18, 2013 8:30:30 AM net.sourceforge.filebot.web.CachedResource get
WARNING: java.net.SocketTimeoutException: connect timed out
Jul 18, 2013 8:30:41 AM net.sourceforge.filebot.media.MediaDetection detectMovie
WARNING: OpenSubtitles: A network error occurred.
SocketTimeoutException: connect timed out
java.net.SocketTimeoutException: connect timed out
	at net.sourceforge.filebot.web.WebRequest.fetch(Unknown Source)
	at net.sourceforge.filebot.web.WebRequest.fetchIfModified(Unknown Source)
	at net.sourceforge.filebot.web.CachedResource.fetchData(Unknown Source)
	at net.sourceforge.filebot.web.TMDbClient$1.fetchData(Unknown Source)
	at net.sourceforge.filebot.web.CachedResource.fetch(Unknown Source)
	at net.sourceforge.filebot.web.CachedResource.get(Unknown Source)
	at net.sourceforge.filebot.web.TMDbClient.request(Unknown Source)
	at net.sourceforge.filebot.web.TMDbClient.getMovieInfo(Unknown Source)
	at net.sourceforge.filebot.web.TMDbClient.getMovieDescriptor(Unknown Source)
	at net.sourceforge.filebot.media.MediaDetection.detectMovie(Unknown Source)
	at net.sourceforge.filebot.media.MediaDetection$detectMovie.call(Unknown Source)
	at Script2.detectMovie(Script2.groovy:237)
	at Script2.detectMovie(Script2.groovy)
	at Script3$_run_closure31.doCall(Script3.groovy:136)
	at Script3.run(Script3.groovy:121)
	at net.sourceforge.filebot.cli.ScriptShell.evaluate(Unknown Source)
	at net.sourceforge.filebot.cli.ScriptShell.runScript(Unknown Source)
	at net.sourceforge.filebot.cli.ArgumentProcessor.process(Unknown Source)
	at net.sourceforge.filebot.Main.main(Unknown Source)
Failure (°_°)

[rednoah]

Would it be enough if we had a shell script that re-runs the filebot call until we get a positive result?

In this case it's easy since we get a negative exit code from the filebot call:

Code: Select all

Failure (°_°)

Though this is when there is no internet from the beginning. If internet fails sometime in between filebots processing i wouldn't know exactly where to catch it.

PS: You on Windows or Linux/Mac?

[iampaulh]

shell script should be fine - I'm using Windows

For example, from what I can tell Filebot queries TheTVDB for a matching episode, it then queries again for the episode data. If either of these fail, the file is not processed (moved/copied)

[rednoah]

I give up on Windows cmd... it's not made for anything but one-line calls...

So here's a retry-wrapper for fn:amc as filebot script.

retry-amc.groovy

Code: Select all

retry(3, 15*60*1000) {
	executeScript('fn:amc', [:], args)
}

(3 retries in 15min intervals)

Then you just call the local script which in turn will run fn:amc:

Code: Select all

filebot -script /path/to/retry-amc.groovy <amc parameters>

You'll need r1663 for this to work. Just grab the latest jar and override the one in your install folder to upgrade.