Fully automated media downloading

F12 · Post by **F12** » 27 Apr 2012, 05:48

[2012-04-27 12:59:28]  Executing: filebot -script "C:/Batches/utorrent-postprocess.groovy" --output "F:" --action copy --conflict override -non-strict -trust-script -Xxbmc=localhost "-Xut_dir=D:\Downloads\Torrents\Seeding\The.Big.Bang.Theory.S03.BDRip.XviD-SAiNTS" "-Xut_file=The.Big.Bang.Theory.S03E01.BDRip.XviD-SAiNTS.avi" "-Xut_label=" "-Xut_state=5" "-Xut_kind=multi"
[2012-04-27 13:05:05]  Executing: filebot -script "http://filebot.sourceforge.net/scripts/utorrent-postprocess.groovy" --output "F:" --action copy --conflict override -non-strict -trust-script -Xxbmc=localhost "-Xut_dir=D:\Downloads\Torrents\Seeding\The.Big.Bang.Theory.S03.BDRip.XviD-SAiNTS" "-Xut_file=The.Big.Bang.Theory.S03E01.BDRip.XviD-SAiNTS.avi" "-Xut_label=" "-Xut_state=5" "-Xut_kind=multi"

The latter worked as expected, the former did not. Note that the local file is just a straight copy-paste of your remote script. I'm not sure exactly what is happening and the command prompt closes too quickly for me to see where the script gets up to before closing.

From looking at the logs, file is always set (for multi-file torrents it would appear to be the first file of the directory). What I mean by downloading files one at a time is that, for instance, I have a season torrent but choose to download each episode as I watch it (using the uTorrent Files option). The torrent would go to 100% each time a new file is marked for download and completed, and uTorrent will feed the entire directory for that torrent to the script, including any files that were downloaded previously. As far as I can tell there wouldn't be any way to feed it only the most recently downloaded files apart from separately logging which files have already been moved, but I don't think it is really an issue to rescan the files for each new file, unless it is a really large multi-file torrent. (Large multi-file torrents are common for scene releases, now I think about it. Collections of movies and the like, where you wouldn't usually download all the movies at once but might pick and choose at different times.)

Post by **rednoah** » 27 Apr 2012, 06:13

Did you try copying the utorrent cmd that doesnt work into the cmd and see what happens? You can also try using a file-URL instead of a file path.

Well, in that case it depends and what you want. Running multiple times should be fine but you should switch --conflict override to --conflict skip so you don't copy things over and over from the original location.

F12 · Post by **F12** » 27 Apr 2012, 07:08

Darn... my browser had cached the previous version of utorrent-postprocess.groovy so that is what I had been copying into the local file, which explains a lot. It is working much better now (that is to say, it works where previously it didn't). I did have to change the regexp to include a dash as it was previously splitting one of my hostnames inappropriately.

Unless I run into any other issues, I would have to say this is a perfect solution to my initial project of automating my media centre downloads. It is a very powerful app.

Post by **rednoah** » 27 Apr 2012, 10:51

Let me know how it goes. I'll add subtitle and artwork/NFO download later as well. When it runs fine I'll release 2.62 with all of that.

Post by **rednoah** » 28 Apr 2012, 17:46

* Added fetching of TVDB/TMDB artwork/nfo to the utorrent-postprocess script.

Post by **rednoah** » 03 May 2012, 08:57

The new fn:utorrent-postprocess script should take care of everything easily now.

F12 · Post by **F12** » 05 May 2012, 05:23

It is running fantastically so far. My main concern was that Filebot might hang occasionally (for example, if multiple instances of the script are run within a few seconds of one another) but it all seems to run very smoothly. It really is an amazingly powerful tool.

Given that this is a uTorrent completion script, how would one go about adding (very basic) music handling? In my case, it would simply involve moving/copying any music files straight to the "Add to iTunes" directory, although in others' cases it might involve launching a second app to handle the renaming by tags. I would assume by making use of the File.isAudio() function and then simply copying it to the given directory, but not too sure about how to integrate it into the Groovy script.

Post by **rednoah** » 05 May 2012, 10:02

If you start a second instance while the first one is still running then the second one will not get access to the diskcache. Doesn't break anything but it'll be slower since it'll download all the data files that would usually be cached.

Post by **rednoah** » 05 May 2012, 10:24

Here's generally how you copy audio files to where you want:

Code: Select all

args.getFiles{ it.isAudio() }*.copyTo("E:/New Music")

Specifically in the utorrent-postprocess script it'd be like this (before you limit input to video/subtitle files):

Code: Select all

input.findAll{ it.isAudio() }*.copyTo("E:/New Music")

If you want you can do some renaming as well via getMediaInfo/moveTo. This is how you print artist/title info for example:

Code: Select all

args.getFiles{ it.isAudio() }.each{ println getMediaInfo(file:it, format:"{fn} = {media.title} by {media.performer}")}

You should be able to read all important tags via mediainfo. You can test that in the format editor using an audio file as sample input. But keep in mind that often some tags are not defined.

F12 · Post by **F12** » 16 May 2012, 09:16

Perfect! Thanks rednoah. One other thing I noticed is that the XBMC calls at the end are all in a single try...catch statement, rather than trying each telnet call separately. The functional implication is that if the first XBMC instance errors, then none of the other XBMC instances will be updated. In my case, I had one of my XBMC instances closed so my other build did not update automatically as it was listed afterwards in the parameters. Not a big issue for me as I just put all the "always-open" XBMC instances first and then the instance that may sometimes be closed last.

Post by **rednoah** » 16 May 2012, 10:11

That's already fixed in the latest instance of the utorrent-postprocess script.

shuunen · Post by **shuunen** » 31 Jul 2012, 16:03

Hi !

I tried all I could to rename audio but none works :

Code: Select all

def musicDir      = "E:/Downloads"
def musicFormat   = "E:/Sorted/Musique/{fn}.{ext}"
musicDir.getFolders{ it.hasFile{ it.isAudio() } }.each{ dir ->
	println "Processing $dir"
	
	def files = dir.listFiles{ it.isAudio() }

	rename(file:files, format:musicFormat)
}

and

Code: Select all

def musicDir      = "E:/Downloads"
def musicFormat   = "E:/Sorted/Musique/{fn}.{ext}"
musicDir.getFiles{ it.isAudio() }.each{
	rename(file:it, format:musicFormat)
}

musicDir contains many music albums that I want to format and copy to "E:/Sorted/Musique"

I call this script with :

Code: Select all

filebot -script "myscript.groovy" --action copy --conflict skip -non-strict -trust-script

Here is the error in the two code examples above :

Code: Select all

Exception: No media files: [E:\Downloads\Album Folder\01 - Track.mp3]

Anyone has an idea ?

Post by **rednoah** » 01 Aug 2012, 00:35

FileBot doesnt support audio, so rename(...) only matches episodes and movies.

Check out the topic "Any Mass Renaming" for examples how to move files with scripting.

Post by **rednoah** » 01 Aug 2012, 00:37

Link:
http://filebot.sourceforge.net/forums/v ... ?f=4&t=148

Post by **rednoah** » 01 Aug 2012, 00:39

Actually it already explains everything in the thread above. Have a look.

shuunen · Post by **shuunen** » 01 Aug 2012, 07:42

I understand, thanks

maybe a future feature for filebot ?

I think I will search for a external tool to call after renaming my movies and tv shows, in order to process a folder with mp3 to a sorted folder like "/Music/Artist - Year - Album/01 - Track 01.mp3" ,

Anyone has an idea ?

Post by **rednoah** » 01 Aug 2012, 09:18

There's excellent tools out there for identifying / tagging mp3s. Just reading tags is easy and I'm sure there is loads of cmdline tools to rename files based on their tags.

shuunen · Post by **shuunen** » 01 Aug 2012, 10:00

I found a very very good product if you want to know : http://beets.radbox.org/

Just awesome

F12 · Post by **F12** » 08 Dec 2012, 10:18

I've been happily using this script since I last posted but I am keen to just improve it a little bit and iron out any small issues. It seems to treat MP4 video as both audio and video, in that it both gets transferred to my iTunes folder and moved to my videos folder (as per normal). I'm not too sure why this is happening. Also, how can I integrate subtitle fetching into this if they are not present in the file?

The major issue was that the cmd kept stealing focus from XBMC but I found your .vbs script to address that so hopefully that issue will be resolved now.

Post by **rednoah** » 08 Dec 2012, 12:30

1. Had both mp4/m4a in media.types -> just gonna keep m4a then. You can quick fix that by not just using f.isAudio() but f.isAudio() && f.length() < 20*1024*1024, something like that, isAudio() only checks for extensions so you can check filesize for sanity.

2. getSubtitles(...) not working? If filebot -get-subtitles <file> works then it should also work in utorrent-postprocess (activate via --def subtitles=y)

F12 · Post by **F12** » 05 Oct 2013, 04:42

Somehow the functionality seems to have broken in the last week or so, not exactly sure where the issue lies. Looks like there might have been a change server-side?

Code: Select all

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\User>filebot -script "C:/Batches/utorrent-postprocess.groovy" --outpu
t "F:" --action copy --conflict override -non-strict -trust-script -Xxbmc=localh
ost "-Xut_dir=D:\Downloads\Torrents\Seeding\Parks.and.Recreation.S06E03.720p.HDT
V.X264-DIMENSION[rarbg]" "-Xut_file=Parks.and.Recreation.S06E03.720p.HDTV.X264-D
IMENSION.mkv" "-Xut_label=" "-Xut_state=5" "-Xut_kind=multi"
Parameter: ut_label =
Parameter: ut_kind = multi
Parameter: ut_dir = D:\Downloads\Torrents\Seeding\Parks.and.Recreation.S06E03.72
0p.HDTV.X264-DIMENSION[rarbg]
Parameter: xbmc = localhost
Parameter: ut_state = 5
Parameter: ut_file = Parks.and.Recreation.S06E03.720p.HDTV.X264-DIMENSION.mkv
Input: D:\Downloads\Torrents\Seeding\Parks.and.Recreation.S06E03.720p.HDTV.X264-
DIMENSION[rarbg]\parks.and.recreation.603.720p-dimension.nfo
Input: D:\Downloads\Torrents\Seeding\Parks.and.Recreation.S06E03.720p.HDTV.X264-
DIMENSION[rarbg]\Parks.and.Recreation.S06E03.720p.HDTV.X264-DIMENSION.mkv
Input: D:\Downloads\Torrents\Seeding\Parks.and.Recreation.S06E03.720p.HDTV.X264-
DIMENSION[rarbg]\RARBG.com.txt
Input: D:\Downloads\Torrents\Seeding\Parks.and.Recreation.S06E03.720p.HDTV.X264-
DIMENSION[rarbg]\Sample\p.n.fn.r.603.720p-dimension.sample.mkv
Oct 05, 2013 12:23:00 PM net.sourceforge.filebot.media.MediaDetection detectSeri
esNames
WARNING: Failed to match folder structure: http://www.filebot.net/data/series.li
st.gz
java.io.FileNotFoundException: http://www.filebot.net/data/series.list.gz
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Sou
rce)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unkno
wn Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
        at net.sourceforge.filebot.web.WebRequest.fetch(Unknown Source)
        at net.sourceforge.filebot.web.WebRequest.fetchIfModified(Unknown Source
)
        at net.sourceforge.filebot.web.CachedResource.fetchData(Unknown Source)
        at net.sourceforge.filebot.web.CachedResource.get(Unknown Source)
        at net.sourceforge.filebot.media.ReleaseInfo.getSeriesList(Unknown Sourc
e)
        at net.sourceforge.filebot.media.MediaDetection.matchSeriesByName(Unknow
n Source)
        at net.sourceforge.filebot.media.MediaDetection.detectSeriesNames(Unknow
n Source)
        at net.sourceforge.filebot.media.MediaDetection$detectSeriesNames.call(U
nknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
teArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
llSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
llSite.java:120)
        at Script2.detectSeriesName(Script2.groovy:168)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMet
hodSite.invoke(PogoMetaMethodSite.java:226)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(P
ogoMetaMethodSite.java:52)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent
(CallSiteArray.java:46)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(Abs
tractCallSite.java:133)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(Abs
tractCallSite.java:145)
        at Script2.detectSeriesName(Script2.groovy)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:
90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1053)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:920)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:883)
        at groovy.lang.Closure.call(Closure.java:410)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.callGlobal(GroovySc
riptEngineImpl.java:380)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.access$000(GroovySc
riptEngineImpl.java:72)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(Groo
vyScriptEngineImpl.java:301)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(Groo
vyScriptEngineImpl.java:290)
        at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java
:44)
        at groovy.lang.Script.invokeMethod(Script.java:78)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeOnDelega
tionObjects(ClosureMetaClass.java:407)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(C
losureMetaClass.java:346)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:883)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(Po
goMetaClassSite.java:66)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent
(CallSiteArray.java:46)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(Abs
tractCallSite.java:133)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(Abs
tractCallSite.java:141)
        at Script3$_run_closure5.doCall(Script3.groovy:27)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:
90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(C
losureMetaClass.java:272)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:883)
        at groovy.lang.Closure.call(Closure.java:410)
        at groovy.lang.Closure.call(Closure.java:423)
        at org.codehaus.groovy.runtime.DefaultGroovyMethods.groupBy(DefaultGroov
yMethods.java:3052)
        at org.codehaus.groovy.runtime.dgm$364.invoke(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMetho
dSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:271)
        at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta
MethodSite.java:53)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
teArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
llSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
llSite.java:116)
        at Script3.run(Script3.groovy:26)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEn
gineImpl.java:315)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEn
gineImpl.java:111)
        at javax.script.AbstractScriptEngine.eval(Unknown Source)
        at net.sourceforge.filebot.cli.ScriptShell.evaluate(Unknown Source)
        at net.sourceforge.filebot.cli.ScriptShell.run(Unknown Source)
        at net.sourceforge.filebot.cli.ScriptShell.run(Unknown Source)
        at net.sourceforge.filebot.cli.ArgumentProcessor.process(Unknown Source)

        at net.sourceforge.filebot.Main.main(Unknown Source)
Caused by: java.io.FileNotFoundException: http://www.filebot.net/data/series.lis
t.gz
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
        at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown So
urce)
        at java.net.URLConnection.getHeaderFieldLong(Unknown Source)
        at java.net.URLConnection.getContentLengthLong(Unknown Source)
        at java.net.URLConnection.getContentLength(Unknown Source)
        ... 72 more

FileNotFoundException: http://api.themoviedb.org/2.1/Movie.search/en/xml/5a6edae
568130bf10617b6d45be99f13/Parks+and+Recreation+S06E03
java.io.FileNotFoundException: http://api.themoviedb.org/2.1/Movie.search/en/xml
/5a6edae568130bf10617b6d45be99f13/Parks+and+Recreation+S06E03
        at net.sourceforge.filebot.web.WebRequest.getReader(Unknown Source)
        at net.sourceforge.filebot.web.CachedPage.openConnection(Unknown Source)

        at net.sourceforge.filebot.web.TMDbClient$1.openConnection(Unknown Sourc
e)
        at net.sourceforge.filebot.web.CachedPage.fetchData(Unknown Source)
        at net.sourceforge.filebot.web.CachedResource.get(Unknown Source)
        at net.sourceforge.filebot.web.TMDbClient.fetchResource(Unknown Source)
        at net.sourceforge.filebot.web.TMDbClient.getMovies(Unknown Source)
        at net.sourceforge.filebot.web.TMDbClient.searchMovie(Unknown Source)
        at net.sourceforge.filebot.media.MediaDetection.queryMovieByFileName(Unk
nown Source)
        at net.sourceforge.filebot.media.MediaDetection.detectMovie(Unknown Sour
ce)
        at net.sourceforge.filebot.media.MediaDetection$detectMovie.call(Unknown
 Source)
        at Script2.detectMovie(Script2.groovy:173)
        at Script2.detectMovie(Script2.groovy)
        at Script3$_run_closure5.doCall(Script3.groovy:28)
        at Script3.run(Script3.groovy:26)
        at net.sourceforge.filebot.cli.ScriptShell.evaluate(Unknown Source)
        at net.sourceforge.filebot.cli.ScriptShell.run(Unknown Source)
        at net.sourceforge.filebot.cli.ScriptShell.run(Unknown Source)
        at net.sourceforge.filebot.cli.ArgumentProcessor.process(Unknown Source)

        at net.sourceforge.filebot.Main.main(Unknown Source)
Failure (░_░)
Launch4j: Failed to run the given command.

On a mostly unrelated issue, would it be possible to include functionality to ignore sample videos downloaded, either in the script or even as core Filebot functionality? Usually I disable downloading them in the client but sometimes I forget and often Filebot and the script end up copying the sample video to the directory rather than the proper file. They are common in scene releases to check the quality. Screening for "sample" in the filename should be sufficient, it is also usually in a "Sample" subfolder and obviously much smaller in file size than the correct file. You can see an example in the cmd line output above.

Post by **rednoah** » 05 Oct 2013, 10:47

series.list.gz has been removed in recent versions. I think your filebot install is too outdated. Try updating.

EDIT: Looks like TheMovieDB turned off the old deprecated 2.1 API

F12 · Post by **F12** » 05 Oct 2013, 11:31

I thought that might be the case so I updated, but the above log is after updating to the most recent version earlier today. Is TheMovieDB issue with the Groovy script or more internal to Filebot functioning?

Post by **rednoah** » 05 Oct 2013, 12:21

Nope, the log above is most definitely made by outdated filebot code. Maybe you have two installs?

F12 · Post by **F12** » 06 Oct 2013, 06:44

Looks like you're right, the GUI asks me to update when launched. It would seem either the installer isn't overwriting for some reason or there's multiple installs, but I can't find any duplicate install in Program Files or a reason for such.

Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading

Re: Fully automated media downloading