Board index Scripting and Automation Patchwork code for XBMC automation

Patchwork code for XBMC automation

Running FileBot from the console, Groovy / FileBot scripting, shell scripts, etc


Posts: 21
I made a patchwork of the code kindly provided by Rednoah and many others here on this board that allows utorrent to:

1) launch a groovy script that
2) extracts rar files if necessary
3) ignore unfinished files (if more than one download is happening at the same time)
4) delete "sample" files so that they aren't processed
5) determine if the file is a movie or tv show
6) collect artwork and get subtitles from theMovieDB or theTVDB respectively
7) rename shows according to XBMC convention
8) create the appropriate directory if necessary
9) Send me a text message when finished to notify me of its completion and what was downloaded

Some of the things that I learned along the way:
- You should troubleshoot by looking at the log (tab at the bottom) from utorrent and cut and paste that into a cmd window. You can also troubleshoot groovy script with the "Groovy Console". I found the later less helpful because you often need the variables passed from utorrent to troubleshoot.
- Java for me (Windows 7 64bit) needed to be moved to the program files (x64) directory and permissions changed.
- sendemail would not work from the suggested directory (system32). I need to place it in the standard windows directory. It would work fine from the command line, but for some reason when called from the groovy script with the same command, it would fail until I moved the program.

I use a separate program "Belvedere" that keeps my downloads folder clean. After appropriate seeding time, it deletes the original downloaded torrent.

Huge thanks to Rednoah! I take no credit for any of the script. I felt like a child trying to build a rocket ship most of the time. He was very patient and helpful in constructing this hodgepodge of other people's work. I hope that some of this information can save others some frustration and time.

// filebot -script "fn:utorrent-postprocess" --output "X:/media" --action copy --conflict override -non-strict -trust-script -Xxbmc=localhost "-Xut_dir=%D" "-Xut_file=%F" "-Xut_label=%L" "-Xut_state=%S" "-Xut_kind=%K"
def input = []

// print input parameters
_args.parameters.each{ k, v -> println "Parameter: $k = $v" }

if (ut_kind == "multi") {
  input += new File(ut_dir).getFiles() // multi-file torrent
} else {
  input += new File(ut_dir, ut_file) // single-file torrent
}

// extract archives if necessary
input += extract(file:input, output:".", conflict:"override")

// process only media files
input = input.findAll{ it.isVideo() || it.isSubtitle() }

// ignore chunk, part, par and hidden files
def incomplete(f) { f.name =~ /[.]incomplete|[.]chunk|[.]par$|[.]dat$/ || f.isHidden() }
[ut_dir].getFolders{ !it.hasFile{ incomplete(it) } && it.hasFile{ it.isVideo() } }.each{ dir ->
   println "Deleting Sample Video Processing $dir"
   def files = dir.listFiles{ it.isVideo() }

// if name contains "sample" and filesize is less than 50 MB
//def selection = args.getFiles{ it.name =~ "sample" && it.length() < 20 * 1024 * 1024}
   def selection = dir.getFiles{ (it.name =~ "sample" || it.name =~ "Sample") && it.length() < 50 * 1024 * 1024}
   selection.each{ println "delete $it" }

// delete all
   selection*.delete()
}

// ignore clutter files
input = input.findAll{ !(it.name =~ /(?i:sample)/) }

// print input fileset
input.each{ println "Input: $it" }

// xbmc artwork/nfo utility
include("fn:lib/xbmc")


// group episodes/movies and rename according to XBMC standards
def groups = input.groupBy{
  def tvs = detectSeriesName(it)
  def mov = detectMovie(it, false)
  println "$it.name [series: $tvs, movie: $mov]"

  // DECIDE EPISODE VS MOVIE (IF NOT CLEAR)
  if (tvs && mov) {
    if (it.name =~ "(?i:$tvs - .+)" || parseEpisodeNumber(it.name) || parseDate(it.name)) {
      println "Exclude Movie: $mov"
      mov = null
    } else if (detectMovie(it, true)) {
      println "Exclude Series: $tvs"
      tvs = null
    }
  }
  return [tvs:tvs, mov:mov]
}

groups.each{ group, files ->
  // fetch subtitles
  def subs = getMissingSubtitles(file:files, output:"srt", encoding:"utf-8")
  if (subs) files += subs

  // EPISODE MODE
  if (group.tvs && !group.mov) {
    def dest = rename(file:files, format:'TV Shows/{n}/{episode.special ? "Special" : "Season "+s}/{n} - {episode.special ? "S00E"+special.pad(2) : s00e00} - {t}', db:'TheTVDB')

    dest.mapByFolder().keySet().each{ dir ->
      println "Fetching artwork for $dir from TheTVDB"
      def query = group.tvs
      def sxe = dest.findResult{ parseEpisodeNumber(it) }
      def options = TheTVDB.search(query)
      if (options.isEmpty()) {
        println "TV Series not found: $query"
        return
      }

   dest.findAll{ it.isVideo() }.each{
   def msg = "${it.name} has been added to the library"
   execute("c:/windows/sendEmail.exe", "-f", "from@gmail.com", "-t", "10digitcellphonenumber@messaging.sprintpcs.com", "-s", "smtp.gmail.com:587", "-o", "tls=yes", "-xu", "from@gmail.com", "-xp", "password", "-m", msg)
   }
      options = options.sortBySimilarity(query, { it.name })
      fetchSeriesArtworkAndNfo(dir.dir, dir, options[0], sxe && sxe.season > 0 ? sxe.season : 1)
    }
  }

  // MOVIE MODE
  if (group.mov && !group.tvs) {
    def dest = rename(file:files, format:'Movies/{n} ({y})/{n} ({y}){" CD$pi"}', db:'TheMovieDB')

    dest.findAll{ it.isVideo() }.each{
   def msg = "${it.name} has been added to the library"
   execute("c:/windows/sendEmail.exe", "-f", "from@gmail.com", "-t", "10digitcellphonenumber@messaging.sprintpcs.com", "-s", "smtp.gmail.com:587", "-o", "tls=yes", "-xu", "from@gmail.com", "-xp", "password", "-m", msg)
   }

    dest.mapByFolder().keySet().each{ dir ->
      println "Fetching artwork for $dir from TheMovieDB"
      fetchMovieArtworkAndNfo(dir, group.mov)
    }
  }
}



// make XBMC scan for new content
xbmc.split(/[\s,|]+/).each{
  println "Notify XBMC: $it"
  invokeScanVideoLibrary(it)
}

rednoah User avatar
The Source

Posts: 2090
Location: 北京

You might wanna rethink how you handles samples. If you delete those doesn't that cause the torrent to be stopped? Or redownloaded, refinished and this script rerun?

You don't have to delete anything, just ignore those files. Exclude them from the list of files that you're processing like this:
input = input.findAll{ !(it.path =~ /\b(?i:sample|trailer|extras|deleted.scenes|music.video|scrapbook)\b/) || it.length() > 20 * 1024 * 1024}
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image


Posts: 21
Hunh, I haven't had any problems with that an I have tested it on some files that contained sample files. That is part of the reason it took me this long to write this; I wanted to make sure that there were no major errors. I do seem to sometimes get an extra page if I donwload a season pack. There may be one episode that I get a couple pages for. Other than that, it has been working great. Having said that, I may go ahead and change my script to your suggested code. Once again, much appreciation!


Posts: 21
Everything seems to be working great, but it does not appear to be fetching clear logos (for Neon skin in XBMC) nor the Fanart. The thumb gets fetched just fine. Is this normal? It's not a huge deal, I end up just refreshing the video file and pulling it in from XBMC, but would be great if the script did this.

rednoah User avatar
The Source

Posts: 2090
Location: 北京

That's what it's doing right now:
viewtopic.php?f=4&t=181

Are you talking about series artwork or movie artwork?
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image


Posts: 21
rednoah wrote:
That's what it's doing right now:
viewtopic.php?f=4&t=181

Are you talking about series artwork or movie artwork?


I was talking about Movie fanart and TV clear logos. Sorry for not being more specific.

rednoah User avatar
The Source

Posts: 2090
Location: 北京

As far as I can tell TheMovieDB doesn't have fanart and I don't know what "clear logos" are, but I don't think TheTVDB has them. Anyway if you run the artwork.* scripts you'll get lots of output as to what artwork is available. If on there I can add support for that.
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image


Posts: 21
I guess TheMovieDB calls it "backdrops" as you referenced previously. Anyway, they don't seem to load for me unless I do a refresh. As far as the clear logos. I believe they come from here: http://fanart.tv/tv-fanart/

rednoah User avatar
The Source

Posts: 2090
Location: 北京

1. artwork.tmdb already downloads backdrop:original as backdrop.jpg. You can modify the script to grab extra types. Let me know which is the one you want.
fetchArtwork(movieDir['backdrop-cover.jpg'], movieInfo, 'backdrop', 'cover')
fetchArtwork(movieDir['backdrop-mid.jpg'], movieInfo, 'backdrop', 'mid')
fetchArtwork(movieDir['backdrop-thumb.jpg'], movieInfo, 'backdrop', 'thumb')


2. I'll look into their API
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image

rednoah User avatar
The Source

Posts: 2090
Location: 北京

It's pretty easy to access fanart.tv so I'll added some support for that with r1092.

e.g.
import static net.sourceforge.filebot.WebServices.*

FanartTV.getArtwork(70327).each {
   println it
   println it.url.saveAs("artwork/$it.type/$it.name")
}


But I'll only add code for fanart.tv in my scripts some time after the next release. But you can play with that yourself for the time being.
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image


Posts: 21
Hmm, it seems to be downloading the backdrops, but they don't show up in XMBC. Not sure what's going on. The XBMC library is getting updated because the new movie shows up along with the right thumb, but the backdrop doesn't. I've tried to exit and restart XMBC as well as "update library", but neither work. The only thing that helps is if I refresh the file and then It just re-downloads everything (my default scraper is different). Maybe there's something wrong with my installation. Not sure, but its not that huge a deal to me considering how automated it currently is.

rednoah User avatar
The Source

Posts: 2090
Location: 北京

I don't use XBMC so maybe the script is not saving things in the proper filenames that XBMC will recognise.
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image


Posts: 21
Thanks again for all your help. Initially, I realized that the newer version of XBMC needs the backdrops to be named "wallpaper" instead, so I changed that in the script. Then I realized that XBMC will automatically bring in the backdrops, episode information, etc. Having them in the folder overrides XBMC. The only thing that it doesn't do by default is bring in subtitles. So, I ended up stripping out the script information for adding artwork and episode or movie information. Here are the updated scripts to accomplish this:

// filebot -script "fn:utorrent-postprocess" --output "X:/media" --action copy --conflict override -non-strict -trust-script -Xxbmc=localhost "-Xut_dir=%D" "-Xut_file=%F" "-Xut_label=%L" "-Xut_state=%S" "-Xut_kind=%K"
def input = []

// print input parameters
_args.parameters.each{ k, v -> println "Parameter: $k = $v" }

if (ut_kind == "multi") {
  input += new File(ut_dir).getFiles() // multi-file torrent
} else {
  input += new File(ut_dir, ut_file) // single-file torrent
}

// extract archives if necessary
input += extract(file:input, output:".", conflict:"override")

// process only media files
input = input.findAll{ it.isVideo() || it.isSubtitle() }

// ignore chunk, part, par and hidden files
def incomplete(f) { f.name =~ /[.]incomplete|[.]chunk|[.]par$|[.]dat$/ || f.isHidden() }
[ut_dir].getFolders{ !it.hasFile{ incomplete(it) } && it.hasFile{ it.isVideo() } }.each{ dir ->
   println "Deleting Sample Video Processing $dir"
   def files = dir.listFiles{ it.isVideo() }

// ignore samples
   input = input.findAll{ !(it.path =~ /\b(?i:sample|trailer|extras|deleted.scenes|music.video|scrapbook)\b/) || it.length() > 50 * 1024 * 1024}

}
// ignore clutter files
input = input.findAll{ !(it.name =~ /(?i:sample)/) }

// print input fileset
input.each{ println "Input: $it" }

// xbmc artwork/nfo utility - ***This refers to the second script below.  Make sure you change to the correct location***
include("X:/Users/YourDirectoryLocation/XBMC.groovy")


// group episodes/movies and rename according to XBMC standards
def groups = input.groupBy{
  def tvs = detectSeriesName(it)
  def mov = detectMovie(it, false)
  println "$it.name [series: $tvs, movie: $mov]"

  // DECIDE EPISODE VS MOVIE (IF NOT CLEAR)
  if (tvs && mov) {
    if (it.name =~ "(?i:$tvs - .+)" || parseEpisodeNumber(it.name) || parseDate(it.name)) {
      println "Exclude Movie: $mov"
      mov = null
    } else if (detectMovie(it, true)) {
      println "Exclude Series: $tvs"
      tvs = null
    }
  }
  return [tvs:tvs, mov:mov]
}

groups.each{ group, files ->
  // fetch subtitles
  def subs = getMissingSubtitles(file:files, output:"srt", encoding:"utf-8")
  if (subs) files += subs

  // EPISODE MODE
  if (group.tvs && !group.mov) {
    def dest = rename(file:files, format:'TV Shows/{n}/{episode.special ? "Special" : "Season "+s}/{n} - {episode.special ? "S00E"+special.pad(2) : s00e00} - {t}', db:'TheTVDB')

   dest.findAll{ it.isVideo() }.each{
   def msg = "${it.name} has been added to the library"
   execute("c:/windows/sendEmail.exe", "-f", "username@gmail.com", "-t", "5555555555@messaging.sprintpcs.com", "-s", "smtp.gmail.com:587", "-o", "tls=yes", "-xu", "username@gmail.com", "-xp", "password", "-m", msg)
   }
  }

  // MOVIE MODE
  if (group.mov && !group.tvs) {
    def dest = rename(file:files, format:'Movies/{n} ({y})/{n} ({y}){" CD$pi"}', db:'TheMovieDB')

    dest.findAll{ it.isVideo() }.each{
   def msg = "${it.name} has been added to the library"
   execute("c:/windows/sendEmail.exe", "-f", "username@gmail.com", "-t", "5555555555@messaging.sprintpcs.com", "-s", "smtp.gmail.com:587", "-o", "tls=yes", "-xu", "username@gmail.com", "-xp", "password", "-m", msg)
   }


  }
}



// make XBMC scan for new content
xbmc.split(/[\s,|]+/).each{
  println "Notify XBMC: $it"
  invokeScanVideoLibrary(it)
}


and:

// xbmc functions
def invokeScanVideoLibrary(host, port = 9090) {
  try {
    telnet(host, port) { writer, reader ->
      writer.println('{"id":1,"method":"VideoLibrary.Scan","params":[],"jsonrpc":"2.0"}') // API call for latest XBMC release
    }
    return true
  } catch(e) {
    println "${e.class.simpleName}: ${e.message}"
    return false
  }
}


Posts: 169
Actually it really should be fanart.jpg or <moviename>-fanart.jpg.

Check here for a pretty good guide (harder to find something official on this surprisingly):
http://forum.xbmc.org/showthread.php?tid=59281

rednoah User avatar
The Source

Posts: 2090
Location: 北京

// fetch series banner, fanart, posters, etc
      fetchMovieArtwork(movieDir['folder.jpg'], movieInfo, 'posters', locale.language)
      fetchMovieArtwork(movieDir['wallpaper.jpg'], movieInfo, 'backdrops', locale.language)
      
      fetchMovieFanart(movieDir['logo.png'], movieInfo, 'movielogo', null, locale)
      fetchMovieFanart(movieDir['fanart.png'], movieInfo, 'movieart', null, locale)
      ['bluray', 'dvd', null].findResult { diskType -> fetchMovieFanart(movieDir['disc.png'], movieInfo, 'moviedisc', diskType, locale) }


So what should I do?

I fetch:
tmdb poster as folder.jpg
tmdb backdrop as wallpaper.jpg (correct?)
fanarttv movielogo as logo.png
fanarttv movieart as fanart.png
fanarttv moviedisc as disk.png

What do I have to change so it matches XBMC better?

This is the latest SVN rev btw, bit different from what you had since i added support for the FanartTV API.
FileBot is free software. Please help support FileBot by writing a review or considering a donation.
Image


Posts: 169
Actually I just started a new post with a request for changes to artwork here:
viewtopic.php?f=4&t=207

rednoah wrote:
tmdb poster as folder.jpg
tmdb backdrop as wallpaper.jpg (correct?)
fanarttv movielogo as logo.png
fanarttv movieart as fanart.png
fanarttv moviedisc as disk.png

1.I say folder.jpg and movie.tbn although personally I also have poster.jpg as well, might really be overkill for some people. Doesn't hurt me to have 1 or 2 extra files in there
2.I've never seen anyone talk about using wallpaper.jpg, should be fanart.jpg or <moviename>-fanart.jpg for ultimate compatibility
3,4,5. I haven't touched my TV stuff yet, still sorting out movies.

You can also check another reference from an xbmc member who has an app just to download artwork for xbmc:
http://wiki.xbmc.org/index.php?title=Ad ... rk_sources:


Posts: 21
part timer wrote:
Actually it really should be fanart.jpg or <moviename>-fanart.jpg.

Check here for a pretty good guide (harder to find something official on this surprisingly):
http://forum.xbmc.org/showthread.php?tid=59281


Sorry, that is correct. It's been a week or so since I was playing with it. I knew it wasn't backdrop, just got fanart and wallpaper mixed up. I know there are some advantages to having the artwork files manually placed in the folders, but I'm fine with XBMC doing it automatically for me.


Posts: 169
This is true. XBMC can do all the heavy lifting of getting all nfo info and artwork and everything, but then it takes a while to do it. My main reason for wanting it locally in the folder with the movie was if xbmc ever crashes which only happened once so far for me, then you have to wait days for it to reload everything.

If it's all right there in the folders though, it takes way less time to grab it and show it. Plus it gives me that chance to confirm as I go that I'm getting the right data for the right movie.

I hate sitting there and waiting for it to scan the library of thousands of files, personally.


Return to Scripting and Automation