script failure becuase of episode or movie name with colon

Running FileBot from the console, Groovy scripting, shell scripts, etc
Post Reply
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

script failure becuase of episode or movie name with colon

Post by Maxxodd »

I am trying to put the finishing touches on my script that is a Frankenstein mis-mash of several other's great work. This script un-rars files if they need it. It deletes sample files so that you don't get duplicates or duplicate notifications. It uses filebot to rename episodes according to XBMC convention. It sends me text messages via sendemail to notify me that an episode has been added to the library and updates XBMC. Everything works great with the exeption that if the file contains a colon, then the script fails. The only problem I am having now is that files that have a colon or apostrophe fail. Is there a way to ask filebot to validate the file name while processing?

Code: Select all

// filebot -script "fn:utorrent-postprocess" --output "X:/media" --action copy --conflict override -non-strict -trust-script -Xxbmc=localhost "-Xut_dir=%D" "-Xut_file=%F" "-Xut_label=%L" "-Xut_state=%S" "-Xut_kind=%K"
def input = []

// print input parameters
_args.parameters.each{ k, v -> println "Parameter: $k = $v" }

if (ut_kind == "multi") {
  input += new File(ut_dir).getFiles() // multi-file torrent
} else {
  input += new File(ut_dir, ut_file) // single-file torrent
}

// extract archives if necessary
input += extract(file:input, output:".", conflict:"override")

// process only media files
input = input.findAll{ it.isVideo() || it.isSubtitle() }

// ignore chunk, part, par and hidden files
def incomplete(f) { f.name =~ /[.]incomplete|[.]chunk|[.]par$|[.]dat$/ || f.isHidden() }
[ut_dir].getFolders{ !it.hasFile{ incomplete(it) } && it.hasFile{ it.isVideo() } }.each{ dir ->
   println "Deleting Sample Video Processing $dir"
   def files = dir.listFiles{ it.isVideo() }
// if name contains "sample" and filesize is less than 50 MB
//def selection = args.getFiles{ it.name =~ "sample" && it.length() < 20 * 1024 * 1024}
   def selection = dir.getFiles{ (it.name =~ "sample" || it.name =~ "Sample") && it.length() < 50 * 1024 * 1024}
   selection.each{ println "delete $it" }

// delete all
   selection*.delete()
}

// print input fileset
input.each{ println "Input: $it" }

// xbmc artwork/nfo utility
include("fn:lib/xbmc")


// group episodes/movies and rename according to XBMC standards
def groups = input.groupBy{
  def tvs = detectSeriesName(it)
  def mov = detectMovie(it, false)
  println "$it.name [series: $tvs, movie: $mov]"

  // DECIDE EPISODE VS MOVIE (IF NOT CLEAR)
  if (tvs && mov) {
    if (it.name =~ "(?i:$tvs - .+)" || parseEpisodeNumber(it.name) || parseDate(it.name)) {
      println "Exclude Movie: $mov"
      mov = null
    } else if (detectMovie(it, true)) {
      println "Exclude Series: $tvs"
      tvs = null
    }
  }
  return [tvs:tvs, mov:mov]
}

groups.each{ group, files ->
  // fetch subtitles
  def subs = getMissingSubtitles(file:files, output:"srt", encoding:"utf-8")
  if (subs) files += subs

  // EPISODE MODE
  if (group.tvs && !group.mov) {
    def dest = rename(file:files, format:'TV Shows/{n}/{episode.special ? "Special" : "Season "+s}/{n} - {episode.special ? "S00E"+special.pad(2) : s00e00} - {t}', db:'TheTVDB')

    dest.mapByFolder().keySet().each{ dir ->
      println "Fetching artwork for $dir from TheTVDB"
      def query = group.tvs
      def sxe = dest.findResult{ parseEpisodeNumber(it) }
      def options = TheTVDB.search(query)
      if (options.isEmpty()) {
        println "TV Series not found: $query"
        return
      }

   dest.findAll{ it.isVideo() }.each{
   def msg = "${it.name} has been added to the library"
   execute("c:/windows/sendEmail.exe", "-f", "[email protected]", "-t", "[email protected]", "-s", "smtp.gmail.com:587", "-o", "tls=yes", "-xu", "[email protected]", "-xp", "password", "-m", msg)
   }
      options = options.sortBySimilarity(query, { it.name })
      fetchSeriesArtworkAndNfo(dir.dir, dir, options[0], sxe && sxe.season > 0 ? sxe.season : 1)
    }
  }

  // MOVIE MODE
  if (group.mov && !group.tvs) {
    def dest = rename(file:files, format:'Movies/{n} ({y})/{n} ({y}){" CD$pi"}', db:'TheMovieDB')

    dest.findAll{ it.isVideo() }.each{
   def msg = "${it.name} has been added to the library"
   execute("c:/windows/sendEmail.exe", "-f", "[email protected]", "-t", "[email protected]", "-s", "smtp.gmail.com:587", "-o", "tls=yes", "-xu", "[email protected]", "-xp", "password", "-m", msg)
   }

    dest.mapByFolder().keySet().each{ dir ->
      println "Fetching artwork for $dir from TheMovieDB"
      fetchMovieArtworkAndNfo(dir, group.mov)
    }
  }
}



// make XBMC scan for new content
xbmc.split(/[\s,|]+/).each{
  println "Notify XBMC: $it"
  invokeScanVideoLibrary(it)
}
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

Even windows allows ; in filenames. What part of your script is the colon breaking?

If colon in in bindings like {n} {t} etc are breaking things it's easy to validate that in your expression:

Code: Select all

{t.replaceAll(';')}
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

I think it happens not when the file contains a colon :, but when the TVDB episode title contains a colon. I am not sure this is the case, but it seems to only fail when that scenerio plays out. It will extract the rar files. It deletes the sample files, but then fails when trying to rename the episode to an episode title that contains a colon. When I use filebot directly, the file is in red and I have to click "validate" to change the episode title from one with a colon in it to one without a colon. That is the basis of my reasoning in thinking that that is the problem. Does this make any sense?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

Wait, you mean colon : not semi-colon ;

That should be handled right out of the box. Cmdline will catch that and output "Stripping invalid characters from new path: ..."

Did you ever change the filebot.cmd script? It should say --Dunixfs=false (not set at all is the same as false)
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

Can you run the script in cmdline and send me the output? uTorrent log will contain the exact cmd it calls, just copy and run again in cmdline so you can copy the output.
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

No, I did not. What does --Dunixfs=false do? In the mean time. I'll try to check if that works now. I already deleted the original files. I'll re-download them and let you know what happens. If it fails, I'll try it from cmd line and send you the log.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

-Dunixfs=true is for Linux/Mac users that don't like Windows filename restrictions. It's not default of course.

Works like this:

Code: Select all

E:\Storage\Avatar>filebot -version
FileBot 2.62 (r996) / Java(TM) SE Runtime Environment 1.7.0_03

E:\Storage\Avatar>filebot -rename . --format "{n}::{y}"
Filename pattern: [0,00] SxE, [0,00] CWS
Rename movies using [OpenSubtitles]
Looking up movie by filehash via [OpenSubtitles]
Stripping invalid characters from new path: Avatar::2009
[MOVE] Rename [E:\Storage\Avatar\.\Avatar.mp4] to [Avatar2009.mp4]
Processed 1 files
Done ?(?????)?
So rename(...) should work as expected. Maybe other parts of the script are at fault? I'll need the console output to know more.
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

How do I find the utorrent log? I tried a google search for utorrent log and there was nothing useful.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

Ahh, do you mean the "logger"? Download almost finished. Will let you know in a few. I'm sure you are on the edge of your seat....
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

Yep, the Logger tab. It'll contain the command so you know exactly how utorrent calls filebot. So you just copy that and run it in a new cmdline window where you can see the output.
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

Here is the output. I noticed that it seems to place the file in the output directory until the full copy is complete and when it fails, the file disappears. It seems to be trying to process the sample file that was deleted. Why does that make the file that was copied suddenly get deleted mid script after the copy is complete? Is this the problem? Any way to stop it from doing so?

Code: Select all

C:\Users\me>filebot -script "C:/Users/me/Downloads/utorrent1.groov
y" --output "\\READYNAS\media\Videos" --action copy --conflict override -non-str
ict -trust-script -Xxbmc=192.168.1.8 "-Xut_dir=\\READYNAS\media\Videos\Downloads
\So.You.Think.You.Can.Dance.S09E04.720p.HDTV.x264-2HD" "-Xut_file=so.you.think.y
ou.can.dance.s09e04.720p.hdtv.x264-2hd.sfv" "-Xut_label=" "-Xut_state=5" "-Xut_k
ind=multi"
Parameter: ut_label =
Parameter: ut_kind = multi
Parameter: ut_dir = \\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance
.S09E04.720p.HDTV.x264-2HD
Parameter: xbmc = 192.168.1.8
Parameter: ut_state = 5
Parameter: ut_file = so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.sfv
Extract archive [so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.rar] to [\
\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance.S09E04.720p.HDTV.x26
4-2HD\.]
Extracting files [so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.mkv]
Deleting Sample Video Processing \\READYNAS\media\Videos\Downloads\So.You.Think.
You.Can.Dance.S09E04.720p.HDTV.x264-2HD
delete \\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance.S09E04.720p.
HDTV.x264-2HD\Sample\sample-so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd
.mkv
Deleting Sample Video Processing \\READYNAS\media\Videos\Downloads\So.You.Think.
You.Can.Dance.S09E04.720p.HDTV.x264-2HD\Sample
Input: \\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance.S09E04.720p.
HDTV.x264-2HD\Sample\sample-so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd
.mkv
Input: \\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance.S09E04.720p.
HDTV.x264-2HD\.\so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.mkv
sample-so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.mkv [series: so you
think you can dance, movie: So You Think You Can Dance (2005)]
Exclude Movie: So You Think You Can Dance (2005)
so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.mkv [series: so you think y
ou can dance, movie: So You Think You Can Dance (2005)]
Exclude Movie: So You Think You Can Dance (2005)
Missing subtitles for 2 video files
Looking up subtitles by filehash via OpenSubtitles
Looking up subtitles by filehash via Sublight
Jun 13, 2012 10:33:39 PM net.sourceforge.filebot.web.SublightSubtitleClient getS
ubtitleList
WARNING: Error computing video hash: Failed to open file: \\READYNAS\media\Video
s\Downloads\So.You.Think.You.Can.Dance.S09E04.720p.HDTV.x264-2HD\Sample\sample-s
o.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.mkv
Lookup by hash failed: javax.xml.ws.WebServiceException: Response indicates erro
r: Client not supported. Contact us at: [email protected]
Searching for [so you think you can dance] at [OpenSubtitles]
Jun 13, 2012 10:33:47 PM net.sourceforge.filebot.web.OpenSubtitlesXmlRpc searchM
oviesOnIMDB
INFO: Ignore movie {id=2162228, title=Auditions #1 and #2: Dallas and New York ┬
á-┬áSeason 9, Episode 2:}: Illegal title
Jun 13, 2012 10:33:47 PM net.sourceforge.filebot.web.OpenSubtitlesXmlRpc searchM
oviesOnIMDB
INFO: Ignore movie {id=2162230, title=Auditions #2}: Illegal title
Searching for [so you think you can dance] at [Sublight]
Search for [[so you think you can dance]] failed: Response indicates error: Clie
nt not supported. Contact us at: [email protected]
Searching for [so you think you can dance] at [Subscene]
No matching subtitles found: \\READYNAS\media\Videos\Downloads\So.You.Think.You.
Can.Dance.S09E04.720p.HDTV.x264-2HD\Sample\sample-so.you.think.you.can.dance.s09
e04.720p.hdtv.x264-2hd.mkv
No matching subtitles found: \\READYNAS\media\Videos\Downloads\So.You.Think.You.
Can.Dance.S09E04.720p.HDTV.x264-2HD\.\so.you.think.you.can.dance.s09e04.720p.hdt
v.x264-2hd.mkv
Rename episodes using [TheTVDB]
Auto-detected query: [so you think you can dance]
Fetching episode data for [So You Think You Can Dance]
Fetching episode data for [So You Think You Can Dance UK]
Fetching episode data for [So You Think You Can Dance SA]
Fetching episode data for [So You Think You Can Dance (NL/BE)]
Fetching episode data for [So You Think You Can Dance Canada]
Fetching episode data for [So You Think You Can Dance Australia]
Fetching episode data for [So You Think You Can Dance Scandinavia]
Auto-detected query: [so you think you can dance]
Fetching episode data for [So You Think You Can Dance]
Fetching episode data for [So You Think You Can Dance UK]
Fetching episode data for [So You Think You Can Dance SA]
Fetching episode data for [So You Think You Can Dance (NL/BE)]
Fetching episode data for [So You Think You Can Dance Canada]
Fetching episode data for [So You Think You Can Dance Australia]
Fetching episode data for [So You Think You Can Dance Scandinavia]
Stripping invalid characters from new path: TV Shows/So You Think You Can Dance/
Season 9/So You Think You Can Dance - S09E04 - Auditions: Salt Lake City
Stripping invalid characters from new path: TV Shows/So You Think You Can Dance/
Season 9/So You Think You Can Dance - S09E04 - Auditions: Salt Lake City
[COPY] Rename [\\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance.S09E
04.720p.HDTV.x264-2HD\.\so.you.think.you.can.dance.s09e04.720p.hdtv.x264-2hd.mkv
] to [\\READYNAS\media\Videos\TV Shows\So You Think You Can Dance\Season 9\So Yo
u Think You Can Dance - S09E04 - Auditions Salt Lake City.mkv]
[COPY] Rename [\\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.Dance.S09E
04.720p.HDTV.x264-2HD\Sample\sample-so.you.think.you.can.dance.s09e04.720p.hdtv.
x264-2hd.mkv] to [\\READYNAS\media\Videos\TV Shows\So You Think You Can Dance\Se
ason 9\So You Think You Can Dance - S09E04 - Auditions Salt Lake City.mkv]
[COPY] Failed to rename [\\READYNAS\media\Videos\Downloads\So.You.Think.You.Can.
Dance.S09E04.720p.HDTV.x264-2HD\Sample\sample-so.you.think.you.can.dance.s09e04.
720p.hdtv.x264-2hd.mkv]
Processed 1 files
FileNotFoundException: Source '\\READYNAS\media\Videos\Downloads\So.You.Think.Yo
u.Can.Dance.S09E04.720p.HDTV.x264-2HD\Sample\sample-so.you.think.you.can.dance.s
09e04.720p.hdtv.x264-2hd.mkv' does not exist
NullPointerException: Cannot invoke method mapByFolder() on null object
java.lang.NullPointerException: Cannot invoke method mapByFolder() on null objec
t
        at Script3$_run_closure7.doCall(Script3.groovy:68)
        at Script3.run(Script3.groovy:59)
        at net.sourceforge.filebot.cli.ScriptShell.evaluate(Unknown Source)
        at net.sourceforge.filebot.cli.ScriptShell.runScript(Unknown Source)
        at net.sourceforge.filebot.cli.ArgumentProcessor.process(Unknown Source)

        at net.sourceforge.filebot.Main.main(Unknown Source)
Failure (░_░)

C:\Users\me>
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

You can make it more robust by checking the result of rename(...). If rename(...) fails it can mess things up later.

Code: Select all

if (dest) {
	dest.mapByFolder().keySet().each{ dir ->
		println "Fetching artwork for $dir from TheMovieDB"
		fetchMovieArtworkAndNfo(dir, group.mov)
	}
}
Main problem is how you deal with samples. Cause you delete the files and then you tell filebot to rename them. The latest utorrent-postprocess.groovy handles it by just ignoring samples when copy-moving things to a different folder.

Code: Select all

// ignore clutter files
input = input.findAll{ !(it.name =~ /(?i:sample)/) }
Here's the revision history of the latest script. I've made some tweaks since you copied it:
http://filebot.svn.sourceforge.net/view ... y?view=log
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

That makes sense. I didn't know about being able to cut and paste the utorrent "logger" output into cmd line. That could have saved me an enormous amount of debugging time! Oh well. I kind of like deleting the sample files, so I left that in there and simply added your change input to exclude sample files line after that was complete. I assume that it will exclude "sample" files regardless of case, such as "sample" and "Sample"?

I'll do a write-up this weekend or before. I think that some of the troubleshooting that I have done will be most valuable as none of the actual script is mine and it is found in chunks in other people's posts. None the less, I'll post my completed script with the appropriate credits.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

Yep, the ?i: makes it case-insensitive. I'm sure some how-to on scripting would be helpful to lots of people.
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

I don't feel remotely qualified to write a tutorial on scripting. I'd love to understand more, but some of the fundamentals of groovy elude me and I didn't feel that the web tutorials were that helpful. They jumped around too much without steadily building the skill-set. I'm a surgeon and am probably too set in my ways of expected learning. Trying to pick up on groovy have been a humbling experience. The next time I'm at the bookstore, I'm going to try to pick up a book on Java and if I can find one a book on groovy.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

Actually I kinda meant how pieced things together from parts of the samples. No worries. Any help is appreciated. ;)
:idea: Please read the FAQ and How to Request Help.
Maxxodd
Posts: 23
Joined: 27 May 2012, 21:01

Re: script failure becuase of episode or movie name with col

Post by Maxxodd »

Oh, gotcha. I can do that.
Cantello
Posts: 4
Joined: 04 Oct 2012, 05:46

Re: script failure becuase of episode or movie name with col

Post by Cantello »

Speaking of the colon: is it somehow possible to change the validation scheme to something besides simply deleting the colon in the file name? This sometimes makes for some awkward-looking file names.
Often a dash (" - ") would be better, at least in my opinion.

Other than that: perfect, saved me countless minutes (hours?) of renaming! Thanks! :-)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: script failure becuase of episode or movie name with col

Post by rednoah »

I only force-validate the name when renaming the file. You can always adjust the format expression to fix things the way you want, e.g. {t.replaceAll(/:/, ' - ')}
:idea: Please read the FAQ and How to Request Help.
Cantello
Posts: 4
Joined: 04 Oct 2012, 05:46

Re: script failure becuase of episode or movie name with col

Post by Cantello »

rednoah wrote:I only force-validate the name when renaming the file. You can always adjust the format expression to fix things the way you want, e.g. {t.replaceAll(/:/, ' - ')}
That's it, thanks for helping even though I apparently did not think before asking... ^_^
Post Reply