Page 1 of 1
Hardlinking with "Fully automated media center" script
Posted: 04 Feb 2013, 20:29
by jkaberg
Noticed the script supports hardlinks, which would be very nice for me
Using --action symlink|keeplink|hardlink requires at least Java 7 and (on Windows) also admin priviledges so you'll need to run µTorrent as admin
But how does it work? Will it extract files that need extracting, and hardlink media files (.mkv, .avi etc?) or will it hardlink everything including .rar files?
Im currently running
filebot -script fn:utorrent-postprocess --output "F:/Downloaded/_automated/_postprocess" --action copy --conflict override -non-strict --def "seriesFormat=tv/%N/{fn}" "animeFormat=anime/%N/{fn}" "movieFormat=movie/%N/{fn}" subtitles=n artwork=n "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L" "ut_state=%S"
But with the hardlink option I guess it would look like
filebot -script fn:utorrent-postprocess --output "F:/Downloaded/_automated/_postprocess" --action copy hardlink --conflict override -non-strict --def "seriesFormat=tv/%N/{fn}" "animeFormat=anime/%N/{fn}" "movieFormat=movie/%N/{fn}" subtitles=n artwork=n "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L" "ut_state=%S"
Re: Hardlinking with "Fully automated media center" script
Posted: 04 Feb 2013, 20:41
by jkaberg
Ok so I tried this
Downloaded a torrent wich only contained a file (a series) with file ending .mkv (which got hardlinked), then I downloaded another series with a .rar set inside which ended up in extracting the series to <rarset dir><torrentname>/<torrentname>.mkv and then hardlinked that file
Shouldnt this just extract it directly to destination without unpacking in the rarset dir first?
My point with using this is not ending up with 2 copys of the .mkv file (less diskspace used) but still beeing able to seed the original files
I used this with uTorrent
Code: Select all
filebot -script fn:utorrent-postprocess --output "F:/Downloaded/_automated/_postprocess" --action hardlink --conflict override -non-strict --def "seriesFormat=tv/%N/{fn}" "animeFormat=anime/%N/{fn}" "movieFormat=movie/%N/{fn}" subtitles=n artwork=n "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L" "ut_state=%S"
PS: I also tried using clean=y but no avail,
filebot -script fn:utorrent-postprocess --output "F:/Downloaded/_automated/_postprocess" --action hardlink --conflict override -non-strict --def "seriesFormat=tv/%N/{fn}" "animeFormat=anime/%N/{fn}" "movieFormat=movie/%N/{fn}" subtitles=n artwork=n clean=y "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L" "ut_state=%S"
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 00:49
by rednoah
Yeah, in that case it'd have to be kinda a mix between copy and hardlink on a case by case basis. You can add that yourself. I won't add that to the default utorrent-postprocess though for the sake of simplicity.
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 08:23
by jkaberg
But shouldnt the clean variable remove the extracted content from the "seeded" dir automaticly, atleast thats how I interpreted it
--def clean=y Automatically empty folders and clutter files that may be left behind after moving the video files or temporary extracted files after copying
If thats not the cases, how would one proceed, eg what would need to be altered? Im guessting the extraction process but how do I specify it to extract to destination instead of the "seed dir" and then not hardlink itself in the destination dir (eg hardlink destionation to destinatioul as it would be)
Because as it is now, the only thing I really would need it to do is remove the extracted files in the seed dir (becasuse since they are hardlinked the other "linked file" in the destination dir will still exist and everything will be fine) after makeing the hardlink. And I've tried the clean variable several times, eg clean=y | clean=yes | clean=true with no success, it dosnt remove anything
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 09:43
by rednoah
Yep, that makes sense. Maybe I'll add it next time. Just look the end of the script. "clean" has different meanings depending on --action.
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 10:33
by jkaberg
Thanks for pointing that out! Have now edited the clean function as following
Code: Select all
if (clean) {
if ('COPY'.equalsIgnoreCase(_args.action) && extractedFiles?.size() > 0) {
println 'Clean temporary extracted files'
// delete extracted files
extractedFiles.each{
if(it.isFile()) {
println "Delete $it"
it.delete()
}
}
// delete remaining empty folders
extractedFiles*.dir.unique().findAll{ it.listFiles().length == 0 }.each{
if(it.isDirectory()) {
println "Delete $it"
it.delete()
}
}
}
if ('HARDLINK'.equalsIgnoreCase(_args.action) && extractedFiles?.size() > 0) {
println 'Clean temporary extracted files'
// delete extracted files
extractedFiles.each{
if(it.isFile()) {
println "Delete $it"
it.delete()
}
}
// delete remaining empty folders
extractedFiles*.dir.unique().findAll{ it.listFiles().length == 0 }.each{
if(it.isDirectory()) {
println "Delete $it"
it.delete()
}
}
}
// deleting remaining files only makes sense after moving files
if ('MOVE'.equalsIgnoreCase(_args.action)) {
println 'Clean clutter files and empty folders'
include('fn:cleaner', [:], !args.empty ? args : ut_kind == 'multi' && ut_dir ? [ut_dir as File] : [])
}
}
And it does indeed remove the extracted files in the seed directory, does hardlink but when I try to process the file in the "postprocess" folder (where its hardlinked or extracted to) I cant access the file without haveing to shutdown uTorrent, do you know why? If i search for the file in Process Explorer I can see that uTorrent is accessing the counterpart in my seed directory, but no process is accessing it in my "postprocess" folder :O
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 11:42
by rednoah
So it's extracted to to the current directory and then hardlinked to the new organized structure. And you can't remove the file from the downloads folder where the archives are as well?
In anycase I have no idea how Windows deals with file handles and hardlinks. Maybe adding a wait somewhere helps? Is the script failing or is it that later you can't access files?
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 13:58
by jkaberg
Appreciate your help and knowledge rednoah! The hardlinking problem was related to uTorrent, resolved with setting diskio.flush_files to false in uTorrent (advanced settings)
diskio.flush_files: Enabling this option causes µTorrent to close file handles every minute. It helps to reduce the effect of Windows managing the system cache badly for some people and causing apparent “memory leaks.”
Seems everything works as intended now

Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 17:31
by rednoah
Google just told me about your new thread on Couchpotato:
https://couchpota.to/forum/thread-1454-post-6270.html
You're completely abusing the utorrent-postprocess script:
* by setting label 'movie' you disable any anime/series/movie auto-detection
* your movie format --def movieFormat=movie/%N/{fn} is completely oblivious of whatever movie-detection comes up with. Now that's really just wasting TheMovieDB bandwith for no reason.
So you've disabled any and all processing... What's the point? I guess you need extracting and hardlinking, and nothing else.
e.g. After deleting almost everything from my utorrent-postprocess you should come up with something like this:
Code: Select all
filebot -script "C:/fbcpto.groovy" --output "C:/hl" --def "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L"
Code: Select all
def files = []
if (ut_kind == 'single') { files += new File(ut_dir, ut_file) }
if (ut_kind == 'multi') { files += new File(ut_dir).getFiles() }
// extract archives (zip, rar, etc) that contain at least one video file
def extractedFiles = extract(file: files.findAll{ it.isArchive() }, output: '_extract', conflict: 'override', filter: { it.isVideo() })
files += extractedFiles
files.findAll{ it.isVideo() }.each{ f ->
println net.sourceforge.filebot.StandardRenameAction.HARDLINK.rename(f, new File(_args.output, "${ut_label}/${ut_title}/${f.name}"))
}
// delete extracted files later
extractedFiles*.delete()
Re: Hardlinking with "Fully automated media center" script
Posted: 05 Feb 2013, 21:53
by jkaberg
Im sorry if I miss used your script rednoah, wasnt my intention to "void" anything. But my lack of knowledge in scripting got me to where I am with the "Fully automated media center script" (I've actully tried to do what you did with the script you posted one up)
Anyhow I'll edit my post at couchpotato forums with credits towards you
Thanks again
Re: Hardlinking with "Fully automated media center" script
Posted: 06 Feb 2013, 01:35
by rednoah
Yeah, the one I posted should to pretty much exactly what you want. Except in alot less lines of code (10 lines instead of 300).
Let me know if you have any issues.
Re: Hardlinking with "Fully automated media center" script
Posted: 06 Feb 2013, 06:24
by jkaberg
Yeah there was one thing I would wish to add, did I get this right? I would like the script to quit if the label isnt movie or tv
Code: Select all
if (ut_label != 'movie' || ut_label != 'tv' ) { return null }
eg full code
Code: Select all
// Author of this script is Rednoah at Filebot Forums, http://filebot.sourceforge.net
// Usage: filebot -script "C:/Downloaders/fbcpto.groovy" --output "C:/Downloaded/_postprocess" --def "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L"
def files = []
if (ut_label != 'movie' || ut_label != 'tv' ) { return null }
if (ut_kind == 'single') { files += new File(ut_dir, ut_file) }
if (ut_kind == 'multi') { files += new File(ut_dir).getFiles() }
// extract archives (zip, rar, etc) that contain at least one video file
def extractedFiles = extract(file: files.findAll{ it.isArchive() }, output: '_extract', conflict: 'override', filter: { it.isVideo() })
files += extractedFiles
files.findAll{ it.isVideo() }.each{ f ->
println net.sourceforge.filebot.StandardRenameAction.HARDLINK.rename(f, new File(_args.output, "${ut_label}/${ut_title}/${f.name}"))
}
// delete extracted files later
extractedFiles*.delete()
Re: Hardlinking with "Fully automated media center" script
Posted: 06 Feb 2013, 10:39
by rednoah
Yes, that'll work. Though I'd make it crash with an exception so the process returns with an error exit code. Also fixed your boolean logic as you came up with an tautology
Code: Select all
if (ut_label != 'movie' && ut_label != 'tv' ) {
throw new Exception("Invalid label: $ut_label")
}
Re: Hardlinking with "Fully automated media center" script
Posted: 06 Feb 2013, 14:12
by jkaberg
Thanks! Updated Couchpotato post accordingly
This will be very good for Couchpotato and torrents

Re: Hardlinking with "Fully automated media center" script
Posted: 06 Feb 2013, 15:29
by jkaberg
Ok so one more hehe, seems I cant bother you enough eh?
After I downloaded a few movies to test it, I found myself in need of filtering out the "sample/trailer" media files as I dont want them and the script was hardlinking thoes when it had extracted a movie
This is my attempt to fix it (by borrowing code from utorrent processor)
Code: Select all
// Author of this script is Rednoah at Filebot Forums, http://filebot.sourceforge.net
// Usage: filebot -script "C:/Downloaders/fbcpto.groovy" --output "C:/Downloaded/_postprocess" --def "ut_dir=%D" "ut_file=%F" "ut_kind=%K" "ut_title=%N" "ut_label=%L"
def files = []
if (ut_label != 'movie' && ut_label != 'tv' ) {
throw new Exception("Invalid label: $ut_label")
}
if (ut_kind == 'single') { files += new File(ut_dir, ut_file) }
if (ut_kind == 'multi') { files += new File(ut_dir).getFiles() }
// flatten nested file structure
files = files.flatten()
// extract archives (zip, rar, etc) that contain at least one video file
def extractedFiles = extract(file: files.findAll{ it.isArchive() }, output: '_extract', conflict: 'override', filter: { it.isVideo() })
files += extractedFiles
// sanitize input
files = files.findAll{ it?.exists() }.collect{ it.canonicalFile }.unique()
// ignore clutter files
files = files.findAll{ !(it.path =~ /\b(?i:sample|trailer|extras|deleted.scenes|music.video|scrapbook|behind.the.scenes)\b/) }
// print input fileset
files.each{ f -> _log.finest("Input: $f") }
// process only media files
files.findAll{ it.isVideo() }.each{ f ->
println net.sourceforge.filebot.StandardRenameAction.HARDLINK.rename(f, new File(_args.output, "${ut_label}/${ut_title}/${f.name}"))
}
// delete extracted files later
extractedFiles*.delete()
But it didnt fly to well (read: the script did nothing

), can you spot my error?

Re: Hardlinking with "Fully automated media center" script
Posted: 06 Feb 2013, 15:54
by rednoah
There was some bug with forceExtractAll. Must be set to true as a workaround for now:
Code: Select all
def extractedFiles = extract(file: files.findAll{ it.isArchive() }, output: '_extract', conflict: 'override', filter: { it.isVideo() }, forceExtractAll: true)
Re: Hardlinking with "Fully automated media center" script
Posted: 11 Feb 2013, 17:21
by jkaberg
Thanks red, I hope you would like to awnser one more

So the file permission issue is still there but I've found a program which unlockes "locked" files
So I made this but it dosnt work very well, maybe you could point me in the right direction?
Code: Select all
// process only media files
files.findAll{ it.isVideo() }.each{ f ->
outputThis = _args.output "${ut_label}/${ut_title}/${f.name}"
//adding function to unlock files locked by uTorrent
outputCatcher = new StringBuffer()
errorCatcher = new StringBuffer()
proc = "C:\Program Files (x86)\Unlocker\Unlocker.exe " outputThis " -s".execute()
proc.consumeProcessOutput(outputCatcher, errorCatcher)
proc.waitFor()
println net.sourceforge.filebot.StandardRenameAction.HARDLINK.rename(f, new File(outputThis))
println outputCatcher
}
Thanks again
Re: Hardlinking with "Fully automated media center" script
Posted: 11 Feb 2013, 17:36
by rednoah
Actually I added some extras with FileBot for that:
Code: Select all
execute("C:\Program Files (x86)\Unlocker\Unlocker.exe", outputThis, "-s")
Though I guess this point outputThis as not been created. Do you mean f instead?
Also I think you mean this:
Code: Select all
outputThis = new File(_args.output, "${ut_label}/${ut_title}/${f.name}")
Re: Hardlinking with "Fully automated media center" script
Posted: 11 Feb 2013, 21:42
by jkaberg
rednoah wrote:Actually I added some extras with FileBot for that:
Code: Select all
execute("C:\Program Files (x86)\Unlocker\Unlocker.exe", outputThis, "-s")
Though I guess this point outputThis as not been created. Do you mean f instead?
Also I think you mean this:
Code: Select all
outputThis = new File(_args.output, "${ut_label}/${ut_title}/${f.name}")
Urr, Im not entirely sure about f
This is what I got
Code: Select all
// process only media files
files.findAll{ it.isVideo() }.each{ f ->
outputThis = new File(_args.output, "${ut_label}/${ut_title}/${f.name}")
println net.sourceforge.filebot.StandardRenameAction.HARDLINK.rename(f, outputThis)
execute("C:\Program Files (x86)\Unlocker\Unlocker.exe", outputThis, "-s")
}
But I failed since this dosnt work yet.. What I want it to do is "unlock" every file that the script trys to hardlink (because afterwards a postprocessor will try and move/rename the files in the postprocess directory)
An normal "unlock" would be like
Code: Select all
C:\Program Files (x86)\Unlocker\Unlocker.exe F:\Downloaded\_automated\_postprocess\movie\A.Royal.Affair.2012.1080p.BluRay.x264-PFA\A.Royal.Affair.2012.1080p.BluRay.x264-PFA.mkv -s
Re: Hardlinking with "Fully automated media center" script
Posted: 12 Feb 2013, 04:39
by rednoah
Ah, so first you create a hardlink and then you have other scripts moving those hardlinks? Then first hardlink and then run the unlocker.
Also you can't use \ backslash in "" string expression. Try:
Code: Select all
"C:/Program Files (x86)/Unlocker/Unlocker.exe"