[Hire Request] Rtorrent + Filebot/AMC

[Joon]

Hello,

First of all, i have no idea if this is at all feasible for me 'cost' wise, as is the same in terms of difficulty to do this.
Im looking for an AMC solution for my linux box that currently runs rtorrent/rutorrent on a remote server.
Most of the files will require unpacking after downloading and sorting like AMC currently does into respective directories.
Rutorrent also has a webui possibility which is ofc completely optional in this case, autoupdating or easy updating would be preferred though.

I have found an existing postprocess script that calls most required parameters from rtorrent which filebot would need, which can be found here

Obviously i'd like any possible solution to this to be available to anyone and i am willing to pay for this, but i'd like some .. indication of this first

Thanks in advance.

[rednoah]

You'll just need to replace:

Code: Select all

system.method.set_key=event.download.finished,my_script,"execute={python,/path/to/rprocess/rProcess.py,$d.get_hash=}"

And there's some docs on those rtorrent variables:
https://github.com/cjlucas/rtorrent-pyt ... -Reference

Not sure if all the variables are available though in this context. The example only passes the torrent hash, but if you can also pass download location etc you can make it work.

This is basically grunt work, figuring out rtorrent, but it might be easy, so I'll set the bounty at 50€

[Joon]

Okay, i have been looking for the page you just linked for quite a while, i just didnt know what to search for i guess
Hence when i found the script i thought yes i found something, im thinking it fetches the information from the torrent file as the filenames and stuff are in the torrent file too.

Still as a novice in linux and coding, i wish it was an easy task for me.

If you can help me set it up after you've figured it out you have a deal

[rednoah]

Tried some stuff...

rtorrent.rc

Code: Select all

system.method.set_key=event.download.finished,my_script,"execute={/home/rednoah/app/filebot/filebot,-script,fn:sysenv,--log-file,rtorrent.log,$d.get_base_path=,$d.get_base_filename=,$d.get_custom1=,$d.get_custom=,$d.get_name=}"

log:

Code: Select all

args[0] = -script
args[1] = fn:sysenv
args[2] = --log-file
args[3] = rtorrent.log
args[4] = /home/rednoah/files/Revolution.2012.S02E20.HDTV.x264-LOL.mp4
args[5] = Revolution.2012.S02E20.HDTV.x264-LOL.mp4
args[6] = New%20Episodes
args[7] = 
args[8] = Revolution.2012.S02E20.HDTV.x264-LOL.mp4

Based on that we can create the following. Though untested it should work.


rtorrent-postprocess

Code: Select all

#!/bin/bash
TORRENT_PATH=$1
TORRENT_NAME=$2
TORRENT_LABEL=$3

filebot -script fn:amc --output "$HOME/Videos" --log-file amc.log --action copy --conflict override -non-strict --def "ut_dir=$TORRENT_PATH" "ut_kind=multi" "ut_title=$TORRENT_NAME" "ut_label=$TORRENT_LABEL"

Code: Select all

system.method.set_key=event.download.finished,filebot_amc,"execute={/usr/bin/rtorrent-postprocess,$d.get_base_path=,$d.get_name=,$d.get_custom1=}"

[Joon]

Okay, great!

I will fiddle with this later today
One question though, is it possible to ignore files based on label ?

[rednoah]

Sure, but you might need to adjust the code. Here's the default logic:

Code: Select all

def forceIgnore = { f ->
	label =~ /^(?i:ebook|other|ignore)/ || f.path =~ ignore
}

[Joon]

Okay, first attempt here:

Code: Select all

May 10, 2014 1:57:20 PM net.sourceforge.filebot.Main initializeCache
WARNING: java.io.FileNotFoundException: /home/0x00/.filebot/cache/0/.lock (Permission denied)
java.io.FileNotFoundException: /home/0x00/.filebot/cache/0/.lock (Permission denied)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
	at net.sourceforge.filebot.Main.initializeCache(Main.java:444)
	at net.sourceforge.filebot.Main.main(Main.java:142)

May 10, 2014 1:57:20 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Parameter: ut_dir = /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION
Parameter: ut_kind = multi
Parameter: ut_title = Hannibal.S02E11.720p.HDTV.X264-DIMENSION
Parameter: ut_label = TV
Read archive [hannibal.211.720p-dimension.rar] and extract to [/home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension/Hannibal.S02E11.720p.HDTV.X264-DIMENSION]
Extracting files [/home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/Hannibal.S02E11.720p.HDTV.X264-DIMENSION.mkv]
JNA Warning: IOException removing temporary files: JNA temporary directory '/home/0x00/.filebot/temp' does not exist
May 10, 2014 1:57:40 PM net.sourceforge.filebot.mediainfo.MediaInfo <clinit>
WARNING: Failed to preload libzen
Input: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/Hannibal.S02E11.720p.HDTV.X264-DIMENSION.mkv
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r01
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r04
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.sfv
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r03
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r09
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r11
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r07
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r12
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r15
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r08
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/Sample/hannibal.211.720p-dimension.sample.mkv
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r06
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r05
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r16
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r13
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r02
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r14
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r10
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.r00
Exclude: /home/0x00/downloads/TV/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension.nfo
Group: [tvs:hannibal] => [Hannibal.S02E11.720p.HDTV.X264-DIMENSION.mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Hannibal]
CacheException: Disk store path can't be created: /home/0x00/.filebot/cache/default
Done ヾ(＠⌒ー⌒＠)ノ

It extracted it in

Code: Select all

Hannibal.S02E11.720p.HDTV.X264-DIMENSION/hannibal.211.720p-dimension/Hannibal.S02E11.720p.HDTV.X264-DIMENSION/

without renaming it.

it looks like it does not have access to the directories ?

edit:
i set the directories to 755, still the same error.
Installed JRE 7 (1.0.7.55) and set it as default and still the same error.

Edit2:
fixed it, will continue testing

Sure, but you might need to adjust the code. Here's the default logic:

Code: Select all

def forceIgnore = { f ->
	label =~ /^(?i:ebook|other|ignore)/ || f.path =~ ignore
}

Nice, i put this in the rtorrent-postprocess file(with the labels i want ignored), but it seems to be blocking everything instead of specific labels ?

like this:

Code: Select all

def forceIgnore = { f ->
	label =~ /^(?i:Games|Seeding)/ || f.path =~ ignore
}

Also, is there a convenient way to update the filebot.jar on linux ? i installed filebot with the .deb file

[rednoah]

If you set the label to "ignore" in rtorrent webui, then this label will be passed into amc as ut_label making it ignore every single file of the given batch.

[Joon]

If you set the label to "ignore" in rtorrent webui, then this label will be passed into amc as ut_label making it ignore every single file of the given batch.

Im missing something here, do i or do i not use the piece of code you gave me ?
If so, where do i put it ?

if not, using 'Ignore' as label on every torrent i want ignored by filebot defeats the purpose of labeling.

[rednoah]

Then you can either edit the amc script yourself or used the latest dev:amc where I added Games|Seeding to the ignore pattern.

You don't use it, you search for it, so you see where in the amc script this is handled, so you know where to edit.

[Joon]

Then you can either edit the amc script yourself or used the latest dev:amc where I added Games|Seeding to the ignore pattern.

You don't use it, you search for it, so you see where in the amc script this is handled, so you know where to edit.

Oooh, my bad totally misunderstood you, apologies!

i'll check it out!

[Joon]

Okay Rednoah, got the first quirky thing for you..

Code: Select all

Parameter: ut_dir = /home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv
Parameter: ut_kind = multi
Parameter: ut_title = [HorribleSubs] Nisekoi - 18 [720p].mkv
Parameter: ut_label = Anime
Input: /home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv
Group: [tvs:nisekoi] => [[HorribleSubs] Nisekoi - 18 [720p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Nisekoi]
Fetching episode data for [Nisekoi]
[COPY] Rename [/home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv] to [/home/0x00/downloads/TV Shows/Nisekoi/Season 01/Nisekoi - S01E18 - Episode 18.mkv]
Processed 1 files
Done ヾ(＠⌒ー⌒＠)ノ

It seems its not detecting that its anime from either the directory or the label.
Anything i can do from my end to improve this ?

I am using the base release of filebot from the frontpage for linux, i wouldnt know where to start in updating it.
Also, if it is at all usefull, i have the linux version of MediaInfo installed, as well as the rutorrent plugin for MediaInfo.

[rednoah]

Yeah, I guess you'll need to use the latest 4.1 RC3 with -script dev:amc. Just grab the portable, extract it somewhere, and run with filebot.sh

@see http://www.filebot.net/forums/viewtopic.php?f=7&t=1609

[Joon]

Yeah, I guess you'll need to use the latest 4.1 RC3 with -script dev:amc. Just grab the portable, extract it somewhere, and run with filebot.sh

@see http://www.filebot.net/forums/viewtopic.php?f=7&t=1609

okay, im using 4.1 RC3 and modified it to dev:amc in the postprocess file, however its the exact same result

Code: Select all

Parameter: ut_dir = /home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv
Parameter: ut_kind = multi
Parameter: ut_title = [HorribleSubs] Nisekoi - 18 [720p].mkv
Parameter: ut_label = Anime
Input: /home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv
Group: [tvs:nisekoi] => [[HorribleSubs] Nisekoi - 18 [720p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Nisekoi]
Fetching episode data for [Nisekoi]
[COPY] Rename [/home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv] to [/home/0x00/downloads/TV Shows/Nisekoi/Season 01/Nisekoi - S01E18 - At the Beach.mkv]
Processed 1 files
Done ヾ(＠⌒ー⌒＠)ノ
Parameter: ut_dir = /home/0x00/downloads/Seeding/Anime/Mekakucity Actors/[HorribleSubs] Mekakucity Actors - 05 [720p].mkv
Parameter: ut_kind = multi
Parameter: ut_title = [HorribleSubs] Mekakucity Actors - 05 [720p].mkv
Parameter: ut_label = Anime
Input: /home/0x00/downloads/Seeding/Anime/Mekakucity Actors/[HorribleSubs] Mekakucity Actors - 05 [720p].mkv
Group: [tvs:mekaku city actors] => [[HorribleSubs] Mekakucity Actors - 05 [720p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Mekaku City Actors, Mekakucity Actors]
Fetching episode data for [Mekaku City Actors]
[COPY] Rename [/home/0x00/downloads/Seeding/Anime/Mekakucity Actors/[HorribleSubs] Mekakucity Actors - 05 [720p].mkv] to [/home/0x00/downloads/TV Shows/Mekaku City Actors/Season 01/Mekaku City Actors - S01E05 - Kaien Panzermast.mkv]

[rednoah]

Did you previously rename the file with TheTVDB? It's probably tagged. These logs can't happen if it's a clean file.

If you're testing things on the same file you need to reset xattr before each run:

Code: Select all

filebot -script fn:xattr --action clear /path/to/files

@see http://www.filebot.net/forums/viewtopic ... &t=5#p5394

[Joon]

Did you previously rename the file with TheTVDB? It's probably tagged. These logs can't happen if it's a clean file.

If you're testing things on the same file you need to reset xattr before each run:

Code: Select all

filebot -script fn:xattr --action clear /path/to/files

@see http://www.filebot.net/forums/viewtopic ... &t=5#p5394

Previously i did test other files to see if the process worked these were indeed TV Shows
But the anime files arent renamed files, these are fresh downloads..
Every time i tested i made sure to remove the file present and just redownloaded it, for example 'mekakucity actors' was autofetched via RSS and was not downloaded before that, as you can see in the log it was still renamed with thetvdb.

So this is just confusing me .. a lot

can i prevent this from happening? resetting the xattr every single episode seems ..inconvenient

Edit: okay to test this i did the following:
First i made sure permissions were set correctly, to go to the extreme i set this specific folder to 777 as well as the file for testing purposes.

Code: Select all

0x00@0x00:~/downloads/Seeding/Anime/Nisekoi$ filebot -script fn:xattr --action clear \[HorribleSubs\]\ Nisekoi\ -\ 18\ \[720p\].mkv 
Done ?(?????)?

not sure if its correct, only giving it the directory doesnt give any feedback either.

After this i executed filebot for this file manually which was

Code: Select all

filebot -script fn:amc --output “/” --log-file amc.log --action copy --conflict override -non-strict --def "ut_dir=/home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv" "ut_kind=multi" "ut_title=[HorribleSubs] Nisekoi - 18 [720p].mkv" "ut_label=Anime”

and the outcome was
(dont mind the error here, probably because the path is set to /)

Code: Select all

Locking /home/0x00/.filebot/logs/amc.log
Parameter: ut_dir = /home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv
Parameter: ut_kind = multi
Parameter: ut_title = [HorribleSubs] Nisekoi - 18 [720p].mkv
Parameter: ut_label = Anime
Input: /home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv
Group: [tvs:nisekoi] => [[HorribleSubs] Nisekoi - 18 [720p].mkv]
Rename episodes using [TheTVDB]
Auto-detected query: [Nisekoi]
Fetching episode data for [Nisekoi]
[COPY] Rename [/home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv] to [/TV Shows/Nisekoi/Season 01/Nisekoi - S01E18 - At the Beach.mkv]
[COPY] Failed to rename [/home/0x00/downloads/Seeding/Anime/Nisekoi/[HorribleSubs] Nisekoi - 18 [720p].mkv]
java.io.IOException: Failed to create folder: /TV Shows/Nisekoi/Season 01
Finished without processing any files
Failure (°_°)

[rednoah]

Haha, now I see! You have a 0x00 in the path, so amc thinks it should apply TV Mode, kinda like if you had 1x01 in the filename. Just use a normal username, not something weird.

[Joon]

lmao, sharp one
So uhm, yeah .. i feel rather silly now ^_^

will fix! thanks

[Joon]

Seems to be working!

One question though, how does this process handle simultaneous requests ? because my box has an extremely fast connection.
For example 2 packed torrents request the postprocess script, will filebot queue the second ? or does it multithread?

(btw: you should be somewhat richer by now)

[rednoah]

The file lock on the lock file makes sure there is only one active filebot instance at any given time:

Code: Select all

Locking /home/0x00/.filebot/logs/amc.log

So the operating system will queue all the filebot instances that may be called in parallel.

PS: I had my bank pay it out in cents and now I'm having a pool party

[Joon]

Well so far everything seems to be working as it should, if i find anything i will report back here.

Feel free to update the AMC with rtorrent, im sure there are some that will find it useful
Thank you for your effort and patience!

[rednoah]

Can you send me the commands that you're using now? Just so I post something that is confirmed to work.

[Joon]

This chmodded +rx :

rtorrent-postprocess

Code: Select all

#!/bin/bash
TORRENT_PATH=$1
TORRENT_NAME=$2
TORRENT_LABEL=$3

filebot -script dev:amc --output "$HOME/downloads" --log-file "$HOME/amc.log" --action copy --conflict override -non-strict --def "clean=y" "ut_dir=$TORRENT_PATH" "ut_kind=multi" "ut_title=$TORRENT_NAME" "ut_label=$TORRENT_LABEL"

and this one at the bottom of .rtorrent.rc (located in the rtorrent user home directory)

Code: Select all

system.method.set_key=event.download.finished,filebot_amc,"execute={/usr/bin/rtorrent-postprocess,$d.get_base_path=,$d.get_name=,$d.get_custom1=}"

[Joon]

I have found one issue which im not sure if it can be helped..
When filebot processes a packed file, rtorrent and rutorrent both hang, if anything is supposed to be picked up by autodl during that time, it wont.
Last night series1 and series2 were released a few second apart, it picked up series1 which finished probably around the time that series2 was released and in turn did not pick up series2 because everything is unresponsive at the time of 'processing'

[rednoah]

No idea how rtorrent processes the event.download.finished event.

Have tried doing command &?
http://www.cyberciti.biz/faq/linux-comm ... ackground/