Fully Automated Download Center

Running FileBot from the console, Groovy scripting, shell scripts, etc
Post Reply
cheesemaker
Posts: 49
Joined: 03 Sep 2012, 10:52

Fully Automated Download Center

Post by cheesemaker »

Here's my way how my automation media center works. That's not a tutorial - just an inspiration for you - for those who aren't downloading by torrent (OCH).

Tools I use:
pyLoad - Download Manager
flexget - fetches the newest series and movies
filebot - we all know this tool - renamer

So flexget fetches the newest movies and series from my favourite site. It's important that this site has an rss-feed with the downloads links in the description. But that's really rare. So mostly you have to create your own RSS-Feed of your favourite site.
You can do that with a service named Feed43.com, this will create your own RSS-Feed of your favourite page. (For mor information about flexget and how it works flexget)

With a good configuration, flexget will check sequentally for new entries and add them - if their fits to your interest - to your pyload download manager.

My problem with filebot was always these release groups. There just to many of them. So I decided to use the name, which I named the package in Pyload. For Example: This.Is.A.Great.Movie.2012.BluRay.1080p.x264.DTS-RealeaseGroup (Flexget fetched automatically this name with the download links)

After pyLoad downloaded all data of a movie, it'll move them into a folder "extracted" (Pyload Configuration) and automatically run my filebot script (Plugin: ExternalScripts)

For Renaming and Sorting I use the script Fully Automated Media Center from rednoah.

I just changed two important things (for me):
This will ad the language ending to the downloaded subtiltes and also download other languages. For Example:
This.Is.A.Great.Movie.2012.BluRay.1080p.x264.DTS-RealeaseGroup.eng.srt

Code: Select all

	// fetch subtitles
	if (subtitles) {
		files += getMissingSubtitles(file:files, output:"srt", encoding:"utf-8", lang:"en")
		files += getMissingSubtitles(file:files, output:"srt", encoding:"utf-8", lang:"de")
	}
And I changed the format pattern. It will name the movie just like the folder's name - the name flexget added to pyload. (This.Is.A.Great.Movie.2012.BluRay.1080p.x264.DTS-RealeaseGroup). The job of the filebot scripts is to decide if it is a move or an episode. If it's a movie it will sort it in to the movie folder, download the subtitles, but keep the real name without problems matching the release group. If it's an episode it will do "the old filebot way".

Code: Select all

mov:   '''Filme/{n.upperInitial().space('.')}.{y}/{folder.name}{".$lang"}
If you have questions, feel free to ask.
cheesemaker
Scorp1on
Posts: 6
Joined: 03 Dec 2012, 13:32

Re: Fully Automated Download Center

Post by Scorp1on »

Hello everyone,

after switching from WindowsHomeServer to an Synology Nas due to a huge dataloss, i came along this nice piece of software.
And your "inspiration" is exactly what i´m looking for.
After a few days messing around i finally got pyload und filebot working on my Nas(it quite seems so, what i can tell so far :D )
I´ve already tried a few renaming and moving Tasks, to get known of how Filebot works.
But i cant get the powerful Skript from rednoah to work like i need it for my personal use :)

Here are the Things that i´d liked to alter :

1.naming strukture of Movies and Episodes:
Episodes should look like this : /Serien/Stargate Sg1/Staffel 1/Stargate SG-1_[s01]_[e01]_Das Tor zum Universum (1).avi
i tried it and "{n}_[{'s'+s.pad(2)}]_[{'e'+e.pad(2)}]_{t}" should work

Movies should look like this: /Filme/Aviator (2004) [Blu-ray]/Aviator (2004) [180p_DTS 5.1].mkv
"{n} {(y)} {[resolution audio.format]}"??

2.the place where Media is stored (instead of X:Media ->/volume1/Mediathek)
and the Subfolder should be : Filme for Movies, Serien(and Staffel instead of Season) for tv Shows (Anime is ok because it´s the Same in German ;) )

Also i have a few Questions

A. Im using Xbmc on a few Clients. Is it possible to tell one Machine to wake up for scanning the newly added files ?
or is it better to use plex which has a Server to scan the files locally on my nas ?
B. Is it somehow possible to order the Movies to Subfolders for SD, HD and 3D?
C. and whats about extracting. Should pyload or the Skript doing this job ?
D. Also is it possible to recognize Movie Collections somehow ?

Flexget seems really interesting, i will look at it when the "manually downloading+automated renaming/moving" Thing is working for me :)


I´m really greatful for all help i can get :)
and sorry for my bad english :roll: its because i´m a "german guy from hinterwäldle" :D
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Fully Automated Download Center

Post by rednoah »

You probably won't even need to run your own mod of this script. You should be able to specify everything via cmdline arguments.

1 & 2. Check the docs again. Added a new section for that: "Customize Paths". The rest is all in the format.

A. You mean Wake-on-LAN? No idea how that works. Otherwise you can just specify the xmbc/plex hostnames and the script will send API calls telling each instance to rescan it's library.

B. Yep, all in the format. You can use {sdhd} binding. Not sure about 3D, you can probably match that from the filename with {fn.match(/3D/)} or using the generic MediaInfo bindings {media} or {video}.

C. FileBot's 7zip native code doesn't work on Synology so you're gonna have to go with pyload.

D. {collection} binding. Just use it in your format wherever you want.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Fully Automated Download Center

Post by rednoah »

Movie Format:

Code: Select all

Filme/{n} ({y}){" [$source]"}/{n} ({y}) [{"${fn.match(/3D/)}, "}{vf}, {ac}, {af}]
Episode Format:

Code: Select all

Serien/{n}/Staffel {s}/{n}_[s{s.pad(2)}]_[e{e.pad(2)}]_{t}
(does not take double episodes into account for the sake of simplicity)
:idea: Please read the FAQ and How to Request Help.
Scorp1on
Posts: 6
Joined: 03 Dec 2012, 13:32

Re: Fully Automated Download Center

Post by Scorp1on »

first of all thanks for your reply :)

yesterday i tried some format expressions to get more comfortable with renaming.
My formats where almost exactly the same as your recommendations. Thats Great because than im finally getting into this :D

by testing around with the formats i´ve encountered a problem which is related to mediainfo on Atom based Synologysystesms :?
but more here :http://filebot.sourceforge.net/forums/v ... 2615#p2615

Ive also tried to use your script only via cmdline arguments.But i guess here`s something wrong. :?

regarding to cheesmaster i have to tell pyload to run this script after the downloads are finished.
Therefore Pylod has an Plugin called ExternalScripts.
That Plugin offers some Folders like (unrar finished) or (after download) and so on.
So ,as far as i can tell, i must place your modified script in one of that folders??

Ok now back to my Questions :roll:

A: Yes i ment WOL. I do not exactly know what API calls are, but if the next time i´m starting a xbmc client, xbmc will rescan my library, so im totally happy with it :D

B: Ok, i´m guessing i get this now :D

C:Ok im fine with that but i have to get mediainfo working for my format expressions :(

D: I would like to have an extra Folder with the Collection Name! when i´m using this, what is happening to the Movies without a Collection :?:

thanks for help in advance :D
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Fully Automated Download Center

Post by rednoah »

Can't help with with the compiling mediainfo stuff. I'm not really an expert with that and I don't even have the hardware to try anything.

What do you mean with putting the script in a folder? If you want filebot to grab the script from use -script fn:name, if you're running your own script you just do -script /usr/me/myscript.groovy, but since you can configure everything with passing cmdline args you'll be fine with -script fn:utorrent-postprocess.

FileBot uses the XBMC JSON-RPC API via TCP. Just some communication between two programs. If your XBMC client is shutdown FileBot won't be able to talk to it.
http://wiki.xbmc.org/index.php?title=JS ... g_JSON-RPC

Add a {collection} folder if the movie is part of an collection:

Code: Select all

Filme/{"$collection/"}{n} ({y}){" [$source]"}/{n} ({y}) [{"${fn.match(/3D/)}, "}{vf}, {ac}, {af}]
:idea: Please read the FAQ and How to Request Help.
cheesemaker
Posts: 49
Joined: 03 Sep 2012, 10:52

Re: Fully Automated Download Center

Post by cheesemaker »

@rednoah,
hat do you mean with putting the script in a folder?
I think he means the PyLoad "ExternalScripts" Plugin. By activating this script, you're allowed to execute scripts, for example after a finished package or after sucessfull unrar.

@Scorp1on
Hallo,
Das automatisierte Einsortieren der Downloads funktioniert inzwischen gut, jedoch leider immer noch nicht perfekt. Bin zurzeit immer noch auf Lösungssuche. Zurzeit extrahiere ich die Dateien mit PyLoad (ExtractScripts), jedoch funktionert dies leider nicht immer, da falls du bei einem Download zwei verschiedene Hoster hast und einer fehlschlägt, sind zwar alle Parts heruntergeladen aber es wird nicht entpackt.
Wahrscheinlich werde ich dies in naher Zukunft dies per Bash-Skript erledigen, im Moment fehlt mir jedoch noch die Zeit.
Nach jedem Entpacken führt PyLoad mit "ExternalScripts" automatisch das FilebotScript aus, welches das Sortieren einleitet.
Jedoch bin ich noch immer nicht zufrieden, mein Anspruch ist nämlich, dass der Realeasename vollständig erhalten bleibt. Deshalb benenne ich jeden Download (package) in Pyload mit dem Namen, der die Datei später tragen soll:
z.B. A.Great.Movie.1080p.BluRay.x264-ReleaseGroup
Dies funktioniert aber bei mehrfach gepackten Archiven nicht immer.

lg, cheesemaker
Scorp1on
Posts: 6
Joined: 03 Dec 2012, 13:32

Re: Fully Automated Download Center

Post by Scorp1on »

rednoah wrote:Can't help with with the compiling mediainfo stuff. I'm not really an expert with that and I don't even have the hardware to try anything.
Thats to bad, guess i must live with "fn.match" so.

Yeah i ment the Pyload Plugin. Think cheesemaker could help me here ;)

@cheesemaker
das hört sich schonmal gut an. Ich selbst nutze eig meistens nur 1 Hoster. Also wäre dass mit dem Entpacken bei mir wohl hinfällig :)
Leider bin ich gerade noch ziemlich überfordert mit dem Skripting :oops: Verstehe nicht ganz wie ich Pyload erzähle dass nun Filebot die Daten verschieben und umbenennen soll.
Erschwerlicherweiße kommt noch hinzu dass ich Mediainfo einfach nicht zum laufen bringe:(
Falls du mal etwas Zeit hast würde ich mich rießig über ein How To freuen. Denke dass dies viele Leute mit einem Nas erfreulich stimmen würde 8-)

nur rein interesehalber: wiso willst du den Releasename behalten, ich finde den meistens viel zu unübersichtlich ;)
cheesemaker
Posts: 49
Joined: 03 Sep 2012, 10:52

Re: Fully Automated Download Center

Post by cheesemaker »

Also das ExternalScript-Plugin ist nicht schwer. In:
/home/$USER/.pyload/scripts/$PARAMETER
oder /usr/share/pyload/scripts/$PARAMETER

$PARAMETER steht für den Zeitpunkt, wann du das Skript ausführen möchtest. (z.B. all_dls_processed, nachdem alle Downloads beendet wurden)
Dort erstellst du, unter Linux mit "touch DeinSkriptName.sh" die Datei "DeinSkriptName.sh". In diese kannst du nun mit Bash deine Befehle eintragen. (vi Editor: "vi DeinSkriptName.sh"; nano: "nano DeinSkripName.sh")
Wichtig ist, dass du zu begin "#!/bin/sh" einfügst, sonst kann es zu Problemen kommen.
Also zum Beispiel das Skript "test.sh"

Code: Select all

#!/bin/sh
filebot -rename "$DEINDOWNLOADORDNER" --format="{t}"
das mit dem mediainfo muss ich noch anschauen, kenne ich mich nicht aus. Ahja, der Releasename ist mir wichtig, da ansonsten die Information verloren geht. Manchmal gibt es Releases, bei denen zum Beispiel das Audio Async ist, dann möchte ich ja nicht nochmal denselben herunterladen. Auch wenn du mehrere Releases behalten möchtest. Und noch wichtiger: Bei der Subtiltesuche spielt der Releasename eine grosse Rolle. Unübersichtlich finde ich wird es dadurch nicht. Meine Mediathek ist so aufgebaut:

--A.Great.Movie.2012
--/--A.Great.Movie.2012.German.1080p.BluRay.x264-ReleaseGroup.mkv
--/--A.Great.Movie.2012.German.1080p.BluRay.x264-ReleaseGroup.eng.srt
--/--A.Great.Movie.2012.German.1080p.BluRay.x264-ReleaseGroup.deu.srt

lg cheesemaker

// Edit:
Ich verstehe deine Frage zu Mediainfo nicht ganz. Wofür benötigst du es überhaupt?
isadic
Posts: 3
Joined: 25 Apr 2013, 12:17

Re: Fully Automated Download Center

Post by isadic »

cheesemaker wrote:
My problem with filebot was always these release groups. There just to many of them. So I decided to use the name, which I named the package in Pyload. For Example: This.Is.A.Great.Movie.2012.BluRay.1080p.x264.DTS-RealeaseGroup (Flexget fetched automatically this name with the download links)

After pyLoad downloaded all data of a movie, it'll move them into a folder "extracted" (Pyload Configuration) and automatically run my filebot script (Plugin: ExternalScripts)

For Renaming and Sorting I use the script Fully Automated Media Center from rednoah.


If you have questions, feel free to ask.
cheesemaker
Can you explain me how to get the name from pyload? In rednoah's script is %D, for utorrent. Here?
Thanks in advantage.
cheesemaker
Posts: 49
Joined: 03 Sep 2012, 10:52

Re: Fully Automated Download Center

Post by cheesemaker »

isadic wrote: Can you explain me how to get the name from pyload? In rednoah's script is %D, for utorrent. Here?
Thanks in advantage.
Hi isadic,

That's not that easy. I think you're using the "external script" plugin. Pyload gives some parameters to the script when executing that script. (These you can access by storing the values $1, $2, $3, ... I think I had found this parameters somewhere, but I can't find it anymore. You can find out the given parameters by executing each script, which prints you out the variables in a textfile (for each "unrar_finished", "download_finished", etc.)

I have only the parameters for unrar_finished and download_finished.

unrar_finished:

Code: Select all

Scriptname=$0 #scripts/download_finished/download_finished.sh
Hoster=$1 #UploadedTo
DownloadLink=$2 #http://uploaded.net/file/asdfasdfasdf
Archive=$3 #archivname.rar
FullPath=$4  #/mnt/downloads/pyload/paketname/archivname.rar

PackageName=$( echo "$FullPath" | awk -F'/' '{print $5}')
download_finished:

Code: Select all

Skriptname=$0
PfadPacketname=$1 #/mnt/downloads/extracted/ArchiVName
PfadArchiv=$2 #/mnt/downloads/pyload/ArchiVName/archivname.rar

Packetname=$( echo "$PfadPacketname" | awk -F'/' '{print $5}')
best regards,
cheesemaker
Post Reply