JDownloader ore Filebot?

[Crack00r]

Hi, 1 my Setup

Gigabit Internet
Ubuntu-Server 14.04
Headless JD
Filebot

I fill JD with links, and it download it very fast, after extract i use a script to rename and move the files from SSD to my NAS
The problem i have right now, after extract it rename and move the files once by once, so it can be faster, when i rename and move every file instant after extract, so my question is...
Is JD only call once per once the EVENT SCRIPTER to tell FIlebot to rename and move the files, ore is JD give the Event to Filebot after it finished the extract, and Filebot is the slow one, and only rename and move 1 file at the same time, and take the next file after finish the one before?!

Sorry for my bad english.

[rednoah]

That probably depends entirely on what scripts you call and when. FileBot can process 1 or more files at time. It depends on how you call it.

[Crack00r]

jdtofilebot.sh

Code: Select all

#!/bin/sh
##FileBot needs that:
LOG="/root/logs/amc.log"
EXCLUDE="/root/logs//amc.txt"
SERIEN="/Daten/Serien/{n}/{'Staffel '+s}/{n} - {s00e00} - {t}"
FILME="/Daten/Filme/{n} ({y})"
UNSORTIERT="/Daten/unsortiert/{file.structurePathTail}" ### separate folder for filebot for the case that it cannot recognize a movie / series

##execute filebot
filebot -script fn:amc --lang de --conflict override --log-file $LOG --action move "$1" --def "seriesFormat=$SERIEN" "movieFormat=$FILME" "unsortedFormat=$UNSORTIERT" -non-strict --def unsorted=y --def artwork=y --def excludeList=$EXCLU$

## clean everything
filebot -script fn:cleaner "$1" --def root=y "exts=jpg|nfo|rar|nfo|htm|html|url|txt|etc" "terms=sample|trailer|etc"

JDownloader Event-Scripter

Code: Select all

[{"eventTrigger":"ON_ARCHIVE_EXTRACTED", "enabled":true, "name":"FileBot", "script":"var amcFile = \"/root/jdtofilebot.sh\";var path = archive.getFolder(); callAsync(function() {}, amcFile, path);", "eventTriggerSettings":{}}]

[rednoah]

I'd use the official scripts:
viewtopic.php?f=4&t=3478

You still haven't explained what you don't like about your current scripts and their current undesirable behaviour.

[Crack00r]

The problem is, right now JD, the Script ore Filebot, only transfer one after one... so it transfer only allways 1 file at the same time...

When 1 trasnfer is finished, it transfer the next file and so on

and i dont know, if its JD, the Script ore Filebot it self

the guys from JD says, JD execute the Trigger right after extact finished, so it seams the script it self is transfer 1 file after the next, ore its a bug from Filebot itself
https://board.jdownloader.org/showthrea ... post377613

[rednoah]

The amc script will process files in sequential order, so if you're using it to move/copy files there'll only be one at at a time. However, if read/write speed is the bottleneck, then moving/copying multiple files at the same time won't make it any faster.

You probably want to figure out which exact step of your download postprocess is taking longer than it should.

[Crack00r]

manual transfer 8 files at the same time i get 200mb/s

so, can you may help me to get more then one transfer?

[rednoah]

How did you confirm that the MOVE/COPY step is indeed the bottleneck?

FileBot won't move/copy in parallel. But why would a single transfer not max out the connection speed anyway?

[Crack00r]

its a upload to cloud in web, and its limited by connections, so more transfer = more speed

[rednoah]

So you don't have a NAS? What cloud storage provider are you using? What tools or virtual filesystem layer are you using to sync files?

[Crack00r]

Amazon acd_cli, i download and unrar to server and then push to amazon cloud, 1 transfer is ok with speed, but simultan transfer gives much more speed, i think they limit the upload to something around 25/30mb/sec

[rednoah]

FileBot can't do what you want.

What you can do is organize files into a new structure locally with move/symlink/hardlink which is pretty much instant, and then use other tools to parallel transfer as much as you want to it's final destination.

[Crack00r]

https://github.com/yadayada/acd_cli/issues/369

[rednoah]

The VFS parallelizing it's own data transfers is another option of course.

[Crack00r]

The question is, in Windows/Linux
I copy 4 filest, and paste Thema in a other folder
It copy once by once till its finished...

But here,
file 1 is finished, filebot rename and move the file. This take, lets say 5min.
15sec. Later next file ist rdy
File 2 is finished, filebot rename and move file 2, but why not instant, why filebot waiting till 1 is finished

[Crack00r]

How can i force to execute all, and not wait for one to end
https://goo.gl/photos/yN3ZNa1bSjVauSgV6

[rednoah]

You can only have one filebot process active at a time. If you call a second filebot process while the first one is not yet finished, then it'll wait for it's turn.

You can't have parallel filebot tasks, but you can have parallel cp tasks or parallel rsync:

What you can do is organize files into a new structure locally with move/symlink/hardlink which is pretty much instant, and then use other tools to parallel transfer as much as you want to it's final destination.