POSTBUCKET - where random posts in unrelated topics go

Any questions? Need some help?
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: MediaInfo Inspector

Post by rednoah »

{source} is not a MediaInfo binding. {source} is based primary on the filename, i.e. if the filename contains BluRay then {source} will be BluRay, if the filename is abc.mp4 then {source} will be undefined.
:idea: Please read the FAQ and How to Request Help.
redbull666
Donor
Posts: 54
Joined: 08 Jan 2014, 20:45

Re: MediaInfo Inspector

Post by redbull666 »

rednoah wrote: 09 Jun 2019, 07:37 {source} is not a MediaInfo binding. {source} is based primary on the filename, i.e. if the filename contains BluRay then {source} will be BluRay, if the filename is abc.mp4 then {source} will be undefined.
Aaah ok, still I did not get a valid source from the release with this filename:

Django Unchained (2012) (1080p BluRay x265 HEVC 10bit AAC 5.1 Silence)
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: MediaInfo Inspector

Post by rednoah »

{source} does work for this one:

Code: Select all

$ filebot -mediainfo . --format "[{source}] {fn}"
[BluRay] Django Unchained (2012) (1080p BluRay x265 HEVC 10bit AAC 5.1 Silence)
:idea: Please read the FAQ and How to Request Help.
redbull666
Donor
Posts: 54
Joined: 08 Jan 2014, 20:45

Re: MediaInfo Inspector

Post by redbull666 »

rednoah wrote: 10 Jun 2019, 15:50 {source} does work for this one:

Code: Select all

$ filebot -mediainfo . --format "[{source}] {fn}"
[BluRay] Django Unchained (2012) (1080p BluRay x265 HEVC 10bit AAC 5.1 Silence)
Right, I should have looked closer in the log...
Auto-detect movie from context: [/mnt/slow/torrents/media_downloads/Django Unchained (2012) (1080p BluRay x265 HEVC 10bit AAC 5.1 Silence)/Featurettes/The Costume Designs of Sharen Davis.mkv]
[HARDLINK] from [/mnt/slow/torrents/media_downloads/Django Unchained (2012) (1080p BluRay x265 HEVC 10bit AAC 5.1 Silence)/Django Unchained (2012) (1080p BluRay x265 Silence).mkv] to [/mnt/slow/media/Movies/Django Unchained (2012)/Django Unchained (2012) [x265, Blu-ray-1080p, AAC-5.1, KRaLiMaRKo].mkv]
[HARDLINK] from [/mnt/slow/torrents/media_downloads/Django Unchained (2012) (1080p BluRay x265 HEVC 10bit AAC 5.1 Silence)/Featurettes/Reimagining the Spaghetti - Western The Horses And Stunts of Django Unchained.mkv] to [/mnt/slow/media/Movies/Django Unchained (2012)/Django Unchained (2012) [x265, -1080p, AAC-2.0, ].mkv]
[OVERRIDE] Delete [/mnt/slow/media/Movies/Django Unchained (2012)/Django Unchained (2012) [x265, -1080p, AAC-2.0, ].mkv]
Some shitty Featurettes messed with this one...

Thanks for the help!
chimera
Posts: 3
Joined: 12 Jun 2019, 06:25

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by chimera »

Hi Rednoah,

after some hours of troubleshooting I have a question and would appreciate your help.

My setup is a OMV Host that runs a jdownloader docker instance. Filebot is installed on the OMV host.
The filebot script itself runs fine. I needed a way to call the filebot script, so I came up with a sshpass script from the docker instance which logs into the OMV instance to start the filebot script.
The only thing that is currently not working is calling the script from jdownloader, it always gives error code 2: file or directory not found.
After searching I ruled out the most obvious mistakes:

- The script, filebotrun.sh has chmod 777 and executable
- Has unix EOL encoding
- Within the jdownloader docker instance on bash, I can execute the shell script successfully but not from jdownloader itself (path is varied from jdownloader GUI, i can see the filebotrun.sh, so in theory file level access is given

Can you think of any reason why the docker jdownloader instance cannot access the shell script?
Thank you in advance for your help

Code: Select all

var script = JD_HOME + '/filebotrun.sh'

var path = package.getDownloadFolder()
var name = package.getName()
var label = package.getComment() ? package.getComment() : 'N/A'
var links = package.getDownloadLinks() ? package.getDownloadLinks() : []

function isReallyFinished() {
    for (var i = 0; i < links.length; i++) {
        if (links[i].getArchive() != null && links[i].getExtractionStatus() != "SUCCESSFUL" || !package.isFinished()) {
            return false
        }
    }
    return true
}

if (isReallyFinished()) {
    var command = [script, path, name, label, 'PACKAGE_FINISHED']
    log(command)
    log(callSync(command))
}

Shell script filebotrun.sh:

Code: Select all

#!/bin/bash
LOG="/sharedfolders/Storage/logs/amc.log"
EXC="/sharedfolders/Storage/logs/amc.txt"
SERIEN="/sharedfolders/plexroot/{plex}"
FILME="/sharedfolders/plexroot/{plex}"
USERNAME=root
HOSTS="172.17.0.1"
SCRIPT="filebot -script fn:amc --output "/sharedfolders/plexroot" --def movieformat="/sharedfolders/plexroot/{plex}" --def seriesformat="/sharedfolders/plexroot/{plex}" --action move --conflict auto -non-strict --log-file $LOG --def skipExtract=y unsorted=n music=n artwork=y --def storeReport=y --def clean=y --def plex=192.168.78.58:42Y97c6VATxkqyiNScD7 "
for HOSTNAME in ${HOSTS} ; do
    sshpass -p "pwd" ssh -o StrictHostKeyChecking=no -l ${USERNAME} ${HOSTNAME} "${SCRIPT}"
done
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by rednoah »

1.
If you're stuck with docker confinement, then i recommend using JD docker and FileBot Node docker.

Then you can create a scheduled task in the FileBot Node docker and execute it via curl from the JD docker.

That'll decoupling everything, making it easy to spot and fix issues.


2.
Is your *.sh script ever called by JD at all? If no, what does the JD log say? If yes, what does the output say?
:idea: Please read the FAQ and How to Request Help.
chimera
Posts: 3
Joined: 12 Jun 2019, 06:25

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by chimera »

Wow, thanks for quick reply.
I'll will check the Filebot node docker, thank you.

Below is the error from the log, even when I validate the script in the jdownloader GUI it says no file or directory but I rly dont get it.

Code: Select all

net.sourceforge.htmlunit.corejs.javascript.WrappedException: Wrapped org.jdownloader.extensions.eventscripter.EnvironmentException: Line 20
java.io.IOException: Cannot run program "/config/filebotrun.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:215)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:201)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:196)
	at org.jdownloader.extensions.eventscripter.sandboxobjects.ScriptEnvironment.callSync(ScriptEnvironment.java:263)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:120)
	at net.sourceforge.htmlunit.corejs.javascript.NativeJavaMethod.call(NativeJavaMethod.java:213)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1531)
	at script(:20)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:411)
	at org.jdownloader.scripting.JSHtmlUnitPermissionRestricter$SandboxContextFactory.doTopCall(JSHtmlUnitPermissionRestricter.java:119)
	at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3057)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
	at net.sourceforge.htmlunit.corejs.javascript.Context.evaluateString(Context.java:1212)
	at org.jdownloader.extensions.eventscripter.ScriptThread.evalUNtrusted(ScriptThread.java:275)
	at org.jdownloader.extensions.eventscripter.ScriptThread.executeScipt(ScriptThread.java:167)
	at org.jdownloader.extensions.eventscripter.ScriptThread.run(ScriptThread.java:147)
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 21 more
 (#20)
	at net.sourceforge.htmlunit.corejs.javascript.Context.throwAsScriptRuntimeEx(Context.java:1907)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:146)
	at net.sourceforge.htmlunit.corejs.javascript.NativeJavaMethod.call(NativeJavaMethod.java:213)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1531)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:411)
	at org.jdownloader.scripting.JSHtmlUnitPermissionRestricter$SandboxContextFactory.doTopCall(JSHtmlUnitPermissionRestricter.java:119)
	at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3057)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
	at net.sourceforge.htmlunit.corejs.javascript.Context.evaluateString(Context.java:1212)
	at org.jdownloader.extensions.eventscripter.ScriptThread.evalUNtrusted(ScriptThread.java:275)
	at org.jdownloader.extensions.eventscripter.ScriptThread.executeScipt(ScriptThread.java:167)
	at org.jdownloader.extensions.eventscripter.ScriptThread.run(ScriptThread.java:147)
Caused by: org.jdownloader.extensions.eventscripter.EnvironmentException: Line 20
java.io.IOException: Cannot run program "/config/filebotrun.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:215)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:201)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:196)
	at org.jdownloader.extensions.eventscripter.sandboxobjects.ScriptEnvironment.callSync(ScriptEnvironment.java:263)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:120)
	at net.sourceforge.htmlunit.corejs.javascript.NativeJavaMethod.call(NativeJavaMethod.java:213)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1531)
	at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:411)
	at org.jdownloader.scripting.JSHtmlUnitPermissionRestricter$SandboxContextFactory.doTopCall(JSHtmlUnitPermissionRestricter.java:119)
	at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3057)
	at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
	at net.sourceforge.htmlunit.corejs.javascript.Context.evaluateString(Context.java:1212)
	at org.jdownloader.extensions.eventscripter.ScriptThread.evalUNtrusted(ScriptThread.java:275)
	at org.jdownloader.extensions.eventscripter.ScriptThread.executeScipt(ScriptThread.java:167)
	at org.jdownloader.extensions.eventscripter.ScriptThread.run(ScriptThread.java:147)
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 21 more

	at org.jdownloader.extensions.eventscripter.sandboxobjects.ScriptEnvironment.callSync(ScriptEnvironment.java:266)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:120)
	... 12 more
Caused by: java.io.IOException: Cannot run program "/config/filebotrun.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:215)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:201)
	at org.appwork.utils.processes.ProcessBuilderFactory.runCommand(ProcessBuilderFactory.java:196)
	at org.jdownloader.extensions.eventscripter.sandboxobjects.ScriptEnvironment.callSync(ScriptEnvironment.java:263)
	... 17 more
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 21 more
chimera
Posts: 3
Joined: 12 Jun 2019, 06:25

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by chimera »

Hey Rednoah,

after looking at Filebot node, I would just use the filebot watcher to monitor my jdownloader extraction destination folders and go via amc from there.
That is way easier than I thought in comparison to my sshpass approach between containers lool ;-)

Impressive work btw, thank you for the tools mate!
leejk
Posts: 30
Joined: 03 Oct 2015, 04:18

Re: FileBot on the Mac App Store

Post by leejk »

Hi,

I have the MAS version, and previously I had install the Homebrew version installed as well in order to run scripts. However I had to uninstall/reinstall Homebrew, but now the Homebrew version wants a license. Has there been a change in the license policy?
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: FileBot on the Mac App Store

Post by rednoah »

Yes, FileBot is now available independently of any App Store:
viewtopic.php?f=9&t=6265

You can request a complimentary license if you need to work around Apple limitations:
viewtopic.php?f=8&t=9517
:idea: Please read the FAQ and How to Request Help.
OutrageousGem
Posts: 11
Joined: 12 Apr 2019, 06:20

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by OutrageousGem »

rednoah wrote: 12 Jun 2019, 06:57 1.
If you're stuck with docker confinement, then i recommend using JD docker and FileBot Node docker.

Then you can create a scheduled task in the FileBot Node docker and execute it via curl from the JD docker.

That'll decoupling everything, making it easy to spot and fix issues.
I ended up with the same idea in my own download setup. But when I try to schedule a task in FileBot Node docker I just get a

Code: Select all

{"success":false,"error":"NOT IMPLEMENTED"}
I doubt you would recommend unimplemented features. Any idea what might be going wrong?
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by rednoah »

Ups. Looks like this feature is only enabled for SYNO and QNAP builds even though it probably would work for docker too. I guess it's not as easy as I thought with the current release. I'll see what I can do for the next release.
:idea: Please read the FAQ and How to Request Help.
OutrageousGem
Posts: 11
Joined: 12 Apr 2019, 06:20

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by OutrageousGem »

Ah. I will hope for the next release then. Thank you.
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by rednoah »

I've update the FileBot Node docker container to FileBot Node 0.2.8.4. Please try the latest container and let me know if you encounter any problems.
:idea: Please read the FAQ and How to Request Help.
OutrageousGem
Posts: 11
Joined: 12 Apr 2019, 06:20

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by OutrageousGem »

Works great. Thank you! Two questions though:

1, Is it possible to somehow un-schedule a task? Can not find that option.

2, Is there any way to also schedule it to run at a specified time? Currently only a API call seems possible.
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by rednoah »

1.
Yes, but only by directly modifying the FileBot Node application data files, which are stored in /data/.filebot-node (container-point-of-view).


2.
FileBot Node does not include a scheduler. The scheduling of commands is best done with the built-in facilities of the platform, i.e. cron if you're using a generic Linux system.

:!: But at that point, why would you be using FileBot Node in the first place? You could just use cron to schedule docker filebot amc script calls directly, without having FileBot Node running at all.
:idea: Please read the FAQ and How to Request Help.
OutrageousGem
Posts: 11
Joined: 12 Apr 2019, 06:20

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by OutrageousGem »

1, Got it.

2, Just because Node makes it much more convenient to rapidly test different iterations of a command and schedule it when it finally does the right thing. (I am just bad at writing shell scripts. Always some typo / unescaped character / etc.... :? ). The API Is all I currently really need.
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by rednoah »

OutrageousGem wrote: 17 Jun 2019, 09:37 Just because Node makes it much more convenient to rapidly test different iterations of a command and schedule it when it finally does the right thing. (I am just bad at writing shell scripts. Always some typo / unescaped character / etc.... :? ). The API Is all I currently really need.
I see. Yes, FileBot Node certainly has value in generating CLI commands. You could always use the WebUI to generate commands and the copy & paste that into your scheduled shell script.

:idea: Use the @file syntax for reading command-line arguments from external text files. That'll make things easy as pie. That's also what prepared tasks do in form of *.args files. ;)
:idea: Please read the FAQ and How to Request Help.
b2un0
Posts: 10
Joined: 09 Jun 2014, 10:41

Re: Metadata and Extended Attributes

Post by b2un0 »

Hi,

is there any option to add filebot xattr to already sorted files?

eg, i have multiple series where on some seasons the xattr from filebot are missing, so ne the " filebot missing script" does not work correct and think the season is missing.
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Metadata and Extended Attributes

Post by rednoah »

You'll have to process files through FileBot again to ensure that xattr are set this time.

You can find files without xattr metadata like so:

Code: Select all

filebot -mediainfo . --filter '!f.metadata' --format {f}
:idea: Please read the FAQ and How to Request Help.
b2un0
Posts: 10
Joined: 09 Jun 2014, 10:41

Re: Metadata and Extended Attributes

Post by b2un0 »

thanks for your fast reply!

thats the first thing i did, but filebot says "Skipped, already exists". :(
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Metadata and Extended Attributes

Post by rednoah »

1.
You can move all untagged files elsewhere first:

Code: Select all

filebot -mediainfo . --filter '!f.metadata' -exec mv -v {f} /path/to/untagged/files/{f.name}

2.
However, the general smart approach is to never touch files in your Media folder, and only ever move files there via FileBot. That way you can ensure that everything is always in perfect order.

i.e. just process all your files into a new folder, and then remove the old one if you're happy with the new one

Code: Select all

--format /path/to/new/folder/{plex}
:idea: Please read the FAQ and How to Request Help.
eggsplorer
Posts: 9
Joined: 21 Jun 2019, 23:05

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by eggsplorer »

I followed your steps in post #1 and it works very well I just can't change any of the downloaded files because I have no permission.

How can I get filebot to create files/folders with correct permissions?

On my Media Drive all my files have:
Owner: 1000
Group: 100
Permissions: 775

Files renamed and moved from Filebot have:
Owner: 0
Group: 0
Permissions: 775
User avatar
rednoah
The Source
Posts: 19140
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: [JDownloader] Setup for Windows, Linux and Mac OS X

Post by rednoah »

FileBot does not set or change permissions in any way. It does whatever the OS does by default.

The amc script does have the --def exec option though, which allows you to run your own post-processing commands on newly processed files:

Code: Select all

--def "exec=chmod -R 775 {quote folder} && chown -R admin:administrators {quote folder}"
:idea: Please read the FAQ and How to Request Help.
ZeroByDivide
402
Posts: 157
Joined: 16 Dec 2014, 01:39

Re: [DOCS] Use --mapper expressions for AniDB / TheTVDB cross-entity matching

Post by ZeroByDivide »

Any idea when 4.8.6 is coming to the windows store?
Post Reply