Filebot in python

Running FileBot from the console, Groovy scripting, shell scripts, etc
Post Reply
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Filebot in python

Post by saitoh183 »

I have a python script that calls filebot to rename my files and then uses exec to call another python file. When i run the script once, filebot fails to run (does not create a log either) but when i run the same script a second time, filebot does run properly. Im running this on a headless seedbox using filebot.sh. I will explain what the script doesn and where it fails.

The script:

Code: Select all

#!/usr/bin/env python

import os
import re
import subprocess
from os.path import getsize
import shutil
import fnmatch
import logging
from logging.handlers import RotatingFileHandler
import argparse
import time
import unpack_transfer

#Set time zone
os.environ['TZ'] = 'EST+05EDT,M4.1.0,M10.5.0'
time.tzset()

#Parser get external vairables
parser = argparse.ArgumentParser()
parser.add_argument('-v','--verbose', help="Enable debug",action="store_true")
args = parser.parse_args()

log_filename = "/media/sdu1/home/saitoh183/private/logs/MediaMgmt_remote.log"

logger = logging.getLogger('Media_Processing')
if args.verbose:
    logger.setLevel(logging.DEBUG)
    dbug = '-v'
else:
    logger.setLevel(logging.INFO)
    dbug = ""

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

fh = RotatingFileHandler(log_filename, maxBytes=100000, backupCount=5)
fh.setFormatter(formatter)
logger.addHandler(fh)

console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(formatter)
logger.addHandler(console)

def get_child2(path):
    return [item for item in os.listdir(path) if item not in ('.stignore', '.stfolder') and getsize(os.path.join(path,item)) > 104857600]

def get_child(path):
    return [item for item in os.listdir(path) if item not in ('.stignore', '.stfolder')]

def splitup(fdir,mediafolder,dir_):
        logger.debug('FDIR: {}.'.format(fdir))
        count = len(get_child2(fdir))
        logger.debug('LENGH: {}.'.format(count))
        newfdir = os.path.join(mediafolder, dir_)
        if count >= 1:
            logger.info('Splitting up Folders in {}.'.format(fdir))
            try:
                shutil.move(fdir, mediafolder)
                return newfdir
            except Exception:
                pass
            logger.info('Splitting up files in {}.'.format(newfdir))
            for root, dirs, files2 in os.walk(newfdir):
               for file_ in files2:
                    dst = os.path.join(mediafolder,file_)
                    current_file = os.path.join(root, file_)
                    logger.debug('Processing {}.'.format(current_file))
                    filesize = getsize(current_file)
                    if filesize > 104857600:
                        logger.debug('Destination1: {}.'.format(dst))
                        logger.debug('Destination2: {}.'.format(mediafolder))
                    if not os.path.exists(dst):
                        os.makedirs(dst)
                        logger.info('Moving {} to {}.'.format(current_file,dst))
                        shutil.move(current_file, dst)

def splitupmovies(mediafolder):
        logger.info('Splitting up...Checking folder {}.'.format(mediafolder))
        for path, subdirs, files in os.walk(mediafolder):
            logger.debug('root folder: {}.'.format(path))
            logger.debug('directories: {}.'.format(subdirs))
            logger.debug('files: {}.'.format(files))
            for dir_ in subdirs:
                fdir = os.path.join(path,dir_)
                newdir = splitup(fdir,mediafolder,dir_)
                logger.debug('NEWDIR: {}'.format(newdir))
                if newdir != None:
                    newcount = len(get_child2(newdir))
                    logger.debug('NEWCOUNT: {}'.format(newcount))
                    if newcount > 1:
                        splitup(newdir,mediafolder,dir_)



def clean_files(folder):
    for path, subdirs, files in os.walk(folder):
        logger.info('Removing small files from {}.'.format(path))
        for name in files:
            current_file = os.path.join(path, name)

            # Check if file is more than 100mb
            try:
                filesize = getsize(current_file)
            except os.error as e:
                logger.error('Was unable to get filesize of {} ({}).'.format(name,e))
            # File doesn't exist or is inaccessible
                pass

            # Remove all files that are 100mb and less from every subfolder
            if name not in ('.stignore', '.stfolder') and filesize <= 104857600 or fnmatch.fnmatch(name,'name.r*'):
                logger.debug('Removing file {}.'.format(name))
                os.remove(current_file)

def convert_files(filepath):
        convert_mp4 = subprocess.Popen([
            "/media/sdu1/home/saitoh183/private/script/Convert/manual.py",
            '-a', '-i', filepath,
            '-pr','-m', final_path
        ],stdout=subprocess.PIPE).communicate()[0]
        logger.debug('{}'.format(convert_mp4))
        logger.info('End of conversion of files in {}'.format(filepath))
        # Remove all empty directories:
        for root, dirs, files in os.walk(filepath, topdown=False):
            logger.info('Checking for empty directories in {}.'.format(root))
            logger.debug('root folder: {}.'.format(root))
            logger.debug('directories: {}.'.format(dirs))
            logger.debug('files: {}.'.format(files))
            for dir_ in dirs:
                try:
                    logger.info('Removing directory {}.'.format(dir_))
                    os.rmdir(os.path.join(root, dir_))
                except OSError as e:
                    logger.warning( '{}'.format(e))

def check_audio(f_path):
    for path, subdirs, files in os.walk(f_path):
        logger.info('Verifiying for Dual Audio Movies in {}.'.format(path))
        logger.debug('path folder: {}.'.format(path))
        logger.debug('directories: {}.'.format(subdirs))
        logger.debug('files: {}.'.format(files))
        for name in files:

            if any(name.endswith(ext) for ext in ('mkv', 'avi', 'mp4')):
                current_file = os.path.join(path, name)
                logger.debug('PROCESSING: {}'.format(current_file))
                audio = subprocess.check_output([
                    "mediainfo",
                    '--Inform=General;%AudioCount%', current_file
                ])
                logger.debug('File {} has {} audio stream(s).'.format(name,audio))
                language = subprocess.check_output([
                    "mediainfo",
                    '--Inform=Audio;%Language%', current_file
                ])
                logger.debug('File {} has the following language(s): {}'.format(name,language))
                if int(audio) > 1 and not fnmatch.fnmatch(language,'fr*fr'): # Dual audio
                    if os.path.join(folder_path, name) == current_file:
                        # File exist at the root
                        logger.info('Moving file {} to dualmovies.'.format(name))
                        dst = os.path.join(subfolders[5], name)
                        os.rename(current_file, dst)
                    else:
                        # File is in subfolder, move subfolder
                        logger.info('Moving directory {} to dualmovies.'.format(path))
                        shutil.move(path, subfolders[5])

                elif "Francais" in language or "fr" in language: # French movie
                    if os.path.join(folder_path, name) == current_file:
                        # File exist at the root
                        logger.info('Moving file {} to french.'.format(name))
                        dst = os.path.join(subfolders[4], name)
                        os.rename(current_file, dst)
                    else:
                        # File is in subfolder, move subfolder
                        logger.info('Moving directory {} to french.'.format(path))
                        shutil.move(path, subfolders[4])
                elif language == "":
                    logger.info('Audio language was not found for {}. It will stay in {}.'.format(name,folder_path))
                else:
                    continue

def get_command(command, extra):
    cmd = list(command)
    cmd.extend(extra)
    return cmd

filebot_cmd = [
    "/media/sdu1/home/saitoh183/filebot/filebot.sh", #ie 'C:\Program Files\Filebot\filebot.exe'

    # I recommend replacing 'fn:amc' with the path to a locally saved copy of
    # amc.groovy. 'fn:amc' downloads the script every time you run filebot.exe
    # from filebot.net.  If filebot.net is down, your file processing won't be
    # able to run. You can download the script from here:
    # https://github.com/filebot/scripts/blob/devel/amc.groovy
    '-script', 'fn:amc',

    # Save log files to same directory where this script file is located
    # '--log-file', logfiles,
    '--action', 'move',
    '--def', 'clean=y',
    '-no-xattr',
    #'--log=Fine'
]

# Declare variables
source = "/media/sdu1/home/saitoh183/private/Transfer"
importing = os.path.join(source, "importing")

subfolders = [
        os.path.join(source, "movies_cps"),
        os.path.join(source, "sonarr_tv"),
        os.path.join(source, "sickrage_tv"),
        os.path.join(source, "ufc"),
        os.path.join(source, "french"),
        os.path.join(source, "dualmovies")
]
importfolders = [
        os.path.join(importing,"clone"),
        os.path.join(source, "manage", "movies," "cps"),
        os.path.join(source, "manage", "movies", "ufc"),
        os.path.join(source, "manage", "movies", "dualmovies"),
        os.path.join(source, "manage", "movies", "french")
]

for folder_path in subfolders:
        logger.info('Processing path {}.'.format(folder_path))
        childcount = len(get_child(folder_path))
        logger.debug('Number of items in {}: {}.'.format(folder_path,childcount))
        if not childcount:
            logger.debug('There is nothing to process in {}.'.format(folder_path))
            logger.info('-----------------------------------------------------------------')
            continue


        # Convert media files and assign final import path

        if "sonarr_tv" in folder_path:
            unpack_transfer.transfer(folder_path.rsplit('/',1)[1])
            final_path = importfolders[0]

        elif "sickrage_tv" in folder_path:
            unpack_transfer.transfer(folder_path.rsplit('/',1)[1])
            final_path = importfolders[0

        elif "movies_cps" in folder_path:
            unpack_transfer.transfer(folder_path.rsplit('/',1)[1])
            final_path = importfolders[1]
            splitupmovies(folder_path)
            check_audio(folder_path)
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/cps/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\"\"\"'
            ])

        elif "ufc" in folder_path:
            final_path = importfolders[2]
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/ufc/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\"\"\"'
            ])

        elif "dualmovies" in folder_path:
            final_path = importfolders[3]
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\" -fr \"{localize.french.name} ({y})\" -en \"{ny}\"\"\"'
            ])

        elif "french" in folder_path:
            final_path = importfolders[4]
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/french/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--lang', 'fr',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\"\"\"'
            ])


        clean_files(folder_path)
        logger.info('Starting converstion in {}'.format(folder_path))
        convert_files(folder_path)
        if any(re.findall(r'ufc|french|dualmovies|cps', final_path, re.IGNORECASE)):
            childcount = len(get_child(final_path))
            if childcount:
                new_filebot.extend(['-non-strict', final_path])
                logger.info('Filebot Processing Started...')
                FB = subprocess.Popen(new_filebot,stdout=subprocess.PIPE).communicate()[0]
                logger.info('Filebot Processing Ended...')
                logger.debug('{}'.format(FB))
                logger.info('-----------------------------------------------------------------')
Steps of script:

The script will loop through each of the folders in my subfolder variable. but only the folders containing movies use FB. So the first stop is movies_cps

1 - File is downloaded to .../Transfer/movies_cps
2 - Files are unpacked/and separated into each their own subfolder
3 - They are checked for their audio and
- if they contain English audio only, they stay in movies_cps
- if they contain French audio only, they go .../Transfer/french
- if they contain DualAudio audio only, they go .../Transfer/dualmovies

4 - Script then removes all non-essential files (samples, nfo, etc..)
5 - I then run convert_files function which launches sickbeard_mp4_automator on movies_cps (or whichever folder folder_path is equal to)
6 - SickMp4Auto output's the file to the final_path variable which would equal to /media/sdu1/home/saitoh183/private/Transfer/manage/movies/(cps,french,dualmovies)

7 - Next I check if there is content in the final_path and if there is, Run Filebot on final_path.

Now I added logging to try and capture the error but my logging comes up empty. I added an info line to see when filebot is called and when it ends and in the log, those lines are both there but nothing in between. I also enabled logs in the filebot command and set it to ALL but nothing was generated either.

So then if I take one of the files that are currently in final_path and move them back to their original location. Then I run the script again, my function convert_files will not convert the file cuz it has already been converted and it will move it directly to final_path, then the script will move on to Filebot, and filebot will do it things and generate logs and the files will be renamed.

So I don't know when on the first pass filebot is failing and doesn't generate any sort of logs but maybe you would have an idea or maybe a way for me to see what is going on with FB on the first attempt.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot in python

Post by rednoah »

If there is no output whatsoever, then filebot is most likely never called. Make sure to check both standard and error output of the filebot.sh call.
:idea: Please read the FAQ and How to Request Help.
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

Here is what i was able to catch: Note that the script is running via cronjob

Code: Select all

2017-01-26 16:27:15,831 - Media_Processing - DEBUG - StandardOut: Exception in thread "main" java.lang.UnsupportedClassVersionError: net/filebot/Main : Unsupported major.minor version 52.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
when i put the files back in the start folder and run the script via SSH, it works

Code: Select all

2017-01-26 18:13:06,836 - Media_Processing - DEBUG - StandardOut: Locking /media/sdu1/home/saitoh183/private/logs/Filebot.log
Run script [fn:amc] at [Thu Jan 26 18:05:30 GMT-05:00 2017]
Parameter: clean = y
Parameter: movieFormat = /media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/{plex[1..-1]}
Parameter: ut_label = Movies
Parameter: exec =  ""python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp "{file}" -fr "{localize.french.name} ({y})" -en "{ny}"""
Argument[0]: /media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies
{duration} => Expression yields empty value: Binding "duration": Native library (com/sun/jna/linux-x86-64/libjnidispatch.so) not found in resource path ([file:/media/sdu1/home/saitoh183/filebot/FileBot.jar])
{duration} => Expression yields empty value: Binding "duration": Could not initialize class net.filebot.mediainfo.MediaInfoLibrary
Input: /media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4/Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4
Input: /media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4/Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4
Group: [mov:planes 2013] => [Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4]
Group: [mov:planes fire rescue 2014] => [Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4]
Rename movies using [TheMovieDB]
Auto-detect movie from context: [/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4/Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4]
[MOVE] Rename [/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4/Planes (2013) MULTi VF2 [1080p] BluRay x264-PopHD.mp4] to [/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes (2013)/Planes (2013).mp4]
Processed 1 files
Rename movies using [TheMovieDB]
Auto-detect movie from context: [/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4/Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4]
[MOVE] Rename [/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4/Planes Fire & Rescue (2014) MULTi VF2 [1080p] BluRay x264-PopHD.mp4] to [/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes - Fire & Rescue (2014)/Planes - Fire & Rescue (2014).mp4]
Processed 1 files
Execute: ""python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp "/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/Planes (2013)/Planes (2013).mp4" -fr "Planes (2013)" -en "Planes (2013)"""
sysinfo:

Code: Select all

/media/sdu1/home/saitoh183/filebot/filebot.sh -script fn:sysinfo
FileBot 4.7.7 (r4678)
JNA Native: java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/linux-x86-64/libjnidispatch.so) not found in resource path ([file:/media/sdu1/home/saitoh183/filebot/FileBot.jar])
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Could not initialize class com.sun.jna.Native
Apache Commons VFS: [zip, rar]
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Script Bundle: 2017-01-05 (r470)
Groovy: 2.4.7
JRE: Java(TM) SE Runtime Environment 1.8.0_65
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 24 Core / 26 GB Max Memory / 38 MB Used Memory
OS: Linux (amd64)
Package: PORTABLE
Data: /media/sdu1/home/saitoh183/filebot/data
uname: Linux erebus 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux
Done ヾ(@⌒ー⌒@)ノ

User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot in python

Post by rednoah »

This error means that you're using Java 7 or older when Java 8 is required:

Code: Select all

Exception in thread "main" java.lang.UnsupportedClassVersionError: net/filebot/Main : Unsupported major.minor version 52.0
Judging by the logs, you have both Java 7 and Java 8 installed.

You'll find that the SSH user session environment is different from the syscall exec environment:
viewtopic.php?f=4&t=3067

:idea: Logging in with SSH and running scripts and running scripts via cron is completely different. It works via SSH because your .profile or .bashrc fix your environment, whereas these user/shell specific startup scripts are not called for cron jobs.
:idea: Please read the FAQ and How to Request Help.
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

I thought it was something like that...i guess i need to add this to my path?

Code: Select all

XDG_SESSION_ID=122225
SHELL=/bin/bash
TERM=xterm
XDG_SESSION_COOKIE=704540afa834052f08385b0000000298-1485351571.834440-51447201

SSH_TTY=/dev/pts/92
LC_ALL=en_GB.UTF-8

LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
PATH=/media/sdu1/home/saitoh183/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/saitoh183
PWD=/media/sdu1/home/saitoh183
LANG=en_GB.UTF-8
HOME=/media/sdu1/home/saitoh183
SHLVL=2
LANGUAGE=en_GB:en
LOGNAME=saitoh183
SSH_CONNECTION=24.202.151.170 57011 185.21.216.146 22
XDG_RUNTIME_DIR=/run/user/1535
_=/usr/bin/printenv
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot in python

Post by rednoah »

Just check which java executable is called:

Code: Select all

ls -l `which java`
Try this in both SSH and cron context so you can see the difference. It'll be using the first executable that appears in the PATH.
:idea: Please read the FAQ and How to Request Help.
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

alright, i will give this a go tomorrow and post back...

Thanks rednoah
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

I ended up fixing it by adding

Code: Select all

. $HOME/.profile,
to my cronjob

Thanks again for pointing me in the right direction
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

rednoah wrote:Just check which java executable is called:

Code: Select all

ls -l `which java`
Try this in both SSH and cron context so you can see the difference. It'll be using the first executable that appears in the PATH.
saitoh183 wrote:I ended up fixing it by adding

Code: Select all

. $HOME/.profile,
to my cronjob

Thanks again for pointing me in the right direction
So the problem seems to be back...so i did as you said and here is the results:

Cron : lrwxrwxrwx 1 root root 22 Mar 19 2013 /usr/bin/java -> /etc/alternatives/java

mine: -rwx------ 1 saitoh183 saitoh183 7734 Oct 7 2015 /media/sdu1/home/saitoh183/bin/java

so what do i do?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot in python

Post by rednoah »

Follow this Guide:
viewtopic.php?f=4&t=3067
:idea: Please read the FAQ and How to Request Help.
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

rednoah wrote:Follow this Guide:
viewtopic.php?f=4&t=3067
Happens in normal shell as well now

sysinfo:

Code: Select all

FileBot 4.7.7 (r4678)
JNA Native: java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/linux-x86-64/libjnidispatch.so) not found in resource path ([file:/media/sdu1/home/saitoh183/filebot/FileBot.jar])
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Could not initialize class com.sun.jna.Native
Apache Commons VFS: [zip, rar]
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2017-02-10 (r480)
Groovy: 2.4.7
JRE: Java(TM) SE Runtime Environment 1.8.0_65
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 24 Core / 26 GB Max Memory / 38 MB Used Memory
OS: Linux (amd64)
Package: PORTABLE
Data: /media/sdu1/home/saitoh183/filebot/data
uname: Linux erebus 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux
Done ヾ(@⌒ー⌒@)ノ
Printenv:

Code: Select all

XDG_SESSION_ID=164658
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=704540afa834052f08385b0000000298-1486489360.651986-458243678
SSH_CLIENT=24.202.151.170 64947 22
SSH_TTY=/dev/pts/187
LC_ALL=en_GB.UTF-8
USER=saitoh183
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
MAIL=/var/mail/saitoh183
PATH=/media/sdu1/home/saitoh183/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
PWD=/media/sdu1/home/saitoh183
LANG=en_GB.UTF-8
SHLVL=1
HOME=/media/sdu1/home/saitoh183
LANGUAGE=en_GB:en
LOGNAME=saitoh183
SSH_CONNECTION=24.202.151.170 64947 185.21.216.146 22
XDG_RUNTIME_DIR=/run/user/1535
_=/usr/bin/printenv
sysenv:

Code: Select all

# Environment Variables #
XDG_SESSION_COOKIE: 704540afa834052f08385b0000000298-1486489360.651986-458243678
_: /media/sdu1/home/saitoh183/bin/java
MAIL: /var/mail/saitoh183
LOGNAME: saitoh183
SHLVL: 2
LD_LIBRARY_PATH: /media/sdu1/home/saitoh183/filebot
LANGUAGE: en_GB:en
LC_ALL: en_US.UTF-8
XDG_RUNTIME_DIR: /run/user/1535
HOME: /media/sdu1/home/saitoh183
SSH_CONNECTION: 24.202.151.170 64947 185.21.216.146 22
PATH: /media/sdu1/home/saitoh183/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
SSH_TTY: /dev/pts/187
SSH_CLIENT: 24.202.151.170 64947 22
XFILESEARCHPATH: /usr/dt/app-defaults/%L/Dt
XDG_SESSION_ID: 164658
SHELL: /bin/bash
TERM: xterm
NLSPATH: /usr/dt/lib/nls/msg/%L/%N.cat
OLDPWD: /media/sdu1/home/saitoh183
LANG: en_US.UTF-8
PWD: /media/sdu1/home/saitoh183
USER: saitoh183
LS_COLORS: rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:


# Java System Properties #
path.separator: :
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
java.io.tmpdir: /media/sdu1/home/saitoh183/filebot/data/tmp
user.country: US
user.home: /media/sdu1/home/saitoh183/filebot/data
net.filebot.util.prefs.file: /media/sdu1/home/saitoh183/filebot/data/prefs.properties
useExtendedFileAttributes: true
net.filebot.Archive.extractor: ApacheVFS
org.apache.commons.logging.Log: org.apache.commons.logging.impl.NoOpLog
java.vm.vendor: Oracle Corporation
java.util.prefs.PreferencesFactory: net.filebot.util.prefs.FilePreferencesFactory
user.language: en
java.vendor: Oracle Corporation
java.vendor.url.bug: http://bugreport.sun.com/bugreport/
swing.crossplatformlaf: javax.swing.plaf.nimbus.NimbusLookAndFeel
sun.net.client.defaultReadTimeout: 60000
java.specification.name: Java Platform API Specification
jna.nosys: false
os.name: Linux
java.runtime.name: Java(TM) SE Runtime Environment
application.deployment: portable
file.separator: /
sun.net.client.defaultConnectTimeout: 10000
java.vm.specification.version: 1.8
sun.boot.class.path: /media/sdu1/home/saitoh183/lib/resources.jar:/media/sdu1/home/saitoh183/lib/rt.jar:/media/sdu1/home/saitoh183/lib/sunrsasign.jar:/media/sdu1/home/saitoh183/lib/jsse.jar:/media/sdu1/home/saitoh183/lib/jce.jar:/media/sdu1/home/saitoh183/lib/charsets.jar:/media/sdu1/home/saitoh183/lib/jfr.jar:/media/sdu1/home/saitoh183/classes
java.version: 1.8.0_65
http.agent: FileBot 4.7.7
java.library.path: /media/sdu1/home/saitoh183/filebot:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
unixfs: false
useGVFS: false
awt.toolkit: sun.awt.X11.XToolkit
java.vm.specification.name: Java Virtual Machine Specification
os.version: 3.16.0-4-amd64
useCreationDate: false
file.encoding: UTF-8
java.awt.printerjob: sun.print.PSPrinterJob
grape.root: /media/sdu1/home/saitoh183/filebot/data/grape
java.class.version: 52.0
java.ext.dirs: /media/sdu1/home/saitoh183/lib/ext:/usr/java/packages/lib/ext
java.specification.version: 1.8
net.filebot.AcoustID.fpcalc: fpcalc
java.vendor.url: http://java.oracle.com/
sun.os.patch.level: unknown
sun.java.launcher: SUN_STANDARD
os.arch: amd64
user.dir: /media/sdu1/home/saitoh183
line.separator: 

user.timezone: 
sun.cpu.endian: little
java.vm.specification.vendor: Oracle Corporation
java.home: /media/sdu1/home/saitoh183
java.net.useSystemProxies: false
sun.management.compiler: HotSpot 64-Bit Tiered Compilers
sun.arch.data.model: 64
java.endorsed.dirs: /media/sdu1/home/saitoh183/lib/endorsed
file.encoding.pkg: sun.io
java.specification.vendor: Oracle Corporation
sun.boot.library.path: /media/sdu1/home/saitoh183/lib/amd64
sun.jnu.encoding: UTF-8
java.runtime.version: 1.8.0_65-b17
java.vm.info: mixed mode
java.vm.version: 25.65-b01
sun.io.unicode.encoding: UnicodeLittle
java.awt.graphicsenv: sun.awt.X11GraphicsEnvironment
application.dir: /media/sdu1/home/saitoh183/filebot/data
java.class.path: /media/sdu1/home/saitoh183/filebot/FileBot.jar
sun.java.command: /media/sdu1/home/saitoh183/filebot/FileBot.jar -script fn:sysenv
jna.nounpack: true
sun.cpu.isalist: 
user.name: saitoh183


# Arguments #
args[0] = -script
args[1] = fn:sysenv


Done ヾ(@⌒ー⌒@)ノ
STDOUT:

Code: Select all

Locking /media/sdu1/home/saitoh183/private/script/Filebot.log
Run script [fn:amc] at [Wed Feb 15 00:24:34 GMT-05:00 2017]
Parameter: clean = y
Parameter: movieFormat = /media/sdu1/home/saitoh183/private/Transfer/manage/movies/ufc/{plex[1..-1]}
Parameter: ut_label = Movies
Parameter: exec =  ""python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp "{file}"""
Argument[0]: /media/sdu1/home/saitoh183/private/Transfer/manage/movies/ufc
{duration} => Expression yields empty value: Binding "duration": Native library (com/sun/jna/linux-x86-64/libjnidispatch.so) not found in resource path ([file:/media/sdu1/home/saitoh183/filebot/FileBot.jar])
Input: /media/sdu1/home/saitoh183/private/Transfer/manage/movies/ufc/ufc.208.ppv.720p.hdtv.x264-plutonium.mp4
Group: [mov:null] => [ufc.208.ppv.720p.hdtv.x264-plutonium.mp4]
Clean clutter files and empty folders
Finished without processing any files
Failure (°_°)
STDERR empty

I followed this to install Filebot on my Feral seedbox

Java:
https://tinyurl.com/zolmmrh

and this to install Filebot:

https://tinyurl.com/jy7b9tb

Also i dont have root access on my seedbox
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot in python

Post by rednoah »

So everything is running fine. What's the problem? What is the expected behaviour?

:!: The file ufc.208.ppv.720p.hdtv.x264-plutonium.mp4 just happens to not be working because it's a shitty file. I'm sure you have tested with more than 1 single shitty file before you conclude that "filebot is not working" right? Some file just don't work.

:!: libjnidispatch / libmediainfo are either missing or incompatible. It's optional though.
:idea: Please read the FAQ and How to Request Help.
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

rednoah wrote:So everything is running fine. What's the problem? What is the expected behaviour?

:!: The file ufc.208.ppv.720p.hdtv.x264-plutonium.mp4 just happens to not be working because it's a shitty file. I'm sure you have tested with more than 1 single shitty file before you conclude that "filebot is not working" right? Some file just don't work.

:!: libjnidispatch / libmediainfo are either missing or incompatible. It's optional though.
Yes.. But i figured out why it wasn't working with my other files... In my command a commas slip in somewhere it shouldnt have been.
carloslozada
Posts: 14
Joined: 12 Jun 2016, 11:06

Re: Filebot in python

Post by carloslozada »

saitoh183 wrote:I have a python script that calls filebot to rename my files and then uses exec to call another python file. When i run the script once, filebot fails to run (does not create a log either) but when i run the same script a second time, filebot does run properly. Im running this on a headless seedbox using filebot.sh. I will explain what the script doesn and where it fails.

The script:

Code: Select all

#!/usr/bin/env python

import os
import re
import subprocess
from os.path import getsize
import shutil
import fnmatch
import logging
from logging.handlers import RotatingFileHandler
import argparse
import time
import unpack_transfer

#Set time zone
os.environ['TZ'] = 'EST+05EDT,M4.1.0,M10.5.0'
time.tzset()

#Parser get external vairables
parser = argparse.ArgumentParser()
parser.add_argument('-v','--verbose', help="Enable debug",action="store_true")
args = parser.parse_args()

log_filename = "/media/sdu1/home/saitoh183/private/logs/MediaMgmt_remote.log"

logger = logging.getLogger('Media_Processing')
if args.verbose:
    logger.setLevel(logging.DEBUG)
    dbug = '-v'
else:
    logger.setLevel(logging.INFO)
    dbug = ""

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

fh = RotatingFileHandler(log_filename, maxBytes=100000, backupCount=5)
fh.setFormatter(formatter)
logger.addHandler(fh)

console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(formatter)
logger.addHandler(console)

def get_child2(path):
    return [item for item in os.listdir(path) if item not in ('.stignore', '.stfolder') and getsize(os.path.join(path,item)) > 104857600]

def get_child(path):
    return [item for item in os.listdir(path) if item not in ('.stignore', '.stfolder')]

def splitup(fdir,mediafolder,dir_):
        logger.debug('FDIR: {}.'.format(fdir))
        count = len(get_child2(fdir))
        logger.debug('LENGH: {}.'.format(count))
        newfdir = os.path.join(mediafolder, dir_)
        if count >= 1:
            logger.info('Splitting up Folders in {}.'.format(fdir))
            try:
                shutil.move(fdir, mediafolder)
                return newfdir
            except Exception:
                pass
            logger.info('Splitting up files in {}.'.format(newfdir))
            for root, dirs, files2 in os.walk(newfdir):
               for file_ in files2:
                    dst = os.path.join(mediafolder,file_)
                    current_file = os.path.join(root, file_)
                    logger.debug('Processing {}.'.format(current_file))
                    filesize = getsize(current_file)
                    if filesize > 104857600:
                        logger.debug('Destination1: {}.'.format(dst))
                        logger.debug('Destination2: {}.'.format(mediafolder))
                    if not os.path.exists(dst):
                        os.makedirs(dst)
                        logger.info('Moving {} to {}.'.format(current_file,dst))
                        shutil.move(current_file, dst)

def splitupmovies(mediafolder):
        logger.info('Splitting up...Checking folder {}.'.format(mediafolder))
        for path, subdirs, files in os.walk(mediafolder):
            logger.debug('root folder: {}.'.format(path))
            logger.debug('directories: {}.'.format(subdirs))
            logger.debug('files: {}.'.format(files))
            for dir_ in subdirs:
                fdir = os.path.join(path,dir_)
                newdir = splitup(fdir,mediafolder,dir_)
                logger.debug('NEWDIR: {}'.format(newdir))
                if newdir != None:
                    newcount = len(get_child2(newdir))
                    logger.debug('NEWCOUNT: {}'.format(newcount))
                    if newcount > 1:
                        splitup(newdir,mediafolder,dir_)



def clean_files(folder):
    for path, subdirs, files in os.walk(folder):
        logger.info('Removing small files from {}.'.format(path))
        for name in files:
            current_file = os.path.join(path, name)

            # Check if file is more than 100mb
            try:
                filesize = getsize(current_file)
            except os.error as e:
                logger.error('Was unable to get filesize of {} ({}).'.format(name,e))
            # File doesn't exist or is inaccessible
                pass

            # Remove all files that are 100mb and less from every subfolder
            if name not in ('.stignore', '.stfolder') and filesize <= 104857600 or fnmatch.fnmatch(name,'name.r*'):
                logger.debug('Removing file {}.'.format(name))
                os.remove(current_file)

def convert_files(filepath):
        convert_mp4 = subprocess.Popen([
            "/media/sdu1/home/saitoh183/private/script/Convert/manual.py",
            '-a', '-i', filepath,
            '-pr','-m', final_path
        ],stdout=subprocess.PIPE).communicate()[0]
        logger.debug('{}'.format(convert_mp4))
        logger.info('End of conversion of files in {}'.format(filepath))
        # Remove all empty directories:
        for root, dirs, files in os.walk(filepath, topdown=False):
            logger.info('Checking for empty directories in {}.'.format(root))
            logger.debug('root folder: {}.'.format(root))
            logger.debug('directories: {}.'.format(dirs))
            logger.debug('files: {}.'.format(files))
            for dir_ in dirs:
                try:
                    logger.info('Removing directory {}.'.format(dir_))
                    os.rmdir(os.path.join(root, dir_))
                except OSError as e:
                    logger.warning( '{}'.format(e))

def check_audio(f_path):
    for path, subdirs, files in os.walk(f_path):
        logger.info('Verifiying for Dual Audio Movies in {}.'.format(path))
        logger.debug('path folder: {}.'.format(path))
        logger.debug('directories: {}.'.format(subdirs))
        logger.debug('files: {}.'.format(files))
        for name in files:

            if any(name.endswith(ext) for ext in ('mkv', 'avi', 'mp4')):
                current_file = os.path.join(path, name)
                logger.debug('PROCESSING: {}'.format(current_file))
                audio = subprocess.check_output([
                    "mediainfo",
                    '--Inform=General;%AudioCount%', current_file
                ])
                logger.debug('File {} has {} audio stream(s).'.format(name,audio))
                language = subprocess.check_output([
                    "mediainfo",
                    '--Inform=Audio;%Language%', current_file
                ])
                logger.debug('File {} has the following language(s): {}'.format(name,language))
                if int(audio) > 1 and not fnmatch.fnmatch(language,'fr*fr'): # Dual audio
                    if os.path.join(folder_path, name) == current_file:
                        # File exist at the root
                        logger.info('Moving file {} to dualmovies.'.format(name))
                        dst = os.path.join(subfolders[5], name)
                        os.rename(current_file, dst)
                    else:
                        # File is in subfolder, move subfolder
                        logger.info('Moving directory {} to dualmovies.'.format(path))
                        shutil.move(path, subfolders[5])

                elif "Francais" in language or "fr" in language: # French movie
                    if os.path.join(folder_path, name) == current_file:
                        # File exist at the root
                        logger.info('Moving file {} to french.'.format(name))
                        dst = os.path.join(subfolders[4], name)
                        os.rename(current_file, dst)
                    else:
                        # File is in subfolder, move subfolder
                        logger.info('Moving directory {} to french.'.format(path))
                        shutil.move(path, subfolders[4])
                elif language == "":
                    logger.info('Audio language was not found for {}. It will stay in {}.'.format(name,folder_path))
                else:
                    continue

def get_command(command, extra):
    cmd = list(command)
    cmd.extend(extra)
    return cmd

filebot_cmd = [
    "/media/sdu1/home/saitoh183/filebot/filebot.sh", #ie 'C:\Program Files\Filebot\filebot.exe'

    # I recommend replacing 'fn:amc' with the path to a locally saved copy of
    # amc.groovy. 'fn:amc' downloads the script every time you run filebot.exe
    # from filebot.net.  If filebot.net is down, your file processing won't be
    # able to run. You can download the script from here:
    # https://github.com/filebot/scripts/blob/devel/amc.groovy
    '-script', 'fn:amc',

    # Save log files to same directory where this script file is located
    # '--log-file', logfiles,
    '--action', 'move',
    '--def', 'clean=y',
    '-no-xattr',
    #'--log=Fine'
]

# Declare variables
source = "/media/sdu1/home/saitoh183/private/Transfer"
importing = os.path.join(source, "importing")

subfolders = [
        os.path.join(source, "movies_cps"),
        os.path.join(source, "sonarr_tv"),
        os.path.join(source, "sickrage_tv"),
        os.path.join(source, "ufc"),
        os.path.join(source, "french"),
        os.path.join(source, "dualmovies")
]
importfolders = [
        os.path.join(importing,"clone"),
        os.path.join(source, "manage", "movies," "cps"),
        os.path.join(source, "manage", "movies", "ufc"),
        os.path.join(source, "manage", "movies", "dualmovies"),
        os.path.join(source, "manage", "movies", "french")
]

for folder_path in subfolders:
        logger.info('Processing path {}.'.format(folder_path))
        childcount = len(get_child(folder_path))
        logger.debug('Number of items in {}: {}.'.format(folder_path,childcount))
        if not childcount:
            logger.debug('There is nothing to process in {}.'.format(folder_path))
            logger.info('-----------------------------------------------------------------')
            continue


        # Convert media files and assign final import path

        if "sonarr_tv" in folder_path:
            unpack_transfer.transfer(folder_path.rsplit('/',1)[1])
            final_path = importfolders[0]

        elif "sickrage_tv" in folder_path:
            unpack_transfer.transfer(folder_path.rsplit('/',1)[1])
            final_path = importfolders[0

        elif "movies_cps" in folder_path:
            unpack_transfer.transfer(folder_path.rsplit('/',1)[1])
            final_path = importfolders[1]
            splitupmovies(folder_path)
            check_audio(folder_path)
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/cps/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\"\"\"'
            ])

        elif "ufc" in folder_path:
            final_path = importfolders[2]
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/ufc/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\"\"\"'
            ])

        elif "dualmovies" in folder_path:
            final_path = importfolders[3]
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/dualmovies/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\" -fr \"{localize.french.name} ({y})\" -en \"{ny}\"\"\"'
            ])

        elif "french" in folder_path:
            final_path = importfolders[4]
            new_filebot = get_command(filebot_cmd, [

                '--def', 'movieFormat=/media/sdu1/home/saitoh183/private/Transfer/manage/movies/french/{plex[1..-1]}',
                '--def', 'ut_label=Movies',
                '--lang', 'fr',
                '--def', 'exec= \"\"python /media/sdu1/home/saitoh183/private/script/rclone_move.py -fp \"{file}\"\"\"'
            ])


        clean_files(folder_path)
        logger.info('Starting converstion in {}'.format(folder_path))
        convert_files(folder_path)
        if any(re.findall(r'ufc|french|dualmovies|cps', final_path, re.IGNORECASE)):
            childcount = len(get_child(final_path))
            if childcount:
                new_filebot.extend(['-non-strict', final_path])
                logger.info('Filebot Processing Started...')
                FB = subprocess.Popen(new_filebot,stdout=subprocess.PIPE).communicate()[0]
                logger.info('Filebot Processing Ended...')
                logger.debug('{}'.format(FB))
                logger.info('-----------------------------------------------------------------')
Steps of script:

The script will loop through each of the folders in my subfolder variable. but only the folders containing movies use FB. So the first stop is movies_cps

1 - File is downloaded to .../Transfer/movies_cps
2 - Files are unpacked/and separated into each their own subfolder
3 - They are checked for their audio and
- if they contain English audio only, they stay in movies_cps
- if they contain French audio only, they go .../Transfer/french
- if they contain DualAudio audio only, they go .../Transfer/dualmovies

4 - Script then removes all non-essential files (samples, nfo, etc..)
5 - I then run convert_files function which launches sickbeard_mp4_automator on movies_cps (or whichever folder folder_path is equal to)
6 - SickMp4Auto output's the file to the final_path variable which would equal to /media/sdu1/home/saitoh183/private/Transfer/manage/movies/(cps,french,dualmovies)

7 - Next I check if there is content in the final_path and if there is, Run Filebot on final_path.

Now I added logging to try and capture the error but my logging comes up empty. I added an info line to see when filebot is called and when it ends and in the log, those lines are both there but nothing in between. I also enabled logs in the filebot command and set it to ALL but nothing was generated either.

So then if I take one of the files that are currently in final_path and move them back to their original location. Then I run the script again, my function convert_files will not convert the file cuz it has already been converted and it will move it directly to final_path, then the script will move on to Filebot, and filebot will do it things and generate logs and the files will be renamed.

So I don't know when on the first pass filebot is failing and doesn't generate any sort of logs but maybe you would have an idea or maybe a way for me to see what is going on with FB on the first attempt.

Hi Thanks for you time and good topic creating this.

I have adopted the script to my PATHs and preferences no a Windows Environment but I'm getting an error for unpack_transfers

Code: Select all

c:\Convert>rclone_move.py
Traceback (most recent call last):
  File "C:\Convert\rclone_move.py", line 13, in <module>
    import unpack_transfer
ImportError: No module named unpack_transfer
Thanks in Advance.
saitoh183
Posts: 112
Joined: 02 Oct 2012, 16:59

Re: Filebot in python

Post by saitoh183 »

Unpack_transfer is another script I called to move content that I Unrar. You can remove the line import unpack_tansfer
Post Reply