Seedbox/Automation Issue

Running FileBot from the console, Groovy scripting, shell scripts, etc
Post Reply
JimmyB4403
Posts: 20
Joined: 28 Oct 2014, 14:07

Seedbox/Automation Issue

Post by JimmyB4403 »

I just started using a seedbox, so while prior to this I've had the AMC script running after a file changes state in uTorrent, I've now had to make some modifications for the new setup. In doing so, I ran into an issue, and I'm hoping someone might be able to give some advice.

I'm using Resilio Sync to automatically transfer the files to my computer as they complete on the seedbox. I also modified the uTorrent run line so that I could use it in the command line, and turned it into a .bat file and have a scheduled task set to run every 15 minutes.

The probably comes into play when Resilio is in the process of downloading a file, but its not yet fully complete when the scheduled task starts to run. If the file that is being transferred is a .rar with several archives, a folder is created in the directory and as each archive is downloaded it gets added to the folder. IF the scheduled task runs while the entire file is not yet completed, it will try to process what is out there and fail out. At first I thought to myself, no problem, because it will pick that main .rar up the next time the scheduled task runs (which it did). The problem is that some of the other files get added to the --def excludeList file and the video essentially gets corrupted because it didn't process all of the archived parts since some where excluded (at least that's what it appears to me). Have you had anyone run into this before, and/or do you have ideas on what I can do to prevent the file from looking like it processed correctly, but actually being junk?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Seedbox/Automation Issue

Post by rednoah »

The amc script should not be called on files that are still being transferred.

There are many solutions. One would be to call the amc script only after everything has been synced. Another would be having a special folder for transferring files (e.g. Download) and then have files moved to another folder (e.g. Complete) for completed downloads which you then call the amc script on periodically.
:idea: Please read the FAQ and How to Request Help.
JimmyB4403
Posts: 20
Joined: 28 Oct 2014, 14:07

Re: Seedbox/Automation Issue

Post by JimmyB4403 »

I get it, and I wasn't implying it was a shortcoming of the script, I know its a shortcoming of my setup.

Based on your comments, I have a few questions though:
1)
There are many solutions. One would be to call the amc script only after everything has been synced.
How would I accomplish this without running the script manually every time something finished? Is this through Task Scheduler? I know there are options for triggers other than time, but I don't know how to create one for when a new folder is created and fully sync'd.

2)
Another would be having a special folder for transferring files (e.g. Download) and then have files moved to another folder (e.g. Complete) for completed downloads which you then call the amc script on periodically.
That's how the folders are setup on my bittorrent devices, but I think this is a byproduct of the application I am using. Resillio Sync quite literally just makes sure that the completed folder on my seedbox matches a local folder on my desktop. As things transfer over, a folder is created and the items are dropped in side as they complete. There is no option to move them as the status changes since it should mirror the drive on the seedbox. This was a better alternative to me than FileZilla which required me to manually transfer files from my seedbox to my desktop. I was unaware of any automation available through FileZilla. Is there a different application you'd recommend to accomplish this?

I'm sure I'm not the first person doing this, so I'm hoping that someone can give me a bit of advice :)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Seedbox/Automation Issue

Post by rednoah »

1.
That depends on the tools you're using. I'd write a simple script that first syncs files and then calls filebot, and then have the scheduler call it once a day in the middle of the night:

Code: Select all

rsync ...
filebot ...

2.
The tool you're using might have some option to download files to a temporary location and only move them to the target destination when the file is complete (rsync has such an option). Some tools might allow you to use some temporary file name (e.g. xyz123.filepart) and rename the file on completion, then you could use the --def ignore option to ignore such files.

I'd probably just use rsync on Linux and Mac, and on Windows use rsync via WSL. I can't really recommend any GUI tools for this task because I wouldn't use them myself. WinSCP should be pretty good though. It has some scripting support anyway.
:idea: Please read the FAQ and How to Request Help.
noze2000
Donor
Posts: 22
Joined: 08 Feb 2015, 20:08

Re: Seedbox/Automation Issue

Post by noze2000 »

HI ,

This is my setup.

On seedbox:
  • Deluge downloads files - sorts downloads in "download complete" folders (e.g. movies, series etc)
  • Once torrent complete, a script will, based on download complete folder, create new copy of file using cp -l in a separate directory called "hard links" (for clarity).
These hard links do not take up extra space on the box and one can delete either copy of the file without affecting the other. File only is removed only once both links are deleted. This enables continued seeding if one were so inclined

On home box, every x min run script that:
  • rsync to seedbox to copy anything in the hard link directory into the relevant folders (media/movies/series/other). use --remove-source as that will remove the link. The file will be there until both rsync removes one link and and removal of torrent removes the second - in what ever order
  • Run filebot on the relevant folders
Would be happy to hear other setups :)
plittlefield
Posts: 101
Joined: 09 Mar 2014, 19:15

Re: Seedbox/Automation Issue

Post by plittlefield »

sshfs to the seedbox, stop seeding at ratio 2, filebot script which is run manually... not ideal, but reliable and works for me :-)
yodaspowart
Posts: 16
Joined: 02 Jun 2017, 01:04

Re: Seedbox/Automation Issue

Post by yodaspowart »

Seedbox automation with Mac running Resilio Sync, Hazel and Filebot
I discovered that the synced files are hidden before completly downloaded in a .sync folder if they are video files. Don't use FOLDERS when syncing - process doesn't work with Plex ready directories.

Setup uses:
Seedbox requires: Torrent client, Filebot, Resilo Sync
Apple computer (Mac Mini) requires: Hazel, Resilio Sync and Filebot to be installed.
Hazel: https://www.noodlesoft.com


Meta-Data issue:
Im predominately using Plex for meta data so if you have cover art and dedicated extras this might not be a good solution for you.

Step 1: (on Seedbox)
On the Seedbox I have Filebot scripts running on torrent completion for:
1) Transmission (Manual torrents)
2) ruTorrent for RSS feeds
Both export on completion to a Filebot script that organises in to dedicated directories on the seedbox.
This is where I name the files without folders ie Film (1999) or TV Show (2000) -> Season 1 etc.
Filebot Script I am using is --action copy so I can continue seeding (not using Symlink)

Step 2: (on Seedbox)
Setup Resilio sync folder for all the Plex folders where Filebot has added the files. Have these set up as default Sync (not selective)

Step 3: (on Apple computer)
1) Add the same folders following names from Seedbox for the ones I want to sync to Apple Mac Mini.
2) Add these Resilio directories/folders to Hazel
Image
Image
Image

3 ) When the file completes it will become visible and Hazel with apply the Filebot Script to the file/files. Resilio moves the file/files from the hidden directory/folder ".sync" and places it ready for Hazel to pickup. Hazel basically watches changes in folders.

4) As the folder is set up for Syncing (not selective sync) on the Apple Mac: The files moved by Filebot are also deleted off the Seedbox automatically by Resilio Sync.

5) I turned my Archive Sync OFF on both Seedbox and remote Apple Mac Mini
Image
You can also limit the archive days instead in Power user setting with:
Image
"Alternatively, to limit the length of time (number of days) files are kept in the archive, change the value of "sync_trash_ttl" (the default is 30 days)"
https://forum.resilio.com/topic/31562-d ... ment=90503

4) Filebot Scripts are:
Seedbox - ruTorrent:

Code: Select all

#!/bin/bash

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_CTYPE="en_US.UTF-8"

TORRENT_PATH=$1
TORRENT_NAME=$2
TORRENT_LABEL=$3

# Subtitle language
SUBLANG=en
SKIP_EXTRACT=n
MUSIC=y

filebot -script /opt/filebot/scripts/amc.groovy \
--output "$HOME/media" \
-non-strict --encoding utf8 --log all --log-file amc-rtorrent.log --action copy --conflict auto \
--def artwork=false ut_kind=multi "ut_dir=$TORRENT_PATH" "ut_title=$TORRENT_NAME" subtitles=$SUBLANG \
--def "movieFormat={fn =~ /2160p|4K|4k|UHD/ ? 'Movies 4K' : fn =~ /3D|3d|HSBS|3dhsbs|H-SBS|3dhou|H-OU/ ? 'Movies 3D' : 'Movies'}/{n} ({y}) - {s3d} {hd} {vc} {vf}{' CD'+pi}{'.'+lang}" \
--def "seriesFormat={fn =~ /2160p|4K|4k|UHD/ ? 'TV Shows 4K' : fn =~ /1080p/ ? 'TV Shows HD' : 'TV Shows'}/{n} - {episode.special ? 'S00E'+special.pad(2) : s00e00} - {hd} - {vc} - {vf} - {t.replaceAll(/[\`´‘’ʻ]/, /'/).replaceAll(/[!?.]+$/).replacePart(', Part \$1')}{'.'+lang}" \
--def plex=http://seedboxIP:seedboxPort/library/sections/all/refresh?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx \
extractFolder="$HOME/files/_extracted" music=$MUSIC skipExtract=$SKIP_EXTRACT &
Seedbox - Transmission:

Code: Select all

#!/bin/bash

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_CTYPE="en_US.UTF-8"

TORRENT_PATH="$TR_TORRENT_DIR/$TR_TORRENT_NAME"
TORRENT_NAME="$TR_TORRENT_NAME"
TORRENT_LABEL="N/A"

# Subtitle language
SUBLANG=en
SKIP_EXTRACT=n
MUSIC=y

filebot -script /opt/filebot/scripts/amc.groovy \
--output "$HOME/media" \
-non-strict --encoding utf8 --log all --log-file amc-transmission.log --action copy --conflict auto \
--def artwork=false ut_kind=multi "ut_dir=$TORRENT_PATH" "ut_title=$TORRENT_NAME" subtitles=$SUBLANG \
--def "movieFormat={fn =~ /2160p|4K|4k|UHD/ ? 'Movies 4K' : fn =~ /3D|3d|HSBS|3dhsbs|H-SBS|3dhou|H-OU/ ? 'Movies 3D' : 'Movies'}/{n} ({y}) - {s3d} {hd} {vc} {vf}{' CD'+pi}{'.'+lang}" \
--def "seriesFormat={fn =~ /2160p|4K|4k|UHD/ ? 'TV Shows 4K' : fn =~ /1080p/ ? 'TV Shows HD' : 'TV Shows'}/{n} - {episode.special ? 'S00E'+special.pad(2) : s00e00} - {hd} - {vc} - {vf} - {t.replaceAll(/[\`´‘’ʻ]/, /'/).replaceAll(/[!?.]+$/).replacePart(', Part \$1')}{'.'+lang}" \
--def plex=http://seedboxIP:seedboxPort/library/sections/all/refresh?X-Plex-Token=xxxxxxxxxxxxxxxxxxxx \
extractFolder="$HOME/files/_extracted" music=$MUSIC skipExtract=$SKIP_EXTRACT &
Remote Apple Mac Mini - TV_FilebotPlex.sh

Code: Select all

filebot -script fn:amc "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/media/TV Shows" -non-strict --def music=y --action move --conflict auto --def subtitles=eng,pol,por --def "movieFormat={fn =~ /3D/ ? '3D Movies' : 'Movies'}/{n} ({y}) - {hd} {vc} {vf} {' CD'+pi}{'.'+lang}" --def seriesFormat="/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot/TV Shows/{n} - {s00e00} - {hd} {vc} {vf} - {t}{'.'+lang}" --def clean=y --output "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot" --log-file "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot/amc.log"
Remote Apple Mac Mini - Movies_FilebotPlex.sh

Code: Select all

filebot -script fn:amc "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/media/Movies" -non-strict --def music=y --action move --conflict auto --def subtitles=eng,pol,por --def "movieFormat={fn =~ /3D/ ? '3D Movies' : 'Movies'}/{n} ({y}) - {hd} {vc} {vf} {' CD'+pi}{'.'+lang}" --def seriesFormat="/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot/TV Shows/{n} - {s00e00} - {hd} {vc} {vf} - {t}{'.'+lang}" --def clean=y --output "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot" --log-file "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot/amc-movies.log"
Remote Apple Mac Mini - Movies3D_FilebotPlex.sh

Code: Select all

filebot -script fn:amc "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/media/Movies 3D" -non-strict --def music=y --action move --conflict auto --def subtitles=eng,pol,por --def "movieFormat={fn =~ /3D|3d|HSBS|3dhsbs|H-SBS|3dhou|H-OU/ ? '3D Movies' : 'Movies'}/{n} ({y})/{n} ({y}) - {s3d} {hd} {vc} {vf} {' CD'+pi}{'.'+lang}" --def seriesFormat="/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot/TV Shows/{n} - {s00e00} - {hd} {vc} {vf} - {t}{'.'+lang}" --def clean=y --output "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot" --log-file "/Volumes/Drobo/Resilio-Sync/UltraSeedbox/plexFilebot/amc-movies3d.log"
Post Reply