Automated Media Center

Running FileBot from the console, Groovy scripting, shell scripts, etc
Locked
User avatar
rednoah
The Source
Posts: 23459
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Automated Media Center

Post by rednoah »

Overview

The amc script will automatically organize your media. It's a simple command-line tool that can be called manually or by other tools on one or many files. It'll fully automatically organize your TV shows and movies and is smart enough to detect what is what. The script will only touch video / subtitle files and will leave everything else as is.
  1. Unrar archives
  2. Group movies and episodes and then process them accordingly
  3. Fetch subtitles and transcode them to SubRip / UTF-8
  4. Organize movies, episodes and music files into a new file and folder structure according to the {plex.id} format naming scheme:
    • Movies will be sorted into {output}/Movies/Name (Year) {tmdb-id}/Name (Year).ext
    • Episodes will be sorted into {output}/TV Shows/Name {tmdb-id}/Season N/Name - S00E00 - Title.ext
    • Music will be sorted into {output}/Music/Artist/Album/Artist - Title.ext
  5. Fetch artwork and generate .nfo files
  6. Notify Plex, Kodi or Emby to refresh the video library
  7. Send a report about newly added files via Email or PushBullet
There's setup instructions for µTorrent, qBT, rTorrent, Deluge, Transmission, Vuze, JDownloader and Automator. Since filebot is a simple command-line tool you can make it work with any 3rd party tool that can run external programs, or monitor folders for changes and call filebot on newly added files, or just call filebot every once in a while with cron.


Setup Instructions

1. Set Location of Downloaded Files and make sure to move complete downloads into a separate folder.

2. Set Run Program to make your download program call filebot on newly completed downloads. Please follow the setup instructions for your specific download program.

e.g. qBT ➔ Options ➔ Downloads ➔ Run an external program on torrent completion:

Shell: Select all

filebot -script fn:amc --output "X:/Media" --action duplicate --conflict skip -non-strict --log-file amc.log --def excludeList=amc.excludes --def unsorted=y music=y artwork=y "ut_label=%L" "ut_title=%N" "ut_kind=multi" "ut_dir=%F"
e.g. µTorrent ➔ Preferences ➔ Run Program ➔ Run this program when torrent finishes:

Shell: Select all

filebot -script fn:amc --output "X:/Media" --action duplicate --conflict skip -non-strict --log-file amc.log --def unsorted=y music=y artwork=y "ut_label=%L" "ut_state=%S" "ut_title=%N" "ut_kind=%K" "ut_file=%F" "ut_dir=%D"
3. Change --output to where you want to copy / organize your files.

4. Done! This took take less than a minute. Now wait and enjoy how things will be organized fully automatically.

ScreenshotScreenshot


Run from the command-line

Want to try it out first? Running directly from command-line is even easier and if you set --action test you can do a dry-run and see if everything gets matched up correctly:

Shell: Select all

filebot -script fn:amc "/path/to/input" --output "/path/to/output" --action duplicate -non-strict --log-file amc.log --def excludeList=amc.txt
  • If you run this script manually, you can set --mode interactive to enable console-interactive mode.
  • If you run this script automatically in intervals every hour or every day, then you must set --def excludeList or you will get banned.

Options

--def music=y Process music files
--conflict skip Never overwrite existing files
--conflict auto Do overwrite existing files if new files are better
--def subtitles=en,de,fr Download subtitles
--def artwork=y Fetch artwork and generate nfo files
--def plex=host:token Rescan Plex library (see authentication token)
--def kodi=host:port Rescan Kodi library
--def emby=host:apikey Rescan Emby library
--def jellyfin=host:apikey Rescan Jellyfin library
--def pushbullet=apikey Send report via PushBullet
--def pushover=userkey Send notification via Pushover
--def discord=webhook Send notification via Discord (NOTE: Linux and macOS users may prefer the send-discord-log shell script to capture and send console output)
--def gmail=username:password Send report via Gmail (see App Password)
--def mail=host:port:from:username:password Send report via SMTP (NOTE: Linux and macOS users may prefer the send-email-log shell script to capture and send console output)
--def mailto=email Send Email report to this address
--def reportError=y Send Email report on error
--def storeReport=/path/to/folder Save reports
--def extractFolder=/path/to/folder Extract archives to this folder
--def skipExtract=y Skip archives
--def deleteAfterExtract=y Delete archives after extraction
--def clean=y Delete left behind empty folders and clutter files
--def exec="echo {quote f}" Run program on newly processed files (NOTE: use {quote f} to correctly escape file paths that contain spaces, parentheses, apostrophes, etc)
--def unsorted=y Process unidentifiable files (as opposed to skipping them) and move them to the Unsorted folder
--def ignore=regex Ignore file paths
--def minLengthMS=0 Skip short videos (in milliseconds; 10 minutes by default)
--def minFileSize=0 Skip small files (in bytes; 50 MB by default)
--def minFileAge=3 Skip recently created files (in days; 0 by default)
--def excludeLink=y Skip symlinks and hardlinks where the link count is more than 1
--def excludeList=amc.txt Keep a list of already processed file paths to be skipped in future runs (NOTE: enable this if you process the same folder repeatedly)


Change how files will be organized and renamed

This script uses multiple format expressions, one for each content type. You can override the default naming schemes via the --def movieFormat, seriesFormat, animeFormat and musicFormat options. Relative paths will be resolved against --output and you can use absolute paths if you want to use multiple drives.

Shell: Select all

--def movieFormat="M:/Movies/{ ~plex }" seriesFormat="S:/Series/{ ~plex }" animeFormat="J:/Anime/{ ~plex }" musicFormat="T:/Audio/{ ~plex }" unsortedFormat="U:/Unsorted/{ fn }"
  • Start with the {plex} format and then modify your custom format bit by bit to fit your needs. Make sure to prototype your custom format with the Format Editor and not on the command-line.
  • Use @file syntax to read complex arguments from a text file.

Change database for each content type

This script uses multiple databases, so the simple --db option is not supported. You can override the default database via --def movieDB, seriesDB, animeDB and musicDB options for each content type.

Shell: Select all

--def movieDB=TheMovieDB seriesDB=TheMovieDB::TV animeDB=TheMovieDB::TV musicDB=ID3

Common Problems

  • Location of Downloaded Files (input and --output folder) must not end with backslash \ the final backslash \ will escape the subsequent double-quote " and mess up everything. Please use / as separator and not \ because the latter is also used to escape characters. (see Cmdline and Argument Passing)
  • You can (and should) force Movie / Series / Anime mode or force ignore files via labels, e.g. label as Movie to force Movie mode, Series to force TV Series mode, Anime to force Anime mode, or other to ignore all files. Alternatively, standard folder names such as Movies / TV Shows / Anime may also be used to force a specific mode.
  • Anime are inherently TV Shows, so Anime will be detected as Anime and processed in Anime mode if and only if the files have Anime-like characteristics.
  • You must not use your input folder %D as --output folder.
  • You must not use the filesystem root (e.g. X:\ or /) as input or --output folder.
  • You must restart your computer after installing FileBot to propagate the %PATH% environment variable to all running processes. (e.g. uT won't be able to find the filebot command if FileBot was not yet installed and added to the %PATH% at the time when uT was launched)

Advanced Fine-Tuning

You can (and should) force Movie / Series / Anime mode by setting --def ut_label, and you can further define your own --filter rules to fine-tune matching to your exact needs, and avoid mismatches. For example, if one show is confused with another due to bad naming or missing data you can simply exclude the bad one, or if you only process recently aired episodes, then you could exclude episodes that have not aired recently from matching.

e.g. if you only ever process recently aired episodes (i.e. not movies, not episodes from previous seasons, etc) then you can virtually guarantee 100% auto-detection accuracy by adding simple filter rules such as these:

Shell: Select all

--filter "age < 5" --def "ut_label=TV"
e.g. if you're a typical user, and find a simple age filter too restrictive, then this slightly augmented age filter rule might be the magic bullet that fixes all your problems, by applying the age filter only on newly aired episodes of currently airing shows (but not older episodes of currently airing shows) and otherwise accepting all episodes of shows that are not currently airing:

Shell: Select all

--filter "age < 5 || 5 <= model.age.min()" --def "ut_label=TV"
--filter rules are extremely flexible so more complex scenarios, such as applying certain filters only for certain labels or files, is fully supported.


Troubleshooting

If something is not working, please read the Notes above, then run the command manually in a new console window to see what is happening. You CANNOT run the command without replacing the VARIABLES. The variables have to be replaced with actual values first.

e.g.

Shell: Select all

filebot -script fn:amc --output "D:/Organized Media" --log-file amc.log --action duplicate --conflict override -non-strict --def "ut_dir=D:\Media\Avatar" "ut_file=" "ut_kind=multi" "ut_title=Avatar" "ut_label=" "ut_state=5"

:idea: Use @file syntax to refactor command-line arguments (not including variables or substitution patterns) into external text files:

Shell: Select all

filebot @X:/args.txt --def "ut_label=%L" "ut_state=%S" "ut_title=%N" "ut_kind=%K" "ut_file=%F" "ut_dir=%D"

:idea: Configure filebot to dump command-line arguments to the log file before processing so you can see how filebot is called by 3rd party programs:

Shell: Select all

filebot -script fn:properties --def net.filebot.logging.debug=INFO

Logs

Please check the log once in a while:

Shell: Select all

filebot -script fn:logs
:idea: The --log-file is located in the application data folder by default. Please read the manual for details.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 23459
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

[Transmission] Setup for Transmission on Linux or macOS

Post by rednoah »


Button Setup for Transmission on Linux or macOS


1. Create On Torrent Completion Script

Create a On Torrent Completion shell script transmission-postprocess.sh that'll call filebot like this and make sure to chmod +rx it:

Shell: Select all

curl -O https://raw.githubusercontent.com/filebot/plugins/master/bash/transmission-postprocess.sh
chmod +rx transmission-postprocess.sh

2. Set Transmission to call our shell script on completion

Setup via GUI on Mac:
Open TransmissionPreferencesTransfersManagement and set it to call our script when download completes:

Screenshot

Setup via Terminal on Linux:
Open ~/.config/transmission/settings.json and set it to call our shell script on download complete:

json: Select all

"script-torrent-done-enabled": true, 
"script-torrent-done-filename": "/path/to/transmission-postprocess.sh",
Transmission Configuration Files:
https://github.com/transmission/transmi ... n-Files.md


Known Issues:
  • Transmission does not pass along TORRENT_LABEL which can affect detection accuracy
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 23459
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

[Deluge] Setup for Deluge on Linux or macOS

Post by rednoah »


Button Setup for Deluge on Linux or macOS


Create a shell script deluge-postprocess.sh that'll call filebot and make sure to chmod +rx it:

Shell: Select all

curl -O https://raw.githubusercontent.com/filebot/plugins/master/bash/deluge-postprocess.sh
chmod +rx deluge-postprocess.sh
Open Deluge ► PreferencesPlugins and activate the Execute plugin. Then go to PreferencesExecute, add a new Torrent Complete event and set it to call our newly created deluge-postprocess.sh shell script:

Shell: Select all

/path/to/deluge-postprocess.sh

Known Issues:
  • Scripts run as the deluge user so environment variables like $HOME may not work as expected
  • The Deluge Execute plugin does not pass along TORRENT_LABEL which can affect detection accuracy


:!: These instructions DO NOT apply to Microsoft Windows. Integrating FileBot with Deluge for Windows is possible but difficult. Using qBittorrent instead is therefore generally recommended.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 23459
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

[rTorrent] Setup for rTorrent on headless Linux

Post by rednoah »


Setup for rTorrent on headless Linux


1. Run this command to create a shell script rtorrent-postprocess.sh that'll call filebot and the amc script:

Shell: Select all

curl -O https://raw.githubusercontent.com/filebot/plugins/master/bash/rtorrent-postprocess.sh
2. Run this command to make sure that the script is executable:

Shell: Select all

chmod +rx rtorrent-postprocess.sh
3. Run this command to edit ~/.rtorrent.rc and add an event.download.finished hook:

Shell: Select all

echo 'system.method.set_key=event.download.finished,filebot,"execute={'`pwd`/rtorrent-postprocess.sh',$d.get_base_path=,$d.get_name=,$d.get_custom1=}"' >> ~/.rtorrent.rc
:idea: The echo command above will append the following line to your ~/.rtorrent.rc file:

Properties: Select all

system.method.set_key=event.download.finished,filebot,"execute={/path/to/rtorrent-postprocess.sh,$d.get_base_path=,$d.get_name=,$d.get_custom1=}"

Known Issues:
  • The latest revision of rtorrent seems to have introduced breaking changes in how ~/.rtorrent.rc works. Read this thread for details.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 23459
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

[qBT] Setup for qBittorrent on Windows, Linux and macOS

Post by rednoah »


Button Setup for qBittorrent on Windows, Linux and macOS


Instructions for Windows:

OptionsDownloadsRun an external program on torrent completion

Shell: Select all

filebot -script fn:amc --output "X:/Media" --action duplicate --conflict skip -non-strict --log-file amc.log --def excludeList=amc.excludes --def unsorted=y music=y artwork=y "ut_label=%L" "ut_title=%N" "ut_kind=multi" "ut_dir=%F"

Instructions for Linux and macOS:

OptionsDownloadsRun an external program on torrent completion

Shell: Select all

filebot -script fn:amc --output "/path/to/media" --action duplicate --conflict skip -non-strict --log-file amc.log --def excludeList=amc.excludes unsorted=y music=y artwork=y "ut_label=%L" "ut_title=%N" "ut_kind=multi" "ut_dir=%F"
:idea: On macOS, please use /usr/local/bin/filebot instead of filebot because /usr/local/bin is not in the system $PATH by default.

Screenshot


:idea: You may need to use the absolute path to the executable (e.g. /usr/local/bin/filebot) if you get filebot: command not found errors.


:idea: Use @file syntax to refactor command-line arguments (not including variables or substitution patterns) into external text files:

Shell: Select all

filebot @/path/to/args.txt --def "ut_label=%L" "ut_title=%N" "ut_kind=multi" "ut_dir=%F"

:idea: If you are using qBittorrent 3.* then the %variables may need to be adjusted:

Shell: Select all

--def "ut_label=none" "ut_title=%n" "ut_kind=multi" "ut_dir=%f"
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 23459
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Help

Post by rednoah »

If you're having any issues...

1. Make a NEW thread
2. Include log output
3. Include file paths when reporting mismatches


:arrow: Please read How to Request Help for details.
:idea: Please read the FAQ and How to Request Help.
Locked