How can I get AMC to process MKV files and BDMV directories in one script

Any questions? Need some help?
Post Reply
hansooloo
Posts: 30
Joined: 06 Feb 2016, 14:58

How can I get AMC to process MKV files and BDMV directories in one script

Post by hansooloo »

I download packages that, when extracted, result in either of the 2:
  1. a folder with an MKV file in it
  2. a folder with a Blu-ray BDMV folder in it
Today, my script below works fine for folders with MKV files in them, but the same script picks up ONLY 1 file from inside a BDMV structure and copies it over.

How can I use AMC script to handle BOTH of the scenarios above?

Code: Select all

/usr/local/FileBot/filebot.sh                               \
  --log-file /mnt/JDownloader/JDownloaderPostProcess.log    \
  -extract                                                  \
  --action copy                                             \
  -script fn:amc                                            \ 
  --def ut_dir=$UT_DIR                                      \
        ut_kind=multi                                       \
        ut_title=$UT_TITLE                                  \
        movieFormat=@/usr/local/JDownloader/FormatMovie.txt \
        seriesFormat=@/usr/local/JDownloader/FormatTV.txt   \
        clean=y minFileSize=0                               \
  --output /mnt/JDownloader/_02.Format_
FormatMovie.txt

Code: Select all

Movies/{collection.replaceAll(' Collection', '').replaceFirst(/^(?i)(The|A)\s(.+)/, /$2/)+'/'}{n.replaceAll(/:|]/, ", ").replaceFirst(/^(?i)(The|A)\s(.+)/, /$2, $1/)} ({y})/{n.replaceAll(/:|]/, ", ").replaceFirst(/^(?i)(The|A)\s(.+)/, /$2, $1/)} ({y})
FormatTV.txt

Code: Select all

TV/{n}/Season {s.pad(2)}/{n} - {S00E00} - {t}
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by rednoah »

Logs?

If you're processing disk folders, then the amc script will process the disk folder as a single unit, and process the BDMV structure as a whole.

You seem to say that the latter is not working correctly. Why? IDK. The logs will tell. Maybe the structure isn't recognized for some reason. Maybe you're passing a folder that's already inside the structure, instead of the structure root folder.
:idea: Please read the FAQ and How to Request Help.
hansooloo
Posts: 30
Joined: 06 Feb 2016, 14:58

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by hansooloo »

I am using the following file structure to demonstrate the issue (actual BDMV behaves the same way):

Code: Select all

jd@plex:/mnt/JDownloader/_01.Download_$ find Star\ Wars\,\ A\ New\ Hope/
Star Wars, A New Hope/
Star Wars, A New Hope/CERTIFICATE
Star Wars, A New Hope/BDMV
Star Wars, A New Hope/BDMV/MovieObject.bdmv
Star Wars, A New Hope/BDMV/index.bdmv
Star Wars, A New Hope/BDMV/STREAM
Star Wars, A New Hope/BDMV/STREAM/00101.m2ts
Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
I create the RAR volumes like so:

Code: Select all

jd@plex:/mnt/JDownloader/_01.Download_$ rar a -r -vsize=50m -m0 -mc- sw-ep4/sw-ep4 Star\ Wars\,\ A\ New\ Hope/

RAR 5.30 beta 2   Copyright (c) 1993-2015 Alexander Roshal   4 Aug 2015
Trial version             Type RAR -? for help

Evaluation copy. Please register.

Creating archive sw-ep4/sw-ep4.rar

Adding    Star Wars, A New Hope/BDMV/MovieObject.bdmv                 OK
Adding    Star Wars, A New Hope/BDMV/index.bdmv                       OK
Adding    Star Wars, A New Hope/BDMV/STREAM/00101.m2ts                OK
Adding    Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Calculating the checksum

Creating archive sw-ep4/sw-ep4.part2.rar

...       Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Calculating the checksum

Creating archive sw-ep4/sw-ep4.part3.rar

...       Star Wars, A New Hope/BDMV/STREAM/00100.m2ts                OK
Adding    Star Wars, A New Hope/BDMV/STREAM                           OK
Adding    Star Wars, A New Hope/CERTIFICATE                           OK
Adding    Star Wars, A New Hope/BDMV                                  99%
Calculating the checksum
Done
And finally, when I run FileBot on the sw-ep.partX.rar files, I get the following:

Code: Select all

/usr/local/FileBot/filebot.sh                               \
  --log-file /mnt/JDownloader/JDownloaderPostProcess.log    \
  -extract                                                  \
  --action copy                                             \
  -script fn:amc                                            \ 
  --def ut_dir=$UT_DIR                                      \
        ut_kind=multi                                       \
        ut_title=$UT_TITLE                                  \
        movieFormat=@/usr/local/JDownloader/FormatMovie.txt \
        seriesFormat=@/usr/local/JDownloader/FormatTV.txt   \
        clean=y minFileSize=0                               \
  --output /mnt/JDownloader/_02.Format_
Run script [fn:amc] at [Sun Jul 22 18:26:41 EDT 2018]
Parameter: ut_dir = /mnt/JDownloader/_01.Download_/sw-ep4
Parameter: ut_kind = multi
Parameter: ut_title = sw-ep4
Parameter: movieFormat = Movies/{collection.replaceAll(' Collection', '').replaceFirst(/^(?i)(The|A)\s(.+)/, /$2/)+'/'}{n.replaceAll(/:|]/, ", ").replaceFirst(/^(?i)(The|A)\s(.+)/, /$2, $1/)} ({y})/{n.replaceAll(/:|]/, ", ").replaceFirst(/^(?i)(The|A)\s(.+)/, /$2, $1/)} ({y})
Parameter: seriesFormat = TV/{n}/Season {s.pad(2)}/{n} - {S00E00} - {t}
Parameter: clean = y
Parameter: minFileSize = 0

----------------------------------- <README> -----------------------------------

Dear Linux user, jd,

FileBot currently only officially supports Windows 10 and macOS because these platforms make it easy to sell FileBot via their respective app stores. Unfortunately, selling software on Ubuntu, Debian, Red Hat, SUSE, Synology NAS, QNAP NAS, etc is not as easy and not really worth the effort considering the small number of users willing to support the project. This Patreon is an experiment to see how many users are willing to financially contribute towards freely available donation-supported new releases on these platforms.

Please support FileBot for Linux with a pledge of $1 per release on Patreon:

=> https://www.patreon.com/filebot

If every other Linux user were to pledge a small amount of money to the FileBot project, then you could easily fund the project ten times over. Please financially support the software you use and rely on, be it FileBot or any other tool. It's just polite, and it helps the ecosystem.

Thank you,
The FileBot Team

---------------------------------- </README> -----------------------------------

Read archive [sw-ep4.part1.rar] and extract to [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4]
Extracting files [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/MovieObject.bdmv, /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/index.bdmv, /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts, /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts, /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM, /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/CERTIFICATE, /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV]
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts
BDMV [series: Star, movie: Star Wars (1977)]
Exclude Series: Star
00100.m2ts [series: stream, movie: Star Wars (1977)]
Exclude Series: stream
00100.m2ts [series: stream, movie: Star Wars (1977)]
Exclude Series: stream
00101.m2ts [series: stream, movie: Star Wars (1977)]
Exclude Series: stream
00101.m2ts [series: stream, movie: Star Wars (1977)]
Exclude Series: stream
Group: [tvs:null, mov:star wars 1977] => [BDMV, 00100.m2ts, 00100.m2ts, 00101.m2ts, 00101.m2ts]
Rename movies using [TheMovieDB]
Auto-detect movie from context: [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts]
Auto-detect movie from context: [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts]
Auto-detect movie from context: [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV]
Skipped [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV] because [/mnt/JDownloader/_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)] already exists
Skipped [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts] because [/mnt/JDownloader/_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977).m2ts] already exists
Skipped [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts] because [/mnt/JDownloader/_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977).m2ts] already exists
Processed 0 files
Clean temporary extracted files
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/MovieObject.bdmv
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/index.bdmv
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/CERTIFICATE
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV
Delete /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1
Finished without processing any files
Failure (°_°)

Notice where it says

Code: Select all

Skipped [/mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV] because [/mnt/JDownloader/_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)] already exists
The resultant formatted output looks like the following (I was expecting to see the BDMV folder structure intact when copied to the Output folder AND was NOT expecting to see _another_ m2ts file with the name of the movie, since there are ONLY 2 m2ts files in the archive):

Code: Select all

jd@plex:/mnt/JDownloader/_01.Download_$ find ../_02.Format_/Movies/Star\ Wars/
../_02.Format_/Movies/Star Wars/
../_02.Format_/Movies/Star Wars/Star Wars (1977)
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977).m2ts
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)/STREAM
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)/STREAM/00101.m2ts
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)/STREAM/00100.m2ts
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)/index.bdmv
../_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)/MovieObject.bdmv
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by rednoah »

BAD: Since you're explicitly telling FileBot to process these files, it'll do just that, and won't be able to detect which files happen to belong to the same disk folder set:

Code: Select all

Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00100.m2ts
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2ts
Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope/BDMV/STREAM/00101.m2t

OK: You must pass in the root folder, so FileBot can figure things out itself as it traverses the folder structure:

Code: Select all

Input: /mnt/JDownloader/Download

OK: Passing in the folder that contains the BDMV folder should work too:

Code: Select all

Input: /mnt/JDownloader/_01.Download_/sw-ep4/sw-ep4.part1/sw-ep4/Star Wars, A New Hope

EDIT:

It seems like disk folders extracted by FileBot won't be recognized as disk folders later, since FileBot remembers all the file paths for processing, without checking for disk folders as it would usually do when collecting input from an input folders.


WORKAROUND:

Have JDownloader extract files before calling the amc script on the folder that contains the newly extracted files.
:idea: Please read the FAQ and How to Request Help.
hansooloo
Posts: 30
Joined: 06 Feb 2016, 14:58

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by hansooloo »

Makes sense. Would be great if there could be some intelligence built in where it would do that in one shot.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by rednoah »

1.
That one should be fixed with the next release.

2.
The recommended JDownloader setup does assume that JDownloader deals with archive extraction though, so I'd use it that way anyway:
viewtopic.php?f=4&t=3478#p19358
:idea: Please read the FAQ and How to Request Help.
hansooloo
Posts: 30
Joined: 06 Feb 2016, 14:58

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by hansooloo »

So, having followed your suggestion on #2, I am still dependent on #1 (next release, disk folder detection) I guess?

With #2, JD2 does all the extraction, and leaves me with a folder full of media files ... and then I get this (with 4.8.2 now, love the app, support it!)

Also, changed `-extract` ==> `-rename` and added fn:amc parameter of `skipExtract=y` since we have off-loaded extraction duties to JD2 (am I doing this right?).

Code: Select all

/usr/local/FileBot/filebot.sh                               \
  -rename                                                   \
  --action copy                                             \
  -script fn:amc                                            \
  --def ut_dir=$UT_DIR                                      \
        skipExtract=y                                       \
        ut_kind=multi                                       \
        ut_title=$UT_TITLE                                  \
        movieFormat=@/usr/local/JDownloader/FormatMovie.txt \
        seriesFormat=@/usr/local/JDownloader/FormatTV.txt   \
        clean=y minFileSize=0                               \
  --output /mnt/JDownloader/_02.Format_
  
Run script [fn:amc] at [Tue Jul 24 22:09:39 EDT 2018]
Parameter: skipExtract = y
Parameter: ut_dir = /mnt/JDownloader/Star Wars, A New Hope
Parameter: ut_kind = multi
Parameter: ut_title = sw-ep4
Parameter: movieFormat = Movies/{collection.replaceAll(' Collection', '').replaceFirst(/^(?i)(The|A)\s(.+)/, /$2/)+'/'}{n.replaceAll(/:|]/, ", ").replaceFirst(/^(?i)(The|A)\s(.+)/, /$2, $1/)} ({y})/{n.replaceAll(/:|]/, ", ").replaceFirst(/^(?i)(The|A)\s(.+)/, /$2, $1/)} ({y})
Parameter: seriesFormat = TV/{n}/Season {s.pad(2)}/{n} - {S00E00} - {t}
Parameter: clean = y
Parameter: minFileSize = 0
Input: /mnt/JDownloader/Star Wars, A New Hope
Star Wars, A New Hope [series: Star, movie: Star Wars (1977)]
Exclude Series: Star
Group: [tvs:null, mov:star wars 1977] => [Star Wars, A New Hope]
Rename movies using [TheMovieDB]
Auto-detect movie from context: [/mnt/JDownloader/Star Wars, A New Hope]
CmdlineException: Failed to identify or process any files
Finished without processing any files
Failure (°_°)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by rednoah »

Here's what I get:

Code: Select all

$ filebot -script fn:amc "Star Wars, A New Hope" -non-strict --action TEST --log INFO
[TEST] from [Star Wars, A New Hope] to [Movies\Star Wars (1977)\Star Wars (1977)]
-non-strict is generally recommended.


:idea: strict mode requires the movie year to exactly match the year pattern in the filename, so in strict mode Star Wars, A New Hope will be completely ignored from the get-go because it doesn't contain a release year pattern at all.
:idea: Please read the FAQ and How to Request Help.
hansooloo
Posts: 30
Joined: 06 Feb 2016, 14:58

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by hansooloo »

Oh, right! That makes sense!

One last bit: wonder if it would be possible to have the "contents of the Blu-ray disc root" go into the 1st level "Moviename (Year)" folder, instead of getting a 2nd level directory created?

i.e.,
instead of `/mnt/JDownloader/_02.Format_/Movies/Star Wars/Star Wars (1977)/Star Wars (1977)`

do this `/mnt/JDownloader/_02.Format_/Movies/Star Wars/Star Wars (1977)`
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: How can I get AMC to process MKV files and BDMV directories in one script

Post by rednoah »

Sure. It's your format.

e.g.

Code: Select all

Movies/{f.directory ? plex.name : plex.tail}
:arrow: If {f} is a directory, then we have a disk folder.
:idea: Please read the FAQ and How to Request Help.
Post Reply