Not extracting multipart files

Any questions? Need some help?
Post Reply
Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Not extracting multipart files

Post by Muddro » 12 Dec 2018, 01:59

I recently switched to qBittorrent on my FreeNAS. Its running FreeNAS 11.2, qBittorrent 4.1.3.

sysinfo:

Code: Select all

FileBot 4.8.2 (r5789)
JNA Native: 5.2.2
MediaInfo: 18.08.1
Apache Commons VFS: [zip, rar]
FFprobe: 4.0.3
Chromaprint: 1.4.3
Extended Attributes: java.io.IOException: UserDefinedFileAttributeView is not supported
Unicode Filesystem: OK
Script Bundle: 2018-11-29 (r539)
Groovy: 2.5.1
JRE: OpenJDK Runtime Environment 1.8.0_181
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 8 Core / 4 GB Max Memory / 27 MB Used Memory
OS: FreeBSD (amd64)
HW: FreeBSD qbittorrent 11.2-STABLE FreeBSD 11.2-STABLE #0 r325575+fc3d65faae6(freenas/11.2-stable): Wed Dec  5 15:08:42 EST 2018     [email protected]:/freenas-11.2-releng/freenas/_BE/objs/freenas-11.2-releng/freenas/_BE/os/sys/FreeNAS.amd64  amd64
DATA: /Filebot/data/root
Package: TAR
License: FileBot License P5612643 (Valid-Until: 2019-09-14)
Done ヾ(@⌒ー⌒@)ノ
My script:

Code: Select all

/Filebot/filebot.sh -script fn:amc --output "/" --log-file amc.log --action copy --conflict auto -non-strict -no-xattr --def music=n pushbullet=********artwork=n clean=y "seriesFormat=/{vf =~ /2160p/ ? '/media/4K TV Shows' : '/media/TV Shows'}/{n}/Season {s.pad(2)}/{n} - {s00e00} - {t}" "movieFormat=/{vf =~ /2160p/ ? '/media/4K Movies' : '/media/Movies'}/{n} ({y})" plex=192.168.53.6:********* "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"
Torrents that are not RARred copy over and rename just fine. However, files in multipart RAR files do not. My log looks like this:

Code: Select all

Run script [fn:amc] at [Tue Dec 11 20:40:38 EST 2018]
Parameter: music = n
Parameter: pushbullet = *****
Parameter: artwork = n
Parameter: clean = y
Parameter: seriesFormat = /{vf =~ /2160p/ ? '/media/4K TV Shows' : '/media/TV Shows'}/{n}/Season {s.pad(2)}/{n} - {s00e00} - {t}
Parameter: movieFormat = /{vf =~ /2160p/ ? '/media/4K Movies' : '/media/Movies'}/{n} ({y})
Parameter: plex = *****
Parameter: ut_dir = /media/Downloads/Complete/TESTFILE
Parameter: ut_kind = multi
Parameter: ut_title = TESTFILE
Parameter: ut_label =
Read archive [TESTFILE.rar] and extract to [/media/Downloads/Complete/TESTFILE/TESTFILE/TESTFILE]
Read archive [TESTFILE.rar] and extract to [/media/Downloads/Complete/TESTFILE/TESTFILE/TESTFILE]
Input: /media/Downloads/Complete/TESTFILE/TESTFILE.rar
Failed to read xattr: IOException: UserDefinedFileAttributeView is not supported
Group: [tvs:TESTFILE] => [TESTFILE.rar]
Rename episodes using [TheTVDB]
CmdlineException: No media files: [/media/Downloads/Complete/TESTFILE/TESTFILE.rar]
Clean temporary extracted files
Finished without processing any files
Failure (°_°)
Would love some help with this. Can't figure it out.

User avatar
rednoah
The Source
Posts: 15341
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Not extracting multipart files

Post by rednoah » 12 Dec 2018, 06:30

Strange indeed. Same line twice is odd. And looks like no files are extracted. Probably just doesn't work with this particular archive:

Code: Select all

Read archive [TESTFILE.rar] and extract to [/media/Downloads/Complete/TESTFILE/TESTFILE/TESTFILE]
Read archive [TESTFILE.rar] and extract to [/media/Downloads/Complete/TESTFILE/TESTFILE/TESTFILE]
Input: /media/Downloads/Complete/TESTFILE/TESTFILE.rar

1.
Please try the latest beta and see if it makes a difference:
viewtopic.php?f=6&t=9535


2.
Using the 7z executable instead of Apache VFS / junrar-plugin is definitely worth a try:
viewtopic.php?f=3&t=6311
:idea: Please read the FAQ and How to Request Help.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 12 Dec 2018, 12:50

The beta did not fix it.

I have p7zip installed, however, I am unable to use it using

Code: Select all

export FILEBOT_OPTS="-Dnet.filebot.Archive.extractor=SevenZipExecutable"
Is there a specific way to do this on FreeNAS/FreeBSD?

EDIT: Figured out how to get it in the environment variables. In any case it did not fix the problem. Same output as previously posted. Also, this is the case for all RARred torrents, not just one file. The only time it works is when it is not archived.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 13 Dec 2018, 02:11

Still happening. For an example of one that does work and one that doesn't:

Code: Select all

Run script [fn:amc] at [Wed Dec 12 14:36:07 EST 2018]
Parameter: music = n
Parameter: pushbullet = *****
Parameter: artwork = n
Parameter: clean = y
Parameter: seriesFormat = /{vf =~ /2160p/ ? '/media/4K TV Shows' : '/media/TV Shows'}/{n}/Season {s.pad(2)}/{n} - {s00e00} - {t}
Parameter: movieFormat = /{vf =~ /2160p/ ? '/media/4K Movies' : '/media/Movies'}/{n} ({y})
Parameter: plex = *****
Parameter: ut_dir = /media/Downloads/Complete/Peppermint.2018.720p.BluRay.DTS.X264-iFT
Parameter: ut_kind = multi
Parameter: ut_title = Peppermint.2018.720p.BluRay.DTS.X264-iFT
Parameter: ut_label = 
Ignore video extra: /media/Downloads/Complete/Peppermint.2018.720p.BluRay.DTS.X264-iFT/Sample.mkv
Input: /media/Downloads/Complete/Peppermint.2018.720p.BluRay.DTS.X264-iFT/Peppermint.2018.720p.BluRay.DTS.X264-iFT.mkv
Failed to read xattr: IOException: UserDefinedFileAttributeView is not supported
Group: [mov:peppermint 2018] => [Peppermint.2018.720p.BluRay.DTS.X264-iFT.mkv]
Rename movies using [TheMovieDB]
[COPY] from [/media/Downloads/Complete/Peppermint.2018.720p.BluRay.DTS.X264-iFT/Peppermint.2018.720p.BluRay.DTS.X264-iFT.mkv] to [/media/Movies/Peppermint (2018).mkv]
Processed 1 files
Notify Plex: [host:*****************, token:*****************]
GET: http://*****************:32400/library/sections/all/refresh?X-Plex-Token=*****************
Sending PushBullet report
Done ヾ(@⌒ー⌒@)ノ
Run script [fn:amc] at [Wed Dec 12 19:34:18 EST 2018]
Parameter: music = n
Parameter: pushbullet = *****
Parameter: artwork = n
Parameter: clean = y
Parameter: seriesFormat = /{vf =~ /2160p/ ? '/media/4K TV Shows' : '/media/TV Shows'}/{n}/Season {s.pad(2)}/{n} - {s00e00} - {t}
Parameter: movieFormat = /{vf =~ /2160p/ ? '/media/4K Movies' : '/media/Movies'}/{n} ({y})
Parameter: plex = *****
Parameter: ut_dir = /media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS
Parameter: ut_kind = multi
Parameter: ut_title = Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS
Parameter: ut_label = 
Read archive [leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar] and extract to [/media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS]
[leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar] contains [0] files
Read archive [leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar] and extract to [/media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS]
[leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar] contains [0] files
Input: /media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar
Failed to read xattr: IOException: UserDefinedFileAttributeView is not supported
Group: [tvs:leah remini scientology and the aftermath] => [leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar]
Rename episodes using [TheTVDB]
CmdlineException: No media files: [/media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar]
Clean temporary extracted files
Finished without processing any files
Failure (°_°)
Here is the updated sysinfo

Code: Select all

FileBot 4.8.5 (r5976)
JNA Native: 5.2.2
MediaInfo: 18.08.1
p7zip: Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)
FFprobe: 4.0.3
Chromaprint: 1.4.3
Extended Attributes: java.io.IOException: UserDefinedFileAttributeView is not supported
Unicode Filesystem: OK
Script Bundle: 2018-11-29 (r539)
Groovy: 2.5.4
JRE: OpenJDK Runtime Environment 1.8.0_181
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 8 Core / 4 GB Max Memory / 56 MB Used Memory
OS: FreeBSD (amd64)
HW: FreeBSD qbittorrent 11.2-STABLE FreeBSD 11.2-STABLE #0 r325575+fc3d65faae6(freenas/11.2-stable): Wed Dec  5 15:08:42 EST 2018     [email protected]:/freenas-11.2-releng/freenas/_BE/objs/freenas-11.2-releng/freenas/_BE/os/sys/FreeNAS.amd64  amd64
DATA: /Filebot/data/root
Package: TAR
License: FileBot License P5612643 (Valid-Until: 2019-09-14)
Done ヾ(@⌒ー⌒@)ノ
I am not really sure what to do, but would love to get this working. Also, I am able to extract these files manually through the shell, so 7zip is working on that end at least.

User avatar
rednoah
The Source
Posts: 15341
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Not extracting multipart files

Post by rednoah » 13 Dec 2018, 08:44

FileBot apparently can't decode the archive, since it seems to think that the archive is empty:

Code: Select all

[leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar] contains [0] files

What does 7z say if you try to list the archive contents?

Code: Select all

7z l -slt -y /path/to/archive.rar
:idea: Please read the FAQ and How to Request Help.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 13 Dec 2018, 12:40

Code: Select all

[email protected]:/ # 7z l -slt -y /media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 50000000 bytes (48 MiB)

Listing archive: /media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar

--
Path = /media/Downloads/Complete/Leah.Remini.Scientology.and.the.Aftermath.S03E04.720p.WEB.h264-TBS/leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.rar
Type = Rar
Physical Size = 50000000
Total Physical Size = 869984248
Characteristics = Volume FirstVolume VolCRC
Solid = -
Blocks = 1
Multivolume = +
Volume Index = 0
Volumes = 18

----------
Path = leah.remini.scientology.and.the.aftermath.s03e04.720p.web.h264-tbs.mkv
Folder = -
Size = 869981692
Packed Size = 869981692
Modified = 2018-12-12 10:30:08
Created =
Accessed =
Attributes =
Encrypted = -
Solid = -
Commented = -
Split Before = -
Split After = -
CRC = 1B8F72D4
Host OS = Unix
Method = m0:22
Version = 20

User avatar
rednoah
The Source
Posts: 15341
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Not extracting multipart files

Post by rednoah » 13 Dec 2018, 14:09

Well, that looks good. Are you sure FILEBOT_OPTS is set correctly when the amc script is called?

Code: Select all

export FILEBOT_OPTS="-Dnet.filebot.Archive.extractor=SevenZipExecutable"
filebot ...
:idea: Please read the FAQ and How to Request Help.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 13 Dec 2018, 15:33

Should be. Otherwise it would be using apache in sysinfo, correct?

I wasnt able to "export" because its freenas jail, so I manually added it to .cshrc, and when running sysinfo, its using 7zip. I also installed Bash to be able to run that command, and ran it as sysenv. It survives reboot of the jail.

Code: Select all

[email protected]:/ # /filebot.sh -script fn:sysinfo
FileBot 4.8.5 (r5976)
JNA Native: 5.2.2
MediaInfo: 18.08.1
p7zip: Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x6                                                                                                                                     4)
FFprobe: 4.0.3
Chromaprint: 1.4.3
Extended Attributes: java.io.IOException: UserDefinedFileAttributeView is not su                                                                                                                                     pported
Unicode Filesystem: OK
Script Bundle: 2018-11-29 (r539)
Groovy: 2.5.4
JRE: OpenJDK Runtime Environment 1.8.0_181
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 8 Core / 4 GB Max Memory / 54 MB Used Memory
OS: FreeBSD (amd64)
HW: FreeBSD qbittorrent 11.2-STABLE FreeBSD 11.2-STABLE #0 r325575+fc3d65faae6(f                                                                                                                                     reenas/11.2-stable): Wed Dec  5 15:08:42 EST 2018     [email protected]                                                                                                                                     com:/freenas-11.2-releng/freenas/_BE/objs/freenas-11.2-releng/freenas/_BE/os/sys                                                                                                                                     /FreeNAS.amd64  amd64
DATA: /data/root
Package: TAR
License: FileBot License P5612643 (Valid-Until: 2019-09-14)
Done ヾ(@⌒ー⌒@)ノ

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 14 Dec 2018, 00:51

Just thought of something that may be worth considering too. It wasn't an issue when I did all this using uTorrent in a Win10 VM, but the location of the files is on a mounted drive. I am sure that may have been assumed, but perhaps this changes something?

User avatar
rednoah
The Source
Posts: 15341
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Not extracting multipart files

Post by rednoah » 14 Dec 2018, 07:46

Not sure where the point of failure could be at this point. If the CLI tools work for you, then it'll work for filebot, cause filebot is also just executing the command.

Try enabling debug options too, so we can see exactly what command it's running, and what the output says:

Code: Select all

export FILEBOT_OPTS="-Dnet.filebot.Archive.extractor=SevenZipExecutable -Dnet.filebot.logging.debug=ALL"
:idea: viewtopic.php?f=3&t=3913

The log should say something like:

Code: Select all

Execute [7z, l, ...]
:idea: Note that error output is directly inherited to the console, so if there's an error, it'll show up in the console output, but not in the filebot log file, so best to test and run with interactive shell.
:idea: Please read the FAQ and How to Request Help.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 16 Dec 2018, 13:42

ugh I don't know whats happening:

My environment variables are there:

Code: Select all

[email protected]:/ # setenv
USER=root
LOGNAME=root
HOME=/root
MAIL=/var/mail/root
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
TERM=xterm
BLOCKSIZE=K
MM_CHARSET=UTF-8
LANG=en_US.UTF-8
SHELL=/bin/csh
SSH_CLIENT=
SSH_CONNECTION=
SSH_TTY=/dev/pts/0
HOSTTYPE=FreeBSD
VENDOR=amd
OSTYPE=FreeBSD
MACHTYPE=x86_64
SHLVL=2
PWD=/
GROUP=wheel
HOST=qbittorrent
REMOTEHOST=
EDITOR=nano
PAGER=more
FILEBOT_OPTS=-Dnet.filebot.Archive.extractor=SevenZipExecutable -Dnet.filebot.logging.debug=ALL
But my output is not producing the debugged output.

User avatar
rednoah
The Source
Posts: 15341
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Not extracting multipart files

Post by rednoah » 16 Dec 2018, 14:31

If you don't get any 7z related logs, and in it doesn't work, then 7z quite possibly just isn't used.

Here's what I get:

Code: Select all

$ export FILEBOT_OPTS='-Dnet.filebot.Archive.extractor=SevenZipExecutable -Dnet.filebot.logging.debug=ALL'
$ filebot -extract .
...
Read archive [Alias.1x01.avi.zip] and extract to [Alias.1x01.avi]
Execute: [7z, l, -slt, -y, Alias.1x01.avi.zip]

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,4 CPUs x64)

Scanning the drive for archives:
1 file, 201 bytes (1 KiB)

Listing archive: Alias.1x01.avi.zip

--
Path = Alias.1x01.avi.zip
Type = zip
Physical Size = 201

----------
Path = Alias.1x01.avi
Folder = -
Size = 29
Packed Size = 31
Modified = 2018-12-15 13:24:50
Created =
Accessed =
Attributes = _ -rw-r--r--
Encrypted = -
Comment =
CRC = 92A09E2A
Method = Deflate
Host OS = Unix
Version = 20
Volume Index = 0


Extracting files [Alias.1x01.avi/Alias.1x01.avi]
Execute: [7z, x, -y, -aos, Alias.1x01.avi.zip, -oAlias.1x01.avi]

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,4 CPUs x64)

Scanning the drive for archives:
1 file, 201 bytes (1 KiB)

Extracting archive: Alias.1x01.avi.zip
--
Path = Alias.1x01.avi.zip
Type = zip
Physical Size = 201

Everything is Ok

Size:       29
Compressed: 201

Testing with a small zip file that contains a fake avi file:

Code: Select all

date > Alias.1x01.avi
zip Alias.1x01.avi.zip Alias.1x01.avi
:idea: Please read the FAQ and How to Request Help.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 16 Dec 2018, 20:27

Whelp, I decided to install it in a new jail and now its working. So I don't know what the deal was. Thanks for all the help though.

Edit: I think in the original jail I had both openjdk 7 and 8 installed. Thinking that may have been the cause

User avatar
rednoah
The Source
Posts: 15341
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Not extracting multipart files

Post by rednoah » 17 Dec 2018, 03:36

Strange. Well, whatever going on before didn't make sense, so one of our assumptions was wrong.

Glad it's working now though. :)

So 7z now works with all your archives?
:idea: Please read the FAQ and How to Request Help.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 17 Dec 2018, 12:22

7z (p7zip) works perfectly.

Muddro
Posts: 33
Joined: 22 Mar 2014, 21:38

Re: Not extracting multipart files

Post by Muddro » 23 Dec 2018, 18:57

I believe I found the official root cause, for anyone who has the same problem:

qBittorrent 4.1.3 - nox appears to have issues when you move downloads upon completion and execute a program. Hopefully will be fixed in future versions.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest