AMC.log file polluted with Ignore Hidden:

Support for macOS users
Post Reply
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

AMC.log file polluted with Ignore Hidden:

Post by cheaters »

Does anyone have an idea why this behavior would start occurring? Has something changed in the fn.amc script?

I have changed nothing that I know of but for the last few days I am now seeing my AMC.log filled with Ignore hidden: … when new files are being acted on by the amc script. This was not happening in the past so something has changed, but I am unable to figure it out.

Should Parameter: ut_dir = /Volumes/SeedDrive/complete now have a forward slash at the end? Maybe a change in QBittorrent. I recently updated to version 4.3.0.

Code: Select all

filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" --action duplicate --conflict index  -non-strict --log-file amc.log --def pushover=u3d8sk6x1wvopeguaizusctcv4u553:ams63cxz3j6o45q2u5cjx4irtu1f12 --def excludeList=/Users/John/.filebot/amc.excludes --def unsorted=y --def music=y  --def skipExtract=y --def movieFormat="{plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{info:video[0].displayAspectRatioString.colon('"∶"').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"

Code: Select all

[2020-11-05 10:11:18.466] Run script [fn:amc] at [Thu Nov 05 10:11:18 CST 2020]
[2020-11-05 10:11:18.491] Parameter: pushover = *****
[2020-11-05 10:11:18.492] Parameter: excludeList = /Users/John/.filebot/amc.excludes
[2020-11-05 10:11:18.492] Parameter: unsorted = y
[2020-11-05 10:11:18.492] Parameter: music = y
[2020-11-05 10:11:18.492] Parameter: skipExtract = y
[2020-11-05 10:11:18.492] Parameter: movieFormat = {plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{info:video[0].displayAspectRatioString.colon('∶').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}
[2020-11-05 10:11:18.493] Parameter: ut_dir = /Volumes/SeedDrive/complete
[2020-11-05 10:11:18.493] Parameter: ut_kind = multi
[2020-11-05 10:11:18.493] Parameter: ut_title = One.Last.Deal.2018.FINNISH.1080p.BluRay.x265
[2020-11-05 10:11:18.493] Parameter: ut_label = Movies
[2020-11-05 10:11:19.218] Use excludes: /Users/John/.filebot/amc.excludes (5665)
[2020-11-05 10:11:19.263] Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store
…
… continues with 517 lines of Ignore hidden .DS_Store files, Ignore video extra:, Skip short video: for every directory and subdirectory. Then at the end of the log entry:

Code: Select all

[2020-11-05 10:11:21.427] Input: /Volumes/SeedDrive/complete/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265.mp4
[2020-11-05 10:11:21.427] Input: /Volumes/SeedDrive/complete/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265-VXT/Subs/8_English.srt
[2020-11-05 10:11:21.432] Process as Movie [Movies]
[2020-11-05 10:11:21.434] Group: {Movie=true} => [One.Last.Deal.2018.FINNISH.1080p.BluRay.x265.mp4, 8_English.srt]
[2020-11-05 10:11:21.461] Rename movies using [TheMovieDB]
[2020-11-05 10:11:21.462] Auto-detect movie from context [/Volumes/SeedDrive/complete/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265.mp4]
[2020-11-05 10:11:28.303] Auto-detect movie from context [/Volumes/SeedDrive/complete/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265/Subs/8_English.srt]
[2020-11-05 10:11:31.729] [DUPLICATE] from [/Volumes/SeedDrive/complete/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265.mp4] to [/Volumes/PlexMedia/PlexServer_1/Movies/One Last Deal (2019)/One Last Deal (2019) {imdb-tt5851680} [[fi] 2.40∶1 WS 1080p BluRay x265 FEB8A367].mp4]
[2020-11-05 10:11:39.907] [DUPLICATE] from [/Volumes/SeedDrive/complete/One.Last.Deal.2018.FINNISH.1080p.BluRay.x265/Subs/8_English.srt] to [/Volumes/PlexMedia/PlexServer_1/Movies/One Last Deal (2019)/One Last Deal (2019) {imdb-tt5851680} [[fi] 2.40∶1 WS 1080p BluRay x265 FEB8A367].eng.srt]
[2020-11-05 10:11:39.993] Processed 2 files
[2020-11-05 10:11:40.000] Sending Pushover notification
[2020-11-05 10:11:40.749] Done ヾ(@⌒ー⌒@)ノ

Code: Select all

FileBot 4.9.2 (r8046)
JNA Native: 6.1.0
MediaInfo: 20.08
7-Zip-JBinding: 16.02
Chromaprint: 1.5.0
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-10-11 (r675)
Groovy: 3.0.6
JRE: OpenJDK Runtime Environment 15
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 16 Core / 17 GB Max Memory / 67 MB Used Memory
OS: Mac OS X (x86_64)
HW: Darwin iMac191.local 18.7.0 Darwin Kernel Version 18.7.0: Mon Aug 31 20:53:32 PDT 2020; root:xnu-4903.278.44~1/RELEASE_X86_64 x86_64
STORAGE: apfs [/] @ 57 GB | hfs [/Volumes/Mercury-2_9481] @ 1.1 TB | ntfs [/Volumes/BOOTCAMP] @ 43 GB | hfs [/Volumes/Godzilla Clone] @ 2.4 TB | hfs [/Volumes/Godzilla] @ 4.5 TB | apfs [/Volumes/iMacHDD] @ 902 GB | apfs [/Volumes/Backup_iMac191 SSD] @ 687 GB | hfs [/Volumes/SeedDrive] @ 42 GB | hfs [/Volumes/Mercury-1_9940] @ 44 GB | hfs [/Volumes/PlexMedia] @ 1.8 TB
DATA: /Users/john/.filebot
Package: APP
License: FileBot License PX9231992 (Valid-Until: 2069-09-03)
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: AMC.log file polluted with Ignore Hidden:

Post by cheaters »

I don't know if this is the issue but I am looking again at the amc page

And it clearly says that paths must not end with a backslash (forward slash on Mac?). Has something has changed in Qbittorrent v4.3.0 client in the recent update that is coercing a forward slash at the end of the path for my chosen save locations. I tried to remove it but it reappears after a restart of the application.

Image
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC.log file polluted with Ignore Hidden:

Post by rednoah »

.DS_Store starts with . and so it's a hidden file we'll want to ignore:

Code: Select all

Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store
:arrow: https://en.wikipedia.org/wiki/.DS_Store


:idea: .DS_Store is created when you browse a folder with Finder. Kinda like Thumbs.db on Windows. It's very possible that this file doesn't exist until you first browser a given folder with Finder, at which point Finder creates thumbnails and stores them in the .DS_Store file for next time.


:idea: You can use --log FINE to reduce the log verbosity. Alternatively, you can configure Finder to not create .DS_Store files and then delete all the .DS_Store that have already been created.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: AMC.log file polluted with Ignore Hidden:

Post by cheaters »

rednoah wrote: 05 Nov 2020, 17:47 .DS_Store starts with . and so it's a hidden file we'll want to ignore:

Code: Select all

Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store
:arrow: https://en.wikipedia.org/wiki/.DS_Store


:idea: .DS_Store is created when you browse a folder with Finder. Kinda like Thumbs.db on Windows. It's very possible that this file doesn't exist until you first browser a given folder with Finder, at which point Finder creates thumbnails and stores them in the .DS_Store file for next time.


:idea: You can use --log FINE to reduce the log verbosity.
:?: Please, where do I set this? In filebot preferences?

Code: Select all

Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC.log file polluted with Ignore Hidden:

Post by rednoah »

jprokos wrote: 05 Nov 2020, 17:15 I don't know if this is the issue but I am looking again at the amc page

And it clearly says that paths must not end with a backslash (forward slash on Mac?). Has something has changed in Qbittorrent v4.3.0 client in the recent update that is coercing a forward slash at the end of the path for my chosen save locations. I tried to remove it but it reappears after a restart of the application.

Image
This is not the issue.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC.log file polluted with Ignore Hidden:

Post by rednoah »

jprokos wrote: 05 Nov 2020, 17:50 :?: Please, where do I set this? In filebot preferences?
--log FINE is a command-line option:
https://www.filebot.net/cli.html
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: AMC.log file polluted with Ignore Hidden:

Post by cheaters »

Where do I set: Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store

I can also set the log preferences in system.properties,,, yes?
Logging Options:

Enable debug logging: net.filebot.logging.debug = ALL | FINE | INFO

Code: Select all

filebot -script fn:properties --def net.filebot.logging.debug=ALL
Print timestamp for each log record: net.filebot.logging.time = true | false

Code: Select all

filebot -script fn:properties --def net.filebot.logging.time=true
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC.log file polluted with Ignore Hidden:

Post by rednoah »

jprokos wrote: 05 Nov 2020, 18:01 Where do I set: Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store
There is no switch that would allow you to disable a specific log message.

jprokos wrote: 05 Nov 2020, 18:01 I can also set the log preferences in system.properties,,, yes?
No. The Developer Options allow you to enable additional debug logging which is going to be extremely verbose.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: AMC.log file polluted with Ignore Hidden:

Post by cheaters »

It's just a language thing. I interpret what you are writing differently than you intend it to be interpreted.

when you wrote
.DS_Store starts with . and so it's a hidden file we'll want to ignore:
CODE: SELECT ALL
Ignore hidden: /Volumes/SeedDrive/complete/.DS_Store
I assumed that what followed the colon was the action you suggested in the sentence when you chose the verb "ignore". Especially since you put it in a code block
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: AMC.log file polluted with Ignore Hidden:

Post by cheaters »

Why is it acting on directories not in the path anyway? I don't remember it doing this in the past. When I download a file it should only be working on files inside the new directory created by the download not the entire top directory.

it's acting on every directory contained by /Volumes/SeedDrive/complete

Code: Select all

/Volumes/SeedDrive/complete/movie year 
/Volumes/SeedDrive/complete/movieB
/Volumes/SeedDrive/complete/movieC
…
When a file is downloaded why would the amc script start looking outside of the path of the downloaded directory and the files contained inside of it?

This was not happening before.
like I mention above, the log is showing:

Code: Select all

Ignore video extra: /Volumes/SeedDrive/complete/movieA/sample.mkv
When just recently the log was not full of these entries

It appears that all of the instances where this is occurring the numerous log entries are preceded by this line:

Code: Select all

Use excludes: /Users/John/.filebot/amc.excludes (13091)
… right into looking through every directory inside /complete
Whereas the log entries that don't look through the entire contents of the directory have this in the log

Code: Select all

Use excludes: /Users/John/.filebot/amc.excludes (13089)
Input: /Volumes/SeedDrive/MovieFile1
Input: /Volumes/SeedDrive/MovieFile2
So Input is not being found and filebot is processing every file and folder in the directory /complete
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC.log file polluted with Ignore Hidden:

Post by rednoah »

The amc script will process the folders you pass in. No more. No less.


In this case, /Volumes/SeedDrive/complete is passed in as input:
jprokos wrote: 05 Nov 2020, 16:48

Code: Select all

Parameter: ut_dir = /Volumes/SeedDrive/complete

In this case, we can assume that /Volumes/SeedDrive was passed in as input:
jprokos wrote: 05 Nov 2020, 18:29

Code: Select all

Input: /Volumes/SeedDrive/MovieFile1
Input: /Volumes/SeedDrive/MovieFile2
...

:idea: Since ut_dir is defined by your qBT via %F substitution, we can reasonably assume that %F isn't always what you think it should be. It should be the absolute file path to the newly completed file or folder, but according to the results you get, this is not the case.


:!: Perhaps the recent qBT update changed how %variables work, so you might have to update your command template in turn.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: AMC.log file polluted with Ignore Hidden:

Post by cheaters »

rednoah wrote: 06 Nov 2020, 06:10 The amc script will process the folders you pass in. No more. No less.

:idea: Since ut_dir is defined by your qBT via %F substitution, we can reasonably assume that %F isn't always what you think it should be. It should be the absolute file path to the newly completed file or folder, but according to the results you get, this is not the case.


:!: Perhaps the recent qBT update changed how %variables work, so you might have to update your command template in turn.
Well, I didn't notice it in the log but this is exactly what's happening!

This is the path is when the log is flooded:

Code: Select all

[2020-11-06 01:54:06.269] Parameter: ut_dir = /Volumes/SeedDrive/complete
This is the path when things work properly:

Code: Select all

[2020-11-05 12:37:12.560] Parameter: ut_dir = /Volumes/SeedDrive/complete/Psywar - The Real Battlefield is the Mind (2010) Documentary.mp4
I just need to clear my head by putting some ideas down here…

There are path options to save incomplete and complete torrents that I recently implemented (instead of keeping them all in the same directory).

There are three "Supported parameters" that deal with paths in qBT

Code: Select all

%F: Content path (same as root path for multi file torrent)
%R: Root path (first torrent subdirectory path)
%D: Save path
Is there something obvious that I am missing?
When I changed FROM everything (incomplete/complete) saving into /Volumes/SeedDrive TO saving incomplete torrents into /Volumes/SeedDrive/incomplete and complete torrents saving into /Volumes/SeedDrive/complete what needed to change in the amc script?

Code: Select all

filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" --action duplicate --conflict index  -non-strict --log-file amc.log --def pushover=xxx:xxx --def excludeList=/Users/John/.filebot/amc.excludes --def unsorted=y --def music=y  --def skipExtract=y --def movieFormat="{plex.derive{' {imdb-'}{imdbid}{'}'}{' ['+allOf{tags}{audio.language}{info:video[0].displayAspectRatioString.colon('"∶"').replace('?', '')}{ws}{vf}{vs}{vc}{crc32}.join(' ')}{']'}}{if (dc > 1) '.'+di}" "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC.log file polluted with Ignore Hidden:

Post by rednoah »

First we collect information. We can't fix the command above if we don't know what these variables mean exactly. The only thing that we do know is that %F doesn't mean what we want it to mean.


:arrow: I'd add all the variables to the command, so that you can check what values each of these variables have in various cases in the log, so you can then pick the one that works best as input argument:

Code: Select all

--def F="%F" --def R="%R" --def D="%D"
:idea: Please read the FAQ and How to Request Help.
Post Reply