4.9.2 GUI Renaming "bug"

Support for macOS users
Post Reply
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

4.9.2 GUI Renaming "bug"

Post by cheaters »

I think this is a bug but maybe it's a feature? 8-)

In the GUI after, I drag in a bunch of files (that were previously named with filebot) to be re-named, some files are skipped/remaining.

The amount of files that are skipped seems to be in proportion to every 100 files i.e., 100 files one folder is skipped, 200 files two folders.

Looking at the files themselves there doesn't appear to be any rhyme or reason for those particular files to be skipped. Their filebot metadata is intact.

Code: Select all

CRC32:
net.filebot.filename:
net.filebot.metadata:
In Presets. If I use "Format Expression: {mediainfo} "Options Datasource: Extended Attributes" on those remaining files they are given "New Names" that match the previous rule I applied to the group they were added with. Then if I "Rename" they are renamed appropriately.

Looking in the logs I see no warnings for that particular file or for that particular time.

Code: Select all

FileBot 4.9.2 (r8040)
JNA Native: 6.1.0
MediaInfo: 20.08
7-Zip-JBinding: 16.02
Chromaprint: 1.5.0
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2020-09-10 (r673)
Groovy: 3.0.5
JRE: OpenJDK Runtime Environment 15
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 16 Core / 17 GB Max Memory / 567 MB Used Memory
OS: Mac OS X (x86_64)
HW: Darwin iMac191.local 18.7.0 Darwin Kernel Version 18.7.0: Mon Aug 31 20:53:32 PDT 2020; root:xnu-4903.278.44~1/RELEASE_X86_64 x86_64
STORAGE: apfs [/] @ 51 GB | hfs [/Volumes/Mercury-2_9481] @ 1.1 TB | ntfs [/Volumes/BOOTCAMP] @ 43 GB | hfs [/Volumes/Godzilla Clone] @ 2.8 TB | hfs [/Volumes/Godzilla] @ 4.5 TB | hfs [/Volumes/SeedDrive] @ 248 GB | hfs [/Volumes/Mercury-1_9940] @ 9 GB | hfs [/Volumes/PlexMedia] @ 2.8 TB | apfs [/Volumes/iMacHDD] @ 902 GB | apfs [/Volumes/Backup_iMac191 SSD] @ 675 GB
DATA: /Users/john/.filebot
Package: APP
License: FileBot License PX9231992 (Valid-Until: 2069-09-03)
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

1.
If you're using the GUI, please include screenshots that illustrate the problem:
https://snipboard.io/


2.
Can you reproduce this behaviour on different file systems? If it doesn't work on a remote file system, and you reproduce the exact same behaviour on the local file system?


3.
There is no {mediainfo} binding. Using {mediainfo} as format will give you nothing, except perhaps a MissingPropertyException.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

rednoah wrote: 09 Oct 2020, 02:18 1.
If you're using the GUI, please include screenshots that illustrate the problem:
https://snipboard.io/
I would have had to include a screenshot video as screenshots won't really show what's happening.
rednoah wrote: 09 Oct 2020, 02:18 2.
Can you reproduce this behaviour on different file systems? If it doesn't work on a remote file system, and you reproduce the exact same behaviour on the local file system?
I only have an iMac. Unfortunately, I already renamed all of these files.

I attempted to give you a screenshot of this issue by reverting three sample sequences using history, and then re-adding them to Original Files but this time none of them were skipped. Which makes me think it has to be a stale metadata issue.
rednoah wrote: 09 Oct 2020, 02:18 3.
There is no {mediainfo} binding. Using {mediainfo} as format will give you nothing, except perhaps a MissingPropertyException.
Using {mediainfo} as format, as I said stated above, gave me the correct names for the files after they were skipped over during the first pass. I don't understand it myself. I am not imagining this. I went through that same process for many groups of files. I count 41 sequences today for a total of 7361 elements renamed.

After some files were left behind I took at peak at their metadata to see what was there because I was curious why files kept being left behind. I checked permissions on those files and on their folders. They contained filebot metadata as stated above. There was nothing peculiar about those files that I could see. Then I clicked on the Preset I have that just contains Format {mediainfo} with those skipped over movies remaining in "Original Files". They were renamed just as the others were on the first pass - according to my renaming preset.

Code: Select all

{plex.derive{' ['+allOf{tags}{audio.language}{vf}{vs}{vc}{crc32}{']'}.join(' ')}}{if (dc > 1) '.'+di}
The other odd behavior I noticed during those 41 sequences was that.jpg files that were dragged over with the folders were also being renamed to the movie title as well. I didn't mention that before because I couldn't remember if that was normal or abnormal.

If I run across this batch renaming issue in the future I will take a video screenshot. For now you will have to take my word, a video would show you exactly what I described here.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

All I can tell you is that {mediainfo} is not a top-level binding and thus cannot possibly work. However, if you have a large {...} expression and then define def mediainfo = 123 yourself then it's of course possible that the variable exists in your own custom code. Human error is also possible as well, i.e. the thing you think you're doing isn't actually what you're doing. That too can happen. You'll figure it out. ;)


FileBot will process derived-by-name companion files:

Code: Select all

Avatar.mp4
Avatar.xml
Avatar-poster.jpg

However, FileBot will not process random companion files:

Code: Select all

Avatar.mp4
folder.xml
folder.jpg
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

rednoah wrote: 09 Oct 2020, 04:12 All I can tell you is that {mediainfo} is not a top-level binding and thus cannot possibly work. However, if you have a large {...} expression and then define def mediainfo = 123 yourself then it's of course possible that the variable exists in your own custom code. Human error is also possible as well, i.e. the thing you think you're doing isn't actually what you're doing. That too can happen. You'll figure it out. ;)
No human error as far as I can tell. I don't have mediainfo defined anywhere and the only "code" I have is the line that I included above in my preset. but I have clicked on a lot of icons trying to get rid of CD1 :lol:

I imagine that from this point forward, since all of my files have been renamed, I won't be doing much batch processing of the entire library. But one never knows... if strikes my fancy to add the IMDB id for movies now that Plex supports that in the filename I might. If this happens again I will certainly take a video screenshot that shows exactly what I described above.

My setup is really very simple. I run the filebot amc script upon each download:

Code: Select all

filebot -script fn:amc --output "/Volumes/PlexMedia/PlexServer_1" --action duplicate --conflict index  -non-strict --log-file amc.log --def excludeList=/Users/John/.filebot/amc.excludes --def unsorted=y --def music=y  --def movieFormat="{plex.derive{' ['+allOf{tags}{audio.language}{vf}{vs}{vc}{crc32}{']'}.join(' ')}}{if (dc > 1) '.'+di}" "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L"
The batch renaming via the GUI

For renaming in place:

Code: Select all

{plex.derive{' ['+allOf{tags}{audio.language}{vf}{vs}{vc}{crc32}{']'}.join(' ')}}{if (dc > 1) '.'+di}
For copying from the seed drive to plex drive if need be:

Code: Select all

/Volumes/PlexMedia/PlexServer_1/{plex.derive{' ['+{allOf{tags}{audio.language}{vf}{vs}{vc}{crc32}{']'}.join(' ')}}}{if (dc > 1) '.'+di}
I don't really use much else except history if a file is associated with the wrong movie.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

This format will work because {plex} is a top-level binding:

Code: Select all

{plex.derive{' ['+allOf{tags}{audio.language}{vf}{vs}{vc}{crc32}{']'}.join(' ')}}{if (dc > 1) '.'+di}
This format cannot work because {mediainfo} is not a top-level binding:

Code: Select all

{mediainfo}
MediaInfo-based bindings such as {vf}, {vs} and {vc} are top-level bindings and will thus work.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

I don't know what the definition of "work" means.

Here is a screenshot video of what I did today for movies that were stranded from their group.

mediainfo.mov

EDIT

I should add that somehow the {mediainfo} command was aware of how they were supposed to be renamed in the first place... so they were renamed according to how the rest of their group was renamed. i.e. the same format. I thought it was bizarre. I should also add that I do now see in the log file that for each of the renaming sessions there was an error for accessing TMDB.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

Yep, there's a bug. Looks like the format set in the preset is completely ignored if the datasource isn't an online database. :roll: {mediainfo} format is never actually set or used, because of a bug, and so the most recently used format is used instead, whatever it may be. Yep, that explains things. Good catch! :lol:



EDIT:

It's complicated. FileBot uses different formats for different object types. The xattr datasource works for different types of objects, but as a File-based source only the format for the File object type is set. The format for the Movie/Episode types remains unchanged. I guess we need to just override all formats because we can't tell in advanced which object type you're going to be processing.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

I think this may be related. I am attempting to rename a new file that was tossed into "Unsorted" (the movie is listed in TMDB).

For some reason trying to rename the files in this particular folder keeps giving me errors. I used my rename Preset to rename but I notice in the log that it appears to think I am trying to use another Preset. I did have a Preset for {xattr} but deleted it to troubleshoot this error. Deleting it did not resolve this.

The relevant error in log.

I have tried to hold the Shift key to process this file but the New Names field will not populate. The working icon appears briefly then disappears. One part of my renaming script did work, the {crc32} calculation. It was attached to the .mp4 file in this group of three files. The other two files are .srt files.

Could a malformed net.filebot.FileBot.app.plist be causing this?

These files are in /Users/john/Library/Preferences:
net.filebot.FileBot.app.plist
net.filebot.FileBot.pkg.plist
net.filebot.ui.plist

For reference, the Renaming Preset I was attempting to run has the label Format & Rename StrictMode
Options: TMDB, Rename, Strict

Code: Select all

{plex.derive{' ['+allOf{tags}{audio.language}{vf}{vs}{vc}{crc32}{']'}.join(' ')}}{if (dc > 1) '.'+di}
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

{xattr} is expected to be undefined if file doesn't have xattr metadata:

Code: Select all

Expression yields empty value: Binding "xattr": undefined
:arrow: TL;DR {xattr} only works if the file has previously been processed with FileBot (and matched with Episode or Movie object at the time).


:idea: If you're using the amc script, then the Unsorted folder would refer to the folder where files that cannot be identified are moved to. These files won't have xattr metadata because we weren't able to identify the file in the first place when processing them previously.


:!: Your application preferences *.plist files are completely unrelated. You can run filebot -clear-prefs (THIS WILL DELETE YOUR PRESETS TOO!!!) if you think that resetting application preferences will help.


:!: EDIT: There are known issues with "manual input in movie mode when dealing with particularly badly named files in certain folder structures" that have been fixed with r8046.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

TL;DR It was a movie naming issue. :roll: Thank you!

I was just trying to give you as much info as necessary to help me figure out what's going on. Didn't mean to insinuate that any of that information was the cause.
rednoah wrote: 10 Oct 2020, 03:29 :arrow: TL;DR {xattr} only works if the file has previously been processed with FileBot (and matched with Episode or Movie object at the time).
I was checking using the command line arguments xattr -l * after FileBot failed to rename these files to see what, if anything, had been added to these files in that directory.
rednoah wrote: 10 Oct 2020, 03:29 :idea: If you're using the amc script, then the Unsorted folder would refer to the folder where files that cannot be identified are moved to. These files won't have xattr metadata because we weren't able to identify the file in the first place when processing them previously.
I understand this. I see that I should not look at the "error.log" in this instance. I should look at the "amc.log" first or in conjunction when a file isn't matched. My natural instinct was to look for information on errors in the "error.log".

rednoah wrote: 10 Oct 2020, 03:29 :!: Your application preferences *.plist files are completely unrelated. You can run filebot -clear-prefs (THIS WILL DELETE YOUR PRESETS TOO!!!) if you think that resetting application preferences will help.
Yep, I learned this the hard way in the past! :(
rednoah wrote: 10 Oct 2020, 03:29 :!: EDIT: There are known issues with "manual input in movie mode when dealing with particularly badly named files in certain folder structures" that have been fixed with r8046.
:?: This must be the beta?
:idea: Do you think it possible to use standard versioning for your betas?

Code: Select all

major.minor[.build[.revision]] (example: 1.2.12.102)
Reason being that Homebrew will not differentiate between betas when they are all named with just "major.minor.[build]" i.e. 4.9.2 and no "revision" number. We have to uninstall the current version, find and delete the cached version of the download, and then reinstall the new beta each time. Time consuming for something that should be automatically run with "brew update" "brew upgrade"
Last edited by cheaters on 10 Oct 2020, 17:50, edited 3 times in total.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

jprokos wrote: 10 Oct 2020, 16:57 I understand this. I see that I should not look at the "error.log" in this instance. I should look at the "amc.log" first or in conjunction when a file isn't matched. My natural instinct was to look for information on errors in the "error.log".
error.log just so happens to be the default log file, e.g. used by the GUI by default. If you use the CLI with the --log-file option, then that file will be used as the sole log file.

jprokos wrote: 10 Oct 2020, 16:57 Reason being that Homebrew will not differentiate between betas when they are all named with just "major.minor.[build]" i.e. 4.9.2 and no "revision" number. We have to uninstall the current version, find and delete the cached version of the download, and then reinstall the new beta each time. Time consuming for something that should be automatically run with "brew update" "brew upgrade"
A change in version numbers is not planned. You can imagine 4.9.2 (r8046) to be 4.9.2.8046 if you prefer it written down like that. If you have trouble with brew cask updates, just use --force to force install whatever it says in the repository right now:

Code: Select all

brew update && brew cask install filebot --force --no-quarantine

:!: Note that brew is not brew cask. AFAIK, brew upgrade will not upgrade brew cask packages.
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

Thanks for your reply.
rednoah wrote: 11 Oct 2020, 03:14 A change in version numbers is not planned. You can imagine 4.9.2 (r8046) to be 4.9.2.8046 if you prefer it written down like that.
That's not a serious response to that question is it? In your beta repository the file is named FileBot_4.9.2.app.tar.xz. There is no revision number associated with any of your files unless I am completely missing some other repository. Certainly a package manager can't discern between 4.9.2 and 4.9.2 with an imaginary revision number. :lol:

One thing you might consider... when you suggest that we try a package with a certain revision number, which addresses a certain issue, how do we locate that package?
rednoah wrote: 11 Oct 2020, 03:14 If you have trouble with brew cask updates, just use --force to force install whatever it says in the repository right now:

Code: Select all

brew update && brew cask install filebot --force --no-quarantine
Using --force does not resolve this.
Using --force will not skip over the cached file. Brew will check the latest version on your github repository against the version of the cached package, see that the version numbers match exactly, then use the cached file. I have done it enough times to know this... and asked moderators on the homebrew forums before approaching you with the request... which I don't think is that outlandish for beta packages.
rednoah wrote: 11 Oct 2020, 03:14 Note that brew is not brew cask. AFAIK, brew upgrade will not upgrade brew cask packages.
That's incorrect...deprecated:

Code: Select all

Warning: Calling brew cask upgrade is deprecated! Use brew upgrade --cask instead.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

Look at

Code: Select all

brew cleanup --help

Code: Select all

 -s                               Scrub the cache, including downloads for
                                   even the latest versions. Note downloads
                                   for any installed formulae or casks will
                                   still not be deleted. If you want to delete
                                   those too: rm -rf "$(brew --cache)"
I don't want to delete my entire cache to remove the cached version of one package. So, like I said previously, I have to cd into that cache directory and remove that cached file before I force install to overwrite filebot.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

You indeed can't try a specific build. There is only "the latest stable release" and "the latest beta release". That makes things easy for everyone involved. Hasn't been an issue so far.

As for brew official release formula, no idea, never had to clear the cache myself. One could add the revision number to the cask formula if a sha256 change isn't enough to trigger an auto-update.

As for brew formula for daily builds, well we don't really do brew cask for nightly beta packages, for various reasons, you're kinda on your own if you want a daily auto-update for the beta via brew.

:?: Which one of those two use cases are we talking about anyway? :lol:
:idea: Please read the FAQ and How to Request Help.
cheaters
Posts: 214
Joined: 09 Oct 2016, 02:01

Re: 4.9.2 GUI Renaming "bug"

Post by cheaters »

I am speaking of the betas and using homebrew.

Unfortunately you haven't added the SHA to filebot-latest.rb. Any reason? Might that resolve my brew updating issue?

Code: Select all

 sha256 :no_check
https://github.com/filebot/plugins/blob ... -latest.rb

Regarding the non-beta version.... just checking and I see there was a silent update of the release version as well, from 4.9.2 (r8040) to 4.9.2 (r8046). I would never have known.

Code: Select all

==> Finding outdated apps
       Cask                   Current  Latest  A/U    Result    
 5/14  filebot                4.9.2    4.9.2        [   OK   ] 
That is what my script shows me when it checks for cask updates. It says filebot is up to date when it was not. There is no logic I can give it to change that result.

Any way for paid users to be notified of those updates? Maybe a page we can monitor that indicates the release has been updated?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: 4.9.2 GUI Renaming "bug"

Post by rednoah »

1.
Well, filebot-latest.rb can't be used for auto-updates because it's not actively maintained. sha256 :no_check makes it work for any daily build because we don't maintain it. Note that adding a sha256 wouldn't make brew upgrade work neither because filebot-latest.rb isn't in the GitHub repository so it won't ever get updated, so brew will never see a version change, so it'll never upgrade automatically.

:idea: The only way to make auto-updates work for filebot-latest.rb is to create a new cask tap repository on GitHub for that cask and then keep it up to date every single day every day for a small number of users that might be using it. Since the demand for that is low, but the cost is high, this is not something I plan on supporting. This task could easily be managed by someone in the community though. It just wouldn't be a very interesting or rewarding task.


2.
If an issue comes up after release with any specific package, then we'll push a new build, and the timestamps will reflect that:
https://get.filebot.net/filebot/FileBot_4.9.2/

:idea: If there's a major issue then there'll be a follow up release with new major version number. If it's a minor issue that most users will never encounter, then we won't force everyone upgrade again right away and just replace the packages in place. If you for some reason do encounter any issue then installing the latest beta is always recommended anyway which then includes that as well. Few people care about monitoring every single build, but our #dev channel does get auto-notified on build if you're keen.
:idea: Please read the FAQ and How to Request Help.
Post Reply