[Feature] Ability to Detect and Delete Duplicate Movies & TV Shows

[CooLTanG]

I currently have written a rudimentary C++ program that just detects duplicate movies (based off the titles alone) and allows the removal of each one. Your program itself seems way more efficient & accurate because it accesses the online DB to detect the movie. Would be awesome to enable the feature of removing duplicates based off the detection in the DB, especially when u can detect the resolution, source, etc.

-------------

Another small request to the FileBot GUI would be a single left click to the "Match" to use the current preset, or even an automatic search right when files are inserted into the GUI. You could easily just put a drop down box with all presets in it and click Match or something along those lines. Would also be nice to manually rename a file myself by right clicking on the certain incorrect show name without having to edit them all.

---------------

Filebot rename process should detect if there's "PROPER, REAL, REPACK" etc. in the name and use those duplicate files as the one you want to keep also.

[rednoah]

1.
Files processed with FileBot should have full metadata stored in xattr. This would make it trivial for 3rd party scripts and tools to detect duplicates based on those known movie id or series id / episode numbers. Checking the online databases just to find duplicates seems very wasteful of such free resources.

e.g. with bash:

Code: Select all

filebot -mediainfo -r /volume1/data/Media/Movies/ --format "{xattr}" --filter "file.video" | sort | uniq -c

2.
The other 2 requests are generally possible via command-line. Custom command-line droplets and --conflict auto should do what you want better than the UI ever could.

[CooLTanG]

1. Seems to be over my head still, I've ran a few movies through, but doesn't seem to put anything in xattr, nor does the "uniq -c" pipe work, just comes up with an error being an unrecognized command. I'll keep playing with it and see what I can come up with.

2. I didn't know about --conflict auto, prob cuz its not in http://www.filebot.net/cli.html, but thanks I'll try it out.

[rednoah]

1.
Are you using Windows? Doing something like this with CMD is not pleasant.

FYI: Keep in mind that this will only work for files where xattr have been set (i.e. processed with FileBot AND xattr enabled filesystem).


2.
Good catch. That'll need to be updated. Use filebot -help for reference.

[CooLTanG]

Code: Select all

FileBot 4.7 (r3923)
JNA Native: 4.0.1
MediaInfo: MediaInfoLib - v0.7.78
7-Zip-JBinding: 9.20
Chromaprint: fpcalc version 1.1.0 (C:\Program Files\FileBot\fpcalc.exe)
Extended Attributes: OK
Groovy Engine: 2.4.6
JRE: Java(TM) SE Runtime Environment 1.8.0_102
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 8 Core / 3 GB Max Memory / 23 MB Used Memory
OS: Windows 7 (amd64)
Package: MSI
Data: C:\Users\User\AppData\Roaming\FileBot
Done ?(?????)?

Yeah Windows 7, idk says right there "Extended Attributes: OK" so I'm assuming its enabled, idk, I'll keep looking into it.

[rednoah]

You can view xattr metadata with this script:
viewtopic.php?f=4&t=5#p5394

Note that even if xattr is working fine for your local NTFS filesystem, it may not work for remote SMB shares, FAT filesystems on USB drives, etc.

[CooLTanG]

Yeah my problem seems that its not writing the xattr metadata on my NTFS drive.

Code: Select all

Failed to set xattr: F:\test\Allegiant.2016\Allegiant (2016).avi:net.filebot.metadata

viewtopic.php?t=324
I've read that page, not sure where this flag is in Java with Windows (sorry for being such a noobie haha)
-DuseExtendedFileAttributes=[true|false]

[rednoah]

On Windows, these options are set in the *.ini files in the FileBot install folder.

The log indicates that even though it's turned on, it doesn't work on the filesystem level. Maybe you're using FAT/exFAT and not NTFS as filesystem. Maybe permissions. Maybe something else. Google might help.

[Sam86]

Handy feature.

I've had a few times where my script would bugger up, or I'd forget to comment out a test line from scheduled tasks etc... few week later I'd notice every movie is in twice.
Recent one being: Renamed and original in plex folder.

dupeGuru is a simple quick windows gui program, great for things like that.