deleting duplicates

[seibop]

I'm using:

filebot -script fn:duplicates "d:\media\movies" --action delete

Which works perfectly, except when I have situations where a movie is broken into multiple files...
IE

d:\media\movies\spiderman.mp4
d:\media\movies\spiderman cd1.mp4
d:\media\movies\spiderman cd2.mp4
d:\media\movies\spiderman cd3.mp4

In the case where the Spiderman into 3 files, is say 1080 and the single file is say 720. filebot correctly identifies that Spiderman cd1.mp4 is the higher quality - however it will delete spiderman.mp4 as it should - but also delete d:\media\movies\spiderman cd2.mp4 and d:\media\movies\spiderman cd3.mp4 thinking they are also duplicates of the same quality as spiderman CD1.mp4...

I'm not seeing a clear way around this.

[rednoah]

What does the xattr metadata say for each of those files?
viewtopic.php?p=5394#p5394


Please read Metadata and Extended Attributes for details.

[seibop]

Okay

Command:

Code: Select all

filebot -script fn:xattr "d:\media\movies\The Big Ugly (2020)"

Returns:

Code: Select all

D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020).avi
        net.filebot.filename: The.Big.Ugly.2020.BRRip.XviD.AC3-EVO.avi
        net.filebot.metadata: {"@type":"Movie","year":2020,"imdbId":9441638,"tmdbId":714521,"language":"en","id":714521,"name":"The Big Ugly","aliasNames":[]}
D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020).mkv
        net.filebot.filename: The.Big.Ugly.2020.1080p.WEBRip.X264.DD.5.1-EVO.mkv
        net.filebot.metadata: {"@type":"Movie","year":2020,"imdbId":9441638,"tmdbId":714521,"language":"en","id":714521,"name":"The Big Ugly","aliasNames":[]}
D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020) CD1.mkv
        net.filebot.filename: ytJsoRo3b7gWcYSJ3.mkv
        net.filebot.metadata: {"@type":"MoviePart","partIndex":1,"partCount":2,"year":2020,"imdbId":9441638,"tmdbId":714521,"language":"en","id":714521,"name":"The Big Ugly","aliasNames":[]}
D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020) CD2.mkv
        net.filebot.filename: ytJsoRo3b7gWcYSJ3.mkv
        net.filebot.metadata: {"@type":"MoviePart","partIndex":2,"partCount":2,"year":2020,"imdbId":9441638,"tmdbId":714521,"language":"en","id":714521,"name":"The Big Ugly","aliasNames":[]}

Command

Code: Select all

filebot -script fn:duplicates "d:\media\movies" --action test                                                                                                                                                           

returns:

[*] The Big Ugly (2020)

Code: Select all

[+] 1. D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020) CD1.mkv
[-] 2. D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020) CD2.mkv
[-] 3. D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020).mkv
[-] 4. D:\media\Movies\The Big Ugly (2020)\The Big Ugly (2020).avi

So in this case, it will keep the first part of the movie (CD1) and delete the duplicates (MKV and AVI) but will also delete CD2 as well -- deleting 1/2 the movie.

Any suggestion as to what I'm doing incorrectly?

[rednoah]

xattr looks good. I suppose this particular case just doesn't work for some reason.

[seibop]

I've got multiple of instances where filebot will keep CD1, but delete all other instances. Any way to I don't know to set a preference where Filebot will keep the better quality ? For instance In the example CD1 and CD2 are higher quality than the MKV or AVI. Any thoughts on how to ensure the filebot sees that CD1.CD2 are paired and not delete one of the CD's?

I understand the logic... File bot sees CD1 as equal to or better than the other 3 copies. So it deletes the MKV and AVI since they are lower quality, and deletes CD2 since its the same quality as CD1... Except CD2 is 1/2 the movie in this case...

So any way tell filebot not to delete the multi file set?

I tried setting ATTRIB +r to the CD's.. but filebot deletes it regardless of the ATTRIB setting.

[rednoah]

I had assumed that this line would create 3 groups (2x Movie / 1x MoviePart CD1 / 1x MoviePart CD2) but it seems to create 1 group with all 4 files in your case:
https://github.com/filebot/scripts/blob ... groovy#L53


You could use the --file-filter option (or something like find -exec) to ignore CD1/CD2 files. That would make the duplicates script work as if those CD1/CD2 files didn't exist. You'll be able to delete at least single file duplicates this way.




EDIT:

FileBot r8118 changes MoviePart.hashCode() to make Movie and MoviePart objects unequal. This will ensure that CD1/CD2 files will not be considered duplicates against no-CD files. For the use case above, that means that no-CD / CD1 / CD2 are now effectively 3 different movies, and since there's only one of each CD1 / CD 2 file neither will end up with matching duplicates.

[seibop]

Awesome! I'm searching for how to get the R8118... but not finding how to get individual releases anywhere obvious

[rednoah]

Awesome! I'm searching for how to get the R8118... but not finding how to get individual releases anywhere obvious

viewtopic.php?t=1609

[seibop]

So... For the benefit of all... I downloaded the beta. I'm not finding installation instruction and simply coping the files over the old filebot files doesn't work. Where can I find installation instructions for the beta?

After copying the files I get

Code: Select all

Error during startup: Access Denied: C:\Program Files\FileBot\data\cache\0\.lock
java.nio.file.AccessDeniedException: C:\Program Files\FileBot\data\cache\0\.lock
        at net.filebot.DiskStore.acquireDiskStore(Unknown Source)
        at net.filebot.DiskStore.acquire(Unknown Source)
        at net.filebot.Main.main(Unknown Source)

[rednoah]

1. Download and extract the Portable ZIP to your default Downloads folder
2. Click on filebot.launcher.exe to launch FileBot GUI. Run /path/to/filebot.exe to run FileBot CLI.


DO NOT copy & paste the portable application files into the C:\Program Files\FileBot folder or any other write-protected system folder. The portable package uses its application folder for application data which we can't access if the application folder is write-protected.

[seibop]

Substancially better. in treating CD? as a different movie, it prevents deletion of 1/2 the file. But if needing to compare CD1 vs no none... would leave two copies.

Code: Select all

c:\tools\fb\filebot -script fn:duplicates "d:\media\movies\my adventures with santa (2019)" --action test

[*] My Adventures with Santa (2019)
[+] 1. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).mp4
[-] 2. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).avi
[-] 3. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).mkv
[*] My Adventures with Santa (2019) [CD1]
[+] 1. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD1.mkv
[-] 2. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD1.mp4
[*] My Adventures with Santa (2019) [CD2]
[+] 1. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD2.mkv
[-] 2. D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD2.mp4

Code: Select all

filebot -script fn:xattr "d:\media\movies\my adventures with santa (2019)"

D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).avi
        net.filebot.filename: My.Adventures.with.Santa.2019.BRRip.XviD.AC3-EVO.avi
        net.filebot.metadata: {"@type":"Movie","year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).eng.srt
        net.filebot.filename: My Adventures with Santa (2019).en.srt
        net.filebot.metadata: {"@type":"Movie","year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).mkv
        net.filebot.filename: my.adventures.with.santa.2019.dvdrip.x264-fragment.mkv
        net.filebot.metadata: {"@type":"Movie","year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).mp4
        net.filebot.filename: My.Adventures.with.Santa.2019.1080p.BluRay.x265-RARBG.mp4
        net.filebot.metadata: {"@type":"Movie","year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019).rar
        net.filebot.filename: My.Adventures.with.Santa.2019.1080p.BluRay.x265-RARBG.mp4.part1.rar
        net.filebot.metadata: {"@type":"Movie","year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD1.mkv
        net.filebot.filename: BmcBuo8HeR.mkv
        net.filebot.metadata: {"@type":"MoviePart","partIndex":1,"partCount":2,"year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD1.mp4
        net.filebot.filename: My.Adventures.with.Santa.2019.1080p.BluRay.H264.AAC-RARBG.mp4
        net.filebot.metadata: {"@type":"MoviePart","partIndex":1,"partCount":2,"year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD1.rar
        net.filebot.filename: My.Adventures.with.Santa.2019.1080p.BluRay.H264.AAC-RARBG.mp4.part01.rar
        net.filebot.metadata: {"@type":"MoviePart","partIndex":1,"partCount":2,"year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD2.mkv
        net.filebot.filename: BmcBuo8HeR.mkv
        net.filebot.metadata: {"@type":"MoviePart","partIndex":2,"partCount":2,"year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD2.mp4
        net.filebot.filename: My.Adventures.with.Santa.2019.1080p.BluRay.H264.AAC-RARBG.mp4
        net.filebot.metadata: {"@type":"MoviePart","partIndex":2,"partCount":2,"year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
D:\media\Movies\My Adventures with Santa (2019)\My Adventures with Santa (2019) CD2.rar
        net.filebot.filename: My.Adventures.with.Santa.2019.1080p.BluRay.H264.AAC-RARBG.mp4.part01.rar
        net.filebot.metadata: {"@type":"MoviePart","partIndex":2,"partCount":2,"year":2019,"imdbId":11023738,"tmdbId":644435,"language":"en","id":644435,"name":"My Adventures with Santa","aliasNames":[]}
Done ?(?????)?

[rednoah]

Well, we'll have to leave that one as a known limitation. Fixing this corner case would effectively require a rewrite of the script with this specific corner case in mind first and foremost.


You're welcome to try your hand at it though, and see if you can come up with an elegant solution:
https://github.com/filebot/scripts/blob ... tes.groovy