Failed to process [...] because [...] is an exact copy and already exists

[Matt]

On the CLI I tried to use action copy and skip if the file already exist but it's failing if the file is already there:

Code: Select all

Failed to process [/media/animes/Downloads/[HorribleSubs] Major 2nd - 21 [1080p].mkv] because [/media/animes/Watching/Major 2nd/S1/S01E21 [HorribleSubs] Major 2nd - 21 [1080p].mkv] is an exact copy and already exists

Shouldn't the file be skipped? With action test:

Code: Select all

Skipped [/media/animes/Downloads/[HorribleSubs] Major 2nd - 21 [1080p].mkv] because [/media/animes/Watching/Major 2nd/S1/S01E21 [HorribleSubs] Major 2nd - 21 [1080p].mkv] already exists

[rednoah]

Code: Select all

Failed to process [...] because [...] is an exact copy and already exists

This is a sanity check so you can't process the same files over and over and over again in an infinite loop. Indicating a serious issue with your way of using FileBot. It prevents people from processing their whole library over and over after adding 1-2 files, causing unnecessary load on online services.

--action test can help you test if it might work, but it's no guarantee as --action test will not touch files. This particular failure cannot be anticipated until files are processed and compared physically.

[Matt]

Indicating a serious issue with your way of using FileBot. It prevents people from processing their whole library over and over after adding 1-2 files, causing unnecessary load on online services.

My needs:
- Source folder is a sync from remote server. Since it's a mirror, I need to keep original files.
- use filebot to process and rename files with copy
- the script is run each time a change is done, usual frequency is once a day

copy + skip was a good option for me. AMC is not working because I need to use "--q" and "--filter" for filebot to find the correct serie/season.
Is there a way to record and exclude already processed files?
Please let me know if there is a way to make that work.

[rednoah]

1.
The amc script does support --filter (but not --q) and has options to keep track of files that have been processed already.


2.
If you must use -rename calls then you'll need to find a way to exclude files that have already been processed. Seems like a simple time stamp is good enough in your case, i.e. only process files that have been created / modified in the last 24 hours.

e.g.

Code: Select all

find /path/to/files -type f -mtime -1 -exec filebot -script fn:sysenv {} +

Alternatively, depending on how you sync files, you may also be able to simply have your sync tool give you a list of files that have been synced every time, and then feed that list into FileBot.
Alternatively, there might also be tools that can help you keep track of new / old files and the difference and give you a list that you can then feed into FileBot.

[chipinnc]

RedNoah, For some reason you seem to be BLAMING users for using these scripts in a way that YOU disagree with.

The standard FileBot processing has always been to SKIP files that are existing on the destination and continue with the rest of the files in the directory. This has changed, and isn't acceptable.

It doesn't matter WHY people are doing it, it matters that apparently someone made a fundamental change to the way the script works. Since FileBot is now PAID software, we are the customers. We want it back the way it was. Why is that a reason for you to berate people for using it the way we have for years?

[rednoah]

I have to balance the well-being of online databases that all users of FileBot rely on, against potential abuse (often accidental) of these services by a small minority of FileBot users.

The vast majority of users won't run into this corner case, but if you do, then there's typically an easy work around that'll make things work for you and not abuse online databases, e.g. using --def excludeList is usually the easy solution for amc script users.

If you indeed find yourself having a unique unfixable setup that relies on processing the same files over and over, then FileBot is unfortunately not designed for your use case, and I'm happy to issue refunds in this case.


EDIT:

But on the off-chance you're interested in correcting your setup, then I'm always happy to help just as well.

[chipinnc]

As I said in my PM, there's nothing in my setup that needs "correcting", other than having a fully functional FileBot application. Since you've installed yourself as the arbitrator of all that's allowed on the internet, it doesn't look like that "correction" will be forthcoming.

It's so sad that some people feel they have to right to control and micromanage others, and seem to feel justified and empowered as well. I wonder how the folks in Hong Kong feel about that type of mentality?? I guess Taipei is on track for the same, but you'll fit right in with a strong mainland rule, right?

[rednoah]

1.
Please describe your general setup, and explain why you end up processing identical files (i.e. identical according to file content + last modified) multiple times in an automated setup. So that I can understand your unique use case and perhaps fine-tune FileBot a little bit to make it work better for users like you, if sensible.

Apparently you believe you know every "valid" situation that a user would have and anyone with a different situation is wrong, stupid, and in need of your expert guidance to make it through the day.

I do not. In particular, I do not know your use case or situation. You running into this sanity check may or may not be a bug. I can't say way which until I understand the situation. The first step to helping you find a solution, is for me to understand your situation, and then discuss possible solutions or workarounds, and find one that works well for you.

Since you have a high level of technical expertise, that's great, because it'll make things easier for the both of us. The vast majority of FileBot users that need my help have no Linux or command-line skills, which can be frustrating at times.



2.
If you'd like to have a Trekkian discussion on "the good of the many outweighs the good of the one" or cross-straight politics, please create dedicated threads in the Anything and Everything else forum. I'll play.

[rednoah]

The duplicates script has been enhanced with an option --mode binary which can be used to detect (and delete) binary duplicates ahead of time, and thus ensure "exact copy and already exists" class issues cannot happen during subsequent regular amc script processing:

Code: Select all

filebot -script fn:duplicates --mode binary /my/media/folder /newly/added/files

viewtopic.php?f=4&t=5&p=23171#p23171

[rednoah]

Re-organize previously organized files using local xattr metadata