Some files still left behind

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Some files still left behind

Post by Achandab » 18 Oct 2019, 14:51

Using file node it seems some files are still left behind

Code: Select all

Hustlers.2019.720p.HDCAM.900MB.getb8.x264-Max-Rls.part20.rar.1
Is there any further options to the file clean up command? Can we possibly delete all files under a certain size?

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 18 Oct 2019, 17:01

What is *.rar.1 supposed to be? That is not a standard way of naming *.rar archives. As a result, the cleaner script won't touch it, because we can't just go around deleting random files.


You can make your own cleaner calls, and then you get to specify your own cleaner rules as well if you're not happy with the defaults:
viewtopic.php?t=5#p1341
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 18 Oct 2019, 17:32

Cheers for the quick response. I'll check it out

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 20 Oct 2019, 03:44

Had a look at the cleaner script and couldn't figure it out

I sorted the previous file clean issue but can't sem to figure how to get rid of the following sample file

the.legend.of.tarzan.2016.2160p.uhd.bluray.x265-terminal.sample.mkv

its 150mb.

Can we modify cleaner script to delete any file with ""sample" in the name and under 500mb?

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 20 Oct 2019, 05:37

rednoah wrote:
21 Jul 2012, 09:50
--def maxsize=bytes files larger than maxsize will never be deleted. Default is 100 MB.
Since sample is already a blacklisted term by default, so that's we're half-way there by default.

So you just need to increase the --def maxsize sanity check which would otherwise ensure you don't accidentally delete important (i.e. large) files:

Code: Select all

--def maxsize=500000000
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 20 Oct 2019, 09:57

File still being ignored for some reason?

It's the Tarzan sample file at 150MB

[email protected]:~# filebot -script 'fn:amc' /volumeUSB1/usbshare/Downloads/comp lete --output /volume1/Media --action Move -non-strict --def ignore="_UNPACK" -- def maxsize=500000000 --order Airdate --conflict auto --lang en --def 'unsorted= y' 'clean=y' 'deleteAfterExtract=y' 'minLengthMS=300000' 'minFileSize=50000000' 'excludeList=.excludes' --log all --log-file '/usr/local/filebot-node/data/fileb ot.log'
Locking /usr/local/filebot-node/data/filebot.log
Run script [fn:amc] at [Sun Oct 20 20:54:27 AEDT 2019]
Parameter: ignore = _UNPACK
Parameter: maxsize = 500000000
Parameter: unsorted = y
Parameter: clean = y
Parameter: deleteAfterExtract = y
Parameter: minLengthMS = 300000
Parameter: minFileSize = 50000000
Parameter: excludeList = .excludes
Argument[0]: /volumeUSB1/usbshare/Downloads/complete
Use excludes: /volume1/Media/.excludes (376)
Ignore pattern: /volumeUSB1/usbshare/Downloads/complete/_UNPACK_Vikings.S02.EXTENDED.1080p.BluRay.x264-ROVERS
Ignore pattern: /volumeUSB1/usbshare/Downloads/complete/_UNPACK_Marco.Polo.2014.S01.1080p.BluRay.x264-ROVERS
Ignore video extra: /volumeUSB1/usbshare/Downloads/complete/The.Legend.of.Tarzan.2016.2160p.UHD.BluRay.x265-TERMiNAL/the.legend.of.tarzan.2016.2160p.uhd.bluray.x265-terminal.sample.mkv
No files selected for processing
Done ¯\_(ツ)_/¯

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 20 Oct 2019, 10:49

I see. You're using the amc script. The option given above refer to the cleaner script.

** Although the amc may call the cleaner script internally, the cleaner script can't be configured or customized at all in this case.

You'll want to do separate calls for amc script and cleaner script if you need to customize the cleaner script logic.
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 20 Oct 2019, 10:50

rednoah wrote:
20 Oct 2019, 05:37
rednoah wrote:
21 Jul 2012, 09:50
--def maxsize=bytes files larger than maxsize will never be deleted. Default is 100 MB.
Since sample is already a blacklisted term by default, so that's we're half-way there by default.

So you just need to increase the --def maxsize sanity check which would otherwise ensure you don't accidentally delete important (i.e. large) files:

Code: Select all

--def maxsize=500000000

Also just to confirm this wont delete any TV shows under 500mb will it?

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 20 Oct 2019, 10:53

I wouldn't count on it, since you're explicitly disabling the safety feature that prevents that from happening.

e.g. If you are processing files like "Trailer Park Boys" that contains the keyword "trailer" and are relatively small in file size then these files may very well end up being deleted.
:idea: Please read the FAQ and How to Request Help.

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 20 Oct 2019, 10:54

I wouldn't count on it, since you're explicitly disabling the safety feature that prevents that from happening.

e.g. If you are processing files like "Trailer Park Boys" that contains the keyword "trailer" and are relatively small in file size then these files may very well end up being deleted.
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 20 Oct 2019, 11:39

rednoah wrote:
20 Oct 2019, 10:49
I see. You're using the amc script. The option given above refer to the cleaner script.

** Although the amc may call the cleaner script internally, the cleaner script can't be configured or customized at all in this case.

You'll want to do separate calls for amc script and cleaner script if you need to customize the cleaner script logic.
Perfect works great. However another file extension was just left behind. So i guess is there a way to define the cleaner script to delete any file under a certain size?

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 20 Oct 2019, 11:41

Answered my own question

--def minsize=bytes

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 20 Oct 2019, 11:48

It seems cleaner won't delete files that are 0 bytes in size? Is that correct?

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 20 Oct 2019, 15:23

Yes. 0 maybe reported by some remote file system implements when there is an error of some kind, meaning that 0 can mean 0 but can also mean anything, so we side with caution and don't delete things.
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 21 Oct 2019, 04:09

Cheers makes sense, thanks

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 28 Oct 2019, 10:38

Files are still being left behind. This is the cleaner script I have running

Code: Select all

filebot -script fn:cleaner /volumeUSB1/usbshare/Downloads/complete --def maxsize=50000000 --def "exts=jpg|nfo|srv|srr|nzb|sfv|idx|sub|txt|part01|part02|par1|par2|0|rar-missing|rev"
Obviously I wont be able to get all extensions. So is there a way to just delete any files under 50mb regardless of extension?

Or another way is to delete any files but movie file formats ie: mkv, avi, mp4 etc

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 28 Oct 2019, 12:40

Why not just use find then?
https://superuser.com/questions/644272/ ... subfolders


:idea: The cleaner script is primarily about not accidentally deleting important files. If you just wanna delete files, plain and simple, without safety features, then the find command is more suitable for the task at hand.
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 28 Oct 2019, 19:13

Thanks I’ll check that out

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 29 Oct 2019, 03:52

Find won’t work as it will delete files from the “unpack” folders.
Is there a way for cleaner to work in reverse so it deletes anything but the extensions you define? So we can specify .mkv, avi etc in the -def switch and it won’t delete those.

Alternatively can we put wildcards in exts. At the moment I have extensions of .r1, .r2, - .r98, .r99. So can we define any extension that has the letter R in it?

User avatar
rednoah
The Source
Posts: 16765
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Some files still left behind

Post by rednoah » 29 Oct 2019, 04:23

1.
Achandab wrote:
29 Oct 2019, 03:52
Is there a way for cleaner to work in reverse so it deletes anything but the extensions you define? So we can specify .mkv, avi etc in the -def switch and it won’t delete those.
Not really, but you could write your own cleaner script, and then do whatever you want:
https://github.com/filebot/scripts/blob ... ner.groovy


2.
Achandab wrote:
29 Oct 2019, 03:52
Alternatively can we put wildcards in exts. At the moment I have extensions of .r1, .r2, - .r98, .r99. So can we define any extension that has the letter R in it?
Yes, --def exts expects are regular expression, so for example, --def exts='r[0-9]+' will match all extensions from .r1 to .r999 and beyond.
:idea: Please read the FAQ and How to Request Help.

Achandab
Donor
Posts: 109
Joined: 04 Nov 2013, 21:20

Re: Some files still left behind

Post by Achandab » 29 Oct 2019, 04:35

Perfect thank you!

Post Reply