Interest for fn:duplicates VMAF comparison

All your suggestions, requests and ideas for future development
Post Reply
devster
Posts: 417
Joined: 06 Jun 2017, 22:56

Interest for fn:duplicates VMAF comparison

Post by devster »

Would there be an interest to add to the fn:duplicates script a way to automatically run VMAF to compare the two files?
I believe ffmpeg is already built with lavfi filter in the default distribution, log output can be json.
I'm currently running comparisons like so:

Code: Select all

systemd-run --user -u ffqm /usr/bin/docker run -i --rm -v /media/media:ro -v /tmp:/tmp ffmpeg-quality-metrics:latest ffmpeg -i 'reference.file.mkv' -i 'comparison.file.mkv -lavfi "[0:v]settb=AVTB,setpts=PTS-STARTPTS[ref]; [1:v]settb=AVTB,setpts=PTS-STARTPTS[cmp]; [cmp][ref]libvmaf=log_fmt=xml:log_path=/dev/null:model=path=/model/vmaf_v0.6.1.json:n_threads=4:feature=name=float_ms_ssim|name=ciede|name=cambi" -hide_banner -f null -
just beware, the process is very long.
Final output looks something like:

Code: Select all

Jan 24 02:23:03 hostname docker[1266492]: [48.0K blob data]
Jan 24 02:39:40 hostname docker[1266492]: [48.0K blob data]
Jan 24 02:56:14 hostname docker[1266492]: [48.0K blob data]
Jan 24 03:12:39 hostname docker[1266492]: [48.0K blob data]
Jan 24 03:28:25 hostname docker[1266492]: [48.0K blob data]
Jan 24 03:34:52 hostname docker[1266492]: [25.2K blob data]
Jan 24 03:34:52 hostname docker[1266492]: video:26262kB audio:438128kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Jan 24 03:34:54 hostname docker[1266492]: [Parsed_libvmaf_4 @ 0x7925c80] VMAF score: 12.884485
The n_threads option can be customised as well as which features to use and the specific model.
I only work in black and sometimes very, very dark grey. (Batman)
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Interest for fn:duplicates VMAF comparison

Post by rednoah »

Relying heavily on external tools that may not be installed for the majority of users (i.e. Windows and macOS) is not planned. That said, you can modify the script to fit your needs.


I could imagine allowing for a custom Groovy expression to serve as comparator:

Code: Select all

--order "{ f.xattr.VMAF as float }"
(pseudo code; not implemented)


But you would definitely have to have some custom script compute all the VMAF values for each file in advance and store them in an easy-to-read manner:

Code: Select all

args.files.each{
    it.xattr.VMAF = ['ffmpeg', ..., it, ...].execute().text.match(/VMAF score: ([0-9.])/).toFloat()
}
(pseudo code; not tested)
:idea: Please read the FAQ and How to Request Help.
devster
Posts: 417
Joined: 06 Jun 2017, 22:56

Re: Interest for fn:duplicates VMAF comparison

Post by devster »

The only dependency would be ffmpeg (recent version but can't find a specific number).
VMAF score is unfortunately only possible using 2 files, and it represent relative quality of the comparator to a reference file. I'm not sure it can be calculated for each file and stored, it'd have to be done for each duplicate specifically.
The idea would be to use fn:duplicates to find actual duplicates and compare relative quality pair by pair using the highest bitrate and resolution as reference file.
Could it be possible to use --apply to run a command on duplicate sets (maximum of 2 files per comparison), without modifying duplicates.groovy?
Maybe like so:

Code: Select all

filebot -script fn:duplicates /media --apply VMAF
I'm afraid I never really understood how apply works or where it's getting the input from, for example for stuff like --apply clean or --apply url.
I only work in black and sometimes very, very dark grey. (Batman)
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Interest for fn:duplicates VMAF comparison

Post by rednoah »

The --apply post-processing features applies predefined actions on preselected files, i.e. renamed files after -rename calls, or selected files after -find calls.

If you add --apply to -script fn:duplicates then the given post-processing feature is called on the superfluous duplicate files, explicitly not including the "best" duplicate file that we want to keep. This is not every useful, except maybe --apply reveal to reveal duplicates in the file manager to manually manage. The -exec action works exactly the same, but will be much more useful in the context of -script fn:duplicates because it's not limited to the predefined post-processing actions, e.g. -exec mv -v {f} /duplicates to move superfluous duplicates away instead of deleting them.
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22899
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Interest for fn:duplicates VMAF comparison

Post by rednoah »

I've added experimental support for custom comparison functions:
https://github.com/filebot/scripts/comm ... 2a7e3601d8


e.g. "do nothing" custom comparator:

Code: Select all

--order '{ a, b -> 0 }'
Compares its two arguments for order. Returns a negative integer, zero, or a positive integer as the first argument is less than, equal to, or greater than the second.

:!: That said, if you pass in complex code via the --order parameter, then making your own fork might just be easier, because duplicates.groovy is rather simple, and really just boils down 3-4 lines of code if you strip away all the optional and configurable logic:

Code: Select all

args.getFiles{ it.video }.groupBy{ it.metadata }.each{ g, fs ->
	println "$g | $fs.name"
}
:idea: Please read the FAQ and How to Request Help.
Post Reply