App to compare video files based on content, not file attributes

[BarkingGhostar]

Morning all, are there any proven apps that can compare two or more video files at a time and determine duplication based on the video content, like a video version of a still image histogram? Let's say I have a folder with >>100 video files and I know somewhere in there is a handful of duplicates.

Using file size, compression, dimension (resolution), etc., could be coincidence so comparing at something more discrete is desired. I know this can be done for still images but ...

 

[XavierMace]

That seems terribly impractical. It would have to play each video all the way through and what about two videos that are the same but just different codecs, would those still be considered duplicates? I can't picture that being resource friendly. Or cheap.

 

[Chaotic42]

It would be fairly unlikely for you to find two non-trivial videos which are different but have the same file size. If you do, maybe run an MD5 sum on them? If their size is the same and so is their MD5 sum, they're almost certainly the same. Now if you have two videos which have the same content but potentially different codecs, you might want to try grabbing frames at random time indices and running them through some image comparison algorithm like this one: https://huddle.github.io/Resemble.js/

 

[AMDisTheBEST]

Comparing sizes is woefully inaccurate but can narrow down the candidates. Interesting... maybe I will code one such app as my next project.