Detect duplicates

[Daniel-Mietchen]

Fig. 1 of
https://en.wikisource.org/w/index.php?title=Wikisource:WikiProject_Open_Access/Programmatic_import_from_PubMed_Central/Modelling_the_Species_Distribution_of_Flat-Headed_Cats_%28Prionailurus_planiceps%29_an_Endangered_South-East_Asian_Small_Felid&oldid=5032599
was imported into
https://commons.wikimedia.org/wiki/File:Modelling-the-Species-Distribution-of-Flat-Headed-Cats-%28Prionailurus-planiceps%29-an-Endangered-South-pone.0009612.g001.jpg
but the image there already existed (in higher resolution) as
https://commons.wikimedia.org/wiki/File:Plionailurus_planiceps.png .
According to Commons policies, our upload should thus be deleted.

In such cases, it would be best if we could
(a) detect such a duplicate before upload
(b) post a message on that file's talk page with the proper metadata.

[notconfusing]

Its because the sizes are different. We have been over this problem before
(though I can't find an issue for it). Without implementing computer vision
algorithms it'll be diffucult to detect. The other avenue we tried was to
get pubmed to give us the maximum resolution images they had, but after
some time they responded that their API will not support this. So we need
some fresh ideas.

Max Klein
‽ http://notconfusing.com/

On Sun, Sep 7, 2014 at 2:50 PM, Daniel Mietchen [email protected]
wrote:

Assigned #34 #34 to
@notconfusing https://github.com/notconfusing.

—
Reply to this email directly or view it on GitHub
#34 (comment).

[jure]

I think you're right, and this problem won't be easily solved without some image similarity magic. There's a good list of (and discussion about) applicable open source solutions here: http://ejohn.org/blog/image-similarity-search-wanted

For Python specifically, this looks pretty useful: http://www.guguncube.com/1656/python-image-similarity-comparison-using-several-techniques

[notconfusing]

thanks @jure i've never looked into python image similarity before that seems to a be a good starting point.