Open Images is a dataset of ~9 million URLs to images that have been annotated with labels spanning over 6000 categories.
The annotations are licensed by Google Inc. under CC BY 4.0 license.
The images are listed as having a CC BY 2.0 license. Note: while we tried to identify images that are licensed under a Creative Commons Attribution license, we make no representations or warranties regarding the license status of each image and you should verify the license for each image yourself.
- Image URLs and metadata (656 MB) – Added a "Subset" column to make it easier to import the annotations into PostgreSQL)
- Machine image-level annotations (train and validation sets) (330 MB)
- Human image-level annotations (validation set) (7 MB)
Each image has a unique 64-bit ID assigned. In the CSV files they appear as zero-padded hex integers, such as 000060e3121c7305. The dataset is split into a training set (9011219 images) and a validation set (167057 images). Each image has zero, one or more image-level labels assigned. Both sets have machine-populated annotations, while the validation set also has human annotations. The raters have been asked to validate the machine annotations, which allowed to practically eliminate false positive from the validation set (but not false negatives).
Labels are so called mids as can be found in Freebase or Google Knowledge Graph API. A short description of each label is available in dict.csv. There are 7844 distinct labels attached to at least one images, but only around 6000 labels are considered 'trainable' with at least 50 images in the validation set and at least 20 images in the training set.
Each annotation has a confidence number from 0.0 to 1.0 assigned. The human annotations are definite (either positive, 1.0 or negative, 0.0), while machine annotations have fractional confidences, generally, >= 0.5. The higher confidence, the smaller chance for the label to be a false positive.
The data tarballs contain CSV files of two types:
There's one such file for each subset inside train and validation subdirectories. It has image URLs, their OpenImages IDs, titles, authors and license information:
ImageID,Subset,OriginalURL,OriginalLandingURL,License,AuthorProfileURL,Author,Title
...
000060e3121c7305,train,https://c1.staticflickr.com/5/4129/5215831864_46f356962f_o.jpg,\
https://www.flickr.com/photos/brokentaco/5215831864,\
https://creativecommons.org/licenses/by/2.0/,\
"https://www.flickr.com/people/brokentaco/","David","28 Nov 2010 Our new house."
The data is as it appears on the destination websites.
The CSVs of this type attach labels to image IDs:
ImageID,Confidence:Labels...
...
000060e3121c7305,0.9:/m/05wrt,0.9:/m/06ht1,0.7:/m/01l0mw,0.7:/m/03d2wd,0.7:/m/03nxtz,0.6:/m/020g49,\
0.6:/m/023907r,0.6:/m/02rfdq,0.6:/m/038t8_,0.6:/m/03f6tq,0.6:/m/0l7_8,0.5:/m/01nblt,0.5:/m/01s105
...
These can be converted to their short descriptions by looking into dict.csv:
"/m/05wrt","property"
"/m/06ht1","room"
"/m/01l0mw","home"
"/m/03d2wd","dining room"
"/m/03nxtz","cottage"
"/m/020g49","hardwood"
"/m/023907r","real estate"
"/m/038t8_","estate"
"/m/03f6tq","living room"
"/m/0l7_8","floor"
"/m/01nblt","apartment"
"/m/01s105","cabinetry"
The distribution of labels across the images is highly uneven with some labels attached to more than a million images, while others to less than 100:
While the machine annotations are somewhat noisy, in general, the labels with more images are more accurate:
We have trained an Inception v3 model based on Open Images annotations alone, and the model is good enough to be used for fine-tuning applications as well as for other things, like DeepDream or artistic style transfer which require a well developed hierarchy of filters. We hope to improve the quality of the annotations in Open Image the coming months, and therefore the quality of models which can be trained.