Data Repositories

This is a list of public dataset repositories we aim to connect to for getting more varied datasets in OpenML. These have widely varying data formats, so we need both manual selection plus automatic conversion or meta-data extraction to make them easily usable.

• A collection of sources made by different users

  • https://github.com/caesar0301/awesome-public-datasets
  • https://dreamtolearn.com/ryan/1001_datasets

• Machine learning dataset repositories (mostly already in OpenML)

  • UCI: https://archive.ics.uci.edu/ml/index.html
  • KEEL: http://sci2s.ugr.es/keel/datasets.php
  • LIBSVM: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
  • AutoWEKA datasets: http://www.cs.ubc.ca/labs/beta/Projects/autoweka/datasets/

• Time series data:

  • UCR: http://timeseriesclassification.com/
  • Older version: http://www.cs.ucr.edu/~eamonn/time_series_data/

• Causality related datasets:

  • http://www.causality.inf.ethz.ch/repository.php

• Deep learning datasets (mostly image data)

  • http://deeplearning.net/datasets/
  • https://deeplearning4j.org/opendata
  • http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html
  • https://www.kaggle.com/crawford/emnist

• Extreme classification:

  • http://manikvarma.org/downloads/XC/XMLRepository.html

• MLData (will merge with OpenML in 2018)

  http://mldata.org/

• AutoWEKA datasets:

  http://www.cs.ubc.ca/labs/beta/Projects/autoweka/datasets/

• Kaggle public datasets

  https://www.kaggle.com/datasets

• RAMP Challenge datasets

  http://www.ramp.studio/data_domains

• Wolfram data repository

  http://datarepository.wolframcloud.com/

• Data.world

  https://data.world/

• Figshare (needs digging, lots of Excel files)

  https://figshare.com/search?q=dataset&quick=1

• KDNuggets list of data sets (meta-list, lots of stuff here):

  http://www.kdnuggets.com/datasets/index.html

• Benchmark Data Sets for Highly Imbalanced Binary Classification

  http://www.cs.gsu.edu/~zding/research/imbalance-data/x19data.txt

• Feature Selection Challenge Datasets

  http://www.nipsfsc.ecs.soton.ac.uk/datasets/ http://featureselection.asu.edu/datasets.php

• BigML's list of 1000+ data sources

  http://blog.bigml.com/2013/02/28/data-data-data-thousands-of-public-data-sources/

• Massive list from Data Science Central.

  http://www.datasciencecentral.com/profiles/blogs/data-sources-for-cool-data-science-projects

• R packages (also see https://github.com/openml/openml-r/issues/185)

  • http://stat.ethz.ch/R-manual/R-patched/library/datasets/html/00Index.html
  • mlbench
  • Stata datasets: http://www.stata-press.com/data/r13/r.html

• UTwente Activity recognition datasets:

  http://ps.ewi.utwente.nl/Datasets.php

• Vanderbilt:

  http://biostat.mc.vanderbilt.edu/wiki/Main/DataSets

• Quandl

  https://www.quandl.com

• Microarray data:

  http://genomics-pubs.princeton.edu/oncology/ http://svitsrv25.epfl.ch/R-doc/library/multtest/html/golub.html

• Medical data:

  http://www.healthdata.gov/
  http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/PPRPAGES/pprdat.htm
  http://hcup-us.ahrq.gov/
  https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/Medicare-Provider-Charge-Data/Physician-and-Other-Supplier.html
  https://nsduhweb.rti.org/respweb/homepage.cfm
  http://orwh.od.nih.gov/resources/policyreports/womenofcolor.asp

• Nature.com Scientific data repositories list

  https://www.nature.com/sdata/policies/repositories