This will take an xlsx file and generate a project_0.json suitable for uploading to the DSS.
This will then upload the project_0.json and a (mostly) empty links.json which will populate a new project in
the browser.
Source a new environment and install dependencies:
virtualenv -p python3.6 v3nv && . v3nv/bin/activate && pip install -r requirements.txt
Parse the xlsx:
#!/usr/bin/env bash
# E-GEOD-81547_curated_ontologies_07_2019.xlsx
# DSS prod uuid: cddab57b-6868-4be4-806f-395ed9dd635a
python xlsx_to_project_json.py --xlsx data/test_000.xlsx
# Gary_Bader_9_16.xlsx
# DSS prod uuid: 4d6f6c96-2a83-43d8-8fe1-0f53bffd4674
python xlsx_to_project_json.py --xlsx data/test_001.xlsx
# GEOD-93593_HCA_Ontologies_July_2.xlsx
# DSS prod uuid: 2043c65a-1cf8-4828-a656-9e247d4e64f1
python xlsx_to_project_json.py --xlsx data/test_002.xlsx
# hca-metadata-spreadsheet-GSE84133_pancreas.xlsx
# DSS prod uuid: f86f1ab4-1fbb-4510-ae35-3ffd752d4dfc
python xlsx_to_project_json.py --xlsx data/test_003.xlsx
# hca-metadata-spreadsheet-GSE95459-GSE114374-colon.xlsx
# DSS prod uuid: f8aa201c-4ff1-45a4-890e-840d63459ca2
python xlsx_to_project_json.py --xlsx data/test_004.xlsx
# mf-E-GEOD-106540_spreadsheet_v9.xlsx
# DSS prod uuid: 90bd6933-40c0-48d4-8d76-778c103bf545
python xlsx_to_project_json.py --xlsx data/test_005.xlsx
Adding --upload true will upload the data to the DSS.
Note that UUID's are now always programmatically generated from GEO accessions
and cannot be provided via the commandline.
NOTE:
Edited the following fields in "data/test_004.xlsx":
publications.publication_url -> publications.url
publications.publication_title -> publications.title
NOTE
6 ORIGINAL DATASETS (ALREADY IN THE DSS):
spreadsheets/existing/*.xlsx are the original excel files provided that
currently exist in dss prod and we have finished examples to compare against.
71 RAW DATASETS (STATUS NOT PARSED)
The xlsx files in spreadsheets/new were downloaded from a spreadsheet of
spreadsheets and assumed to be (mostly) complete projects. These inputs were
provided with the labels "finished" or "full". Differences assumed are inferred
from skimming over the files. I chose to use the inputs which end in ".0.xlsx"
("finished") rather than the normal ".xlsx" extension ("full").
These are missing fields such as the "funders" section (as opposed to the 6 excel files above). Not sure of other differences yet.