JSON Schemas v1.0.0

[M-casado]

Ticket reference

EEH-2630

Overall changes

• Create JSON Schemas for canonical ENA's XSD metadata schemas, defining the EGA's metadata JSON Schema v1.0.0. Modifications were kept to a minimum, restricted only to the difference between formats in XML and JSON.

Given that there are many accessory files that are modified along the schemas, I would recommend to focus on the schemas and json_validation_tests directories.

For further details, see CHANGELOG. I also moved main to dev, until v2.0.0 schemas are released, to comply with the release documentation. This way, v1.0.0 will be released as a TAG, but also will be the main repository, and development (of v2.0.0) will be kept within the dev branch.

Future TO-DOs and checks

• Check that the object's versions (i.e. meta:version) are updated before creating the PR (see GitHub Action).
• Check that the version manifest is up to date right before merging (see documentation).
• Check that all JSON documents are valid against their respective schemas.
• Check that metadata validation works with the tag pointers (v1.0.0). In other words, Biovalidator downloads automatically the JSON Schemas from this GH repo when schema-agnostic validation of the documents is done. No errors from:

$ python3.8 .github/scripts/request_validation.py "examples/json_validation_tests/" "http://localhost:3020/validate"

• Create v1.0.0 tag from v1.0.0 branch. Once it's been reviewed.
• Check that validation works from tag (should be the same as in the code above)
• Modify, within this PR, the pointers back to main, replacing v1.0.0 in both the $id$ of JSON Schemas and $ref of JSON documents. See documentation.
• Check that validation with main branch pointers works.

[M-casado]

Required GH actions failing to install dependencies. Will fix them tomorrow and re-run workflows.

[M-casado]

How to test that the JSON Documents are valid?

In batch

• Have Biovalidator installed.
• Deploy the Biovalidator server on a terminal, and keep it open. It will open the validation endpoint http://localhost:3020/validate:

node src/biovalidator

• Clone this metadata repository and open another terminal to send validation requests:

cd ega-metadata-schema/

• Execute the request_validation.py python script. You may need to install some dependencies for this, use pip to do so.

python3.8 .github/scripts/request_validation.py "examples/json_validation_tests/" "http://localhost:3020/validate"

If the script ends without any reported issues (i.e., the number of files with errors is 0), great news: everything inside the json_validation_tests directory was validated correctly. Notice how the first time you run it takes a few seconds for the server to download the schemas from GH, but it runs fast once downloaded.

One by one

If you want to validate the files one by one manually, leave the Biovalidator server open, and in the other terminal run:

cd examples/json_validation_tests
curl --data @dataset_valid-1.json -H "Content-Type: application/json" -X POST "http://localhost:3020/validate"

Again, if the result is [], there are no validation errors. Otherwise, it will be a list of errors.

Be imaginative

If you're planning on testing the validation, I would recommend to get into the JSON Documents, and randomly change something that you think should be required. If there is no error, either it wasn't as required as expected, or you found a bug 👏

[M-casado]

GH actions were fixed at #91 - The only one failing is the linter.