submission: uploading hepdata tarballs via API / CLI

[kratsg]

This is a proposal/feature [and I'm not sure this is currently supported in HEPData yet].

The idea of the workflow is that when a HEPData record is created / instantiated, it also comes with an access token (or "client secret") that allows for API access for that particular record. Then I could imagine something like this happening: hepdata upload foo.tar.gz --token ABC123abc$#@! which will upload the tarball (or update the existing one w/ versioning).

This would be useful to somehow automate the entire workflow for submission to HEPData on some level.

[GraemeWatt]

Thanks for opening this issue. I remember you made this suggestion after my talk at the LHC reinterpretation workshop in London last year and I meant to record it as a GitHub issue. I think Zenodo already has this functionality (see https://developers.zenodo.org), so it would probably be possible with HEPData if the appropriate software tools were developed.

[kratsg]

This might be very much ATLAS-specific, but I'll explain our organization a little bit. We use GitLab for our publication process, and make a group for each analysis -- which contains multiple repositories for the internal documentation as well as the paper in-progress.

I was thinking of extending this to also create a "HEPData" project/repository as well, which would contain the CI that does the HEPData checks we do internally in the collaboration, as well as preparing the HEPData for submission, and then submit ("deploy"). It would finish up that chain there, plus makes it very highly-trackable for us.

[GraemeWatt]

@kratsg: CLI or API upload to HEPData records is now possible with v0.2.0 of hepdata-cli released today. The appropriate upload command is included in the automated invitation emails sent to Uploaders and it could be run in a CI/CD workflow. We decided to require authentication with an email address and password in addition to an access token for greater security. Please ensure that the user's password is not stored in plain text on any GitLab repositories (even if access is restricted to ATLAS collaborators). Credentials could be set as encrypted environment variables passed to the hepdata-cli upload command, e.g.

hepdata-cli upload submission.tar.gz -e $HEPDATA_EMAIL -r $HEPDATA_RECORD -i $HEPDATA_COOKIE -s False -p $HEPDATA_PASSWORD

[kratsg]

./cc @madbaron