Mint DOIs for resources and update metadata with that from DataCite #412

alisonrclarke · 2021-10-28T14:22:23Z

Also includes a couple of other fixes from @GraemeWatt.

Release Notes

Requires a new doi field to be added to the data_resource table:

ALTER TABLE dataresource ADD COLUMN doi varchar(128);

(See script)

Once this has been pushed to prod, the DataCite metadata should be updated and resources should be added, using the doi-utils command, probably in batches, e.g.:

hepdata doi-utils register-dois -i all -s 1 -e 2000

(Then -s 2001 -e 4000 etc.)

* Remove line breaks in text elements * Replace IsSupplementedBy with HasPart to ensure data table DOIs are included in JSON-LD

…ion)

* Fixes #406 (hide button unless user is Admin or Coordinator).

* Fixes #403 (modify update_record_info for unfinished records).

* Move the subtitle for data tables and resource files to the title. * Improves results shown in DataCite Search and Google Dataset Search.

@misc

* Give citation information for the versioned DOI of a record. * Follow DataCite preferred format (e.g. include resourceTypeGeneral). * Use BibTeX keys based on the last part of the DOI after first '/'. * Only use standard BibTeX fields for @misc type (closes #408).

Revise HTML/BibTeX content of Cite widget and other changes

coveralls · 2021-10-28T14:49:52Z

Coverage increased (+0.4%) to 88.502% when pulling e460170 on resource-dois into d8ad04a on master.

* Fix behaviour if record has multiple versions (previously incorrect). * Ensure sensible display for unfinished records with missing fields. * Take year from last updated date, different from publication year. * Allow for rivet URL argument when redirecting (fixes #413).

* Fixes #414.

Removing it so it doesn't affect other tests.

Modified E2E tests to include an extra submission with resources in default data.

Also added tests for resource content negotiation

See #415

See #419

* Avoid calling switch_table so clipboard/json links are not mangled * Add filename to header * Fix issue with hide/show publication info

* Fixes #425.

GraemeWatt

Thanks for all your work on this, @alisonrclarke! I'll merge now.

alisonrclarke and others added 27 commits October 14, 2021 13:52

First stab at adding DOIs for resources

1c38adc

Add 'file' to file type

3d99074

Ensure resource DOIs are reserved alongside data DOIs

f79f8a5

Add landing page for resources

d3f1a6e

Remove breakpoint

e67f5e1

Update DOI URL to go to landing page

e4ae7c9

Add DOIs to resources widget

acd3902

Add resources to citation modal, and update styling

d14dc66

Add JSON-LD metadata from DataCite

e7ea007

Remove schema.org microdata (and unused sections)

92bcfd3

Improvements to JSON-LD generated from DataCite metadata

5109bef

* Remove line breaks in text elements * Replace IsSupplementedBy with HasPart to ensure data table DOIs are included in JSON-LD

Don't try to get json-ld from DataCite if sandbox or if TESTING=True

60446e3

Add content_url to JSON-LD coming from DataCite

3cf248a

Use Accept header to determine whether to send file instead of page

23d4325

Add DataDownload to JSON-LD and ensure correct doi metadata is used

5f27d4e

Retrieve JSON-LD on server side so it's present in page source

0266f79

Modify data submission DOIs to go to related_record (not main submiss…

c537ca2

…ion)

dashboard: hide "Attach INSPIRE Record" button

bafc26e

* Fixes #406 (hide button unless user is Admin or Coordinator).

records: fix update_record_info for 'todo' records

0ef1e5d

* Fixes #403 (modify update_record_info for unfinished records).

datacite: combine title/subtitle in DataCite XML

b431129

* Move the subtitle for data tables and resource files to the title. * Improves results shown in DataCite Search and Google Dataset Search.

Merge pull request #410 from HEPData/update-bibtex

51c6c81

Revise HTML/BibTeX content of Cite widget and other changes

Add json-ld to homepage and make it all Google-compatible

5d31a8f

Add noindex meta tag unless on main hepdata site

c910f7b

Fix redirect when adding resource to unfinished submission

51dab96

Check data submissions and resources have DOI as well as submission

be04d3c

Merge branch 'master' into resource-dois

ffa78ad

GraemeWatt added 2 commits October 29, 2021 23:23

records: pass header message if invalid Sandbox ID

b665527

* Fixes #414.

alisonrclarke added 5 commits November 2, 2021 10:33

Switch import to a smaller submission, and remove it when done

b041581

Removing it so it doesn't affect other tests.

Send JSON-LD based on Accept headers (and add tests)

21eb6d9

Modified E2E tests to include an extra submission with resources in default data.

Return 406 error when wrong Accept header given for resource

87411fe

Also added tests for resource content negotiation

Add curl download details to resource landing pages

071c0e1

Add first draft of info page on different formats

a9eb990

See #415

alisonrclarke marked this pull request as ready for review November 5, 2021 07:57

alisonrclarke requested a review from GraemeWatt November 5, 2021 07:57

alisonrclarke and others added 6 commits November 9, 2021 14:51

Acknowledge HEPData paper

476d25d

Avoid errors if submission does not exist

411fcb7

Fix errors in formats page

7bec9d7

Add JSON button to search results page

95a3866

Add JSON button for sandbox

fd26a63

Merge branch 'master' into resource-dois

562dd24

alisonrclarke mentioned this pull request Nov 10, 2021

records: support format URLs in sandbox #419

Closed

alisonrclarke added 6 commits November 10, 2021 10:54

Allow other output formats in sandbox URLs, and add 'copy to clipboard'

7e2fb8d

See #419

Fix bug when python could not guess mimetype

0ce370e

Remove mismatched quote in curl command

f40333a

Fix glitches in resource landing page

ef947cc

* Avoid calling switch_table so clipboard/json links are not mangled * Add filename to header * Fix issue with hide/show publication info

Update e2e test after changing resource landing page heading

4eecb80

Fix sorting of file resources when reserving DOIs

61d8df8

GraemeWatt mentioned this pull request Nov 17, 2021

refactor: Make contrib.utils.download robust to archive file types scikit-hep/pyhf#1697

Merged

4 tasks

matthewfeickert mentioned this pull request Nov 17, 2021

fix: Accept tar and zip headers in contrib.utils.download requests scikit-hep/pyhf#1704

Merged

4 tasks

alisonrclarke and others added 5 commits November 18, 2021 13:58

Return json-formatted error if json-ld is requested but not available

824c8e5

Merge branch 'master' into resource-dois

11156dc

Ensure initial_resource is always set so it works for resource pages

5831945

theme: improve text of new "File Formats" page

85f3635

ci: run nightly and INSPIRE update an hour earlier

e460170

* Fixes #425.

GraemeWatt approved these changes Nov 22, 2021

View reviewed changes

GraemeWatt merged commit 51032dc into master Nov 22, 2021

GraemeWatt deleted the resource-dois branch November 22, 2021 12:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mint DOIs for resources and update metadata with that from DataCite #412

Mint DOIs for resources and update metadata with that from DataCite #412

alisonrclarke commented Oct 28, 2021 •

edited by GraemeWatt

Loading

coveralls commented Oct 28, 2021 •

edited

Loading

GraemeWatt left a comment

Mint DOIs for resources and update metadata with that from DataCite #412

Mint DOIs for resources and update metadata with that from DataCite #412

Conversation

alisonrclarke commented Oct 28, 2021 • edited by GraemeWatt Loading

Release Notes

coveralls commented Oct 28, 2021 • edited Loading

GraemeWatt left a comment

Choose a reason for hiding this comment

alisonrclarke commented Oct 28, 2021 •

edited by GraemeWatt

Loading

coveralls commented Oct 28, 2021 •

edited

Loading