feature/restore mex-model sphinx docs (#163)

### PR Context - this PR will restore the autogenerated sphinx documentation for github pages: https://robert-koch-institut.github.io/mex-model - to try locally: run `pdm doc` and open `docs/dist/index.html` in a browser ### Added - add "$$target" property to model, to help with de-referencing ### Changes - setup json-schema rendering for sphinx docs - fix cruft diffs in README, pyproject and requirements --------- Co-authored-by: RKIMetadataExchange <[email protected]>
robert-koch-institut · Nov 19, 2024 · 75fb2a0 · 75fb2a0
1 parent 4a622e2
commit 75fb2a0
Show file tree

Hide file tree

Showing 27 changed files with 435 additions and 112 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,8 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- add "$$target" property to model, to help with de-referencing
+
 ### Changes
 
+- setup json-schema rendering for sphinx docs
+- fix cruft diffs in README, pyproject and requirements
+
 ### Deprecated
 
 ### Removed

diff --git a/README.md b/README.md
@@ -8,7 +8,7 @@ JSON schema files defining the MEx metadata model.
 [![linting](https://github.com/robert-koch-institut/mex-model/actions/workflows/linting.yml/badge.svg)](https://github.com/robert-koch-institut/mex-model/actions/workflows/linting.yml)
 [![open-code](https://github.com/robert-koch-institut/mex-model/actions/workflows/open-code.yml/badge.svg)](https://gitlab.opencode.de/robert-koch-institut/mex/mex-model)
 
-## project
+## Project
 
 The Metadata Exchange (MEx) project is committed to improve the retrieval of RKI
 research data and projects. How? By focusing on metadata: instead of providing the
@@ -51,18 +51,18 @@ Our metadata model is represented as JSON schema in `mex/model`. There, we defin
 `entities`, described by their properties, 2. `fields`, small objects, that are used as
 `$ref` for certain properties, 3. an `extension`, which contains additional properties,
 that are not in scope of the JSON schema definition, 4. `i18n` files, that hold
-translations of the properties and are to be used in the context of user interfaces
-and 5. `vocabularies`, which are used in context of the `entities`. A more detailed
+translations of the properties and are to be used in the context of user interfaces and
+5. `vocabularies`, which are used in context of the `entities`. A more detailed
 description of the model's context can be found in `/docs/index.md`.
 
-## license
+## License
 
 This package is licensed under the [MIT license](/LICENSE). All other software
 components of the MEx project are open-sourced under the same license as well.
 
-## development
+## Development
 
-### installation
+### Installation
 
 - on unix, consider using pyenv https://github.com/pyenv/pyenv
   - get pyenv `curl https://pyenv.run | bash`
@@ -75,19 +75,19 @@ components of the MEx project are open-sourced under the same license as well.
   - switch version `pyenv global 3.11`
   - run `.\mex.bat install`
 
-### linting and testing
+### Linting and testing
 
 - run all linters with `pdm lint`
 
-### updating dependencies
+### Updating dependencies
 
 - update boilerplate files with `cruft update`
 - update global requirements in `requirements.txt` manually
 - update git hooks with `pre-commit autoupdate`
 - update package dependencies using `pdm update-all`
 - update github actions in `.github/workflows/*.yml` manually
 
-### creating release
+### Creating release
 
 - run `pdm release RULE` to release a new version where RULE determines which part of
   the version to update and is one of `major`, `minor`, `patch`.
diff --git a/docs/conf.py b/docs/conf.py
@@ -1,7 +1,65 @@
 # sphinx configuration
 
-extensions = ["myst_parser"]
+import importlib
+
+extensions = ["sphinx-jsonschema"]
 html_theme = "alabaster"
+html_theme_options = {
+    "extra_nav_links": {
+        "Fields": "#fields",
+        "• Identifier": "#identifier",
+        "• Link": "#link",
+        "• Text": "#text",
+        "Entities": "#entities",
+        "• AccessPlatform": "#accessplatform",
+        "• Activity": "#activity",
+        "• BibliographicResource": "#bibliographicresource",
+        "• Consent": "#consent",
+        "• ContactPoint": "#contactpoint",
+        "• Distribution": "#distribution",
+        "• Organization": "#organization",
+        "• OrganizationalUnit": "#organizationalunit",
+        "• Person": "#person",
+        "• PrimarySource": "#primarysource",
+        "• Resource": "#resource",
+        "• VariableGroup": "#variablegroup",
+        "• Variable": "#variable",
+        "Concepts": "#concepts",
+        "• ConceptScheme": "#conceptscheme",
+        "• Concept": "#concept",
+    },
+    "page_width": "100%",
+    "fixed_sidebar": "true",
+}
 project = "mex-model"
-source_suffix = {".md": "markdown"}
 templates_path = ["."]
+
+
+# Customizing json-schema conversion
+# see https://sphinx-jsonschema.readthedocs.io/en/latest/extensions.html
+
+
+def _patched_sphinx_jsonschema_simpletype(self, schema):
+    """Render the `useScheme` schema properties for every vocabulary type."""
+    rows = _original_sphinx_jsonschema_simpletype(self, schema)
+    if "useScheme" in schema:
+        scheme = schema.pop("useScheme")
+        rows.append(self._line(self._cell("useScheme"), self._cell(scheme)))
+    return rows
+
+
+def _patched_sphinx_jsonschema_kvpairs(self, schema, keys):
+    """Render `default` and `pattern` schema properties as inline code-blocks."""
+    for k in keys:
+        if k in schema:
+            value = schema[k]
+            if k in ("default", "pattern"):
+                schema[k] = f"``{value}``"
+    return _original_sphinx_jsonschema_kvpairs(self, schema, keys)
+
+
+sjs_wide_format = importlib.import_module("sphinx-jsonschema.wide_format")
+_original_sphinx_jsonschema_simpletype = sjs_wide_format.WideFormat._simpletype
+sjs_wide_format.WideFormat._simpletype = _patched_sphinx_jsonschema_simpletype
+_original_sphinx_jsonschema_kvpairs = sjs_wide_format.WideFormat._kvpairs
+sjs_wide_format.WideFormat._kvpairs = _patched_sphinx_jsonschema_kvpairs
diff --git a/docs/index.md b/docs/index.md
diff --git a/docs/index.rst b/docs/index.rst
@@ -0,0 +1,57 @@
+MEx Metadata Schema
+===================
+
+The MEx metadata schema is used to catalog Public Health data, resources and activities
+of the `Robert Koch Institute (RKI) <https://www.rki.de>`_ - the federal Public Health
+institute of Germany - with the goal of making them findable and accessible via a web
+application.
+
+The MEx metadata schema enables the description of data and resources derived from the
+research and other activities of the RKI. In most cases, the RKI's public health data
+are protected by data protection laws. Instead of publishing the data itself, MEx
+provides descriptions about the data. The MEx metadata model is designed to provide
+comprehensive descriptions for RKI's public health data. MEx users can both find and
+assess whether the data matches their interests and needs. To achieve this, we provide
+detailed information about the data in the form of variables. Variables describe how the
+actual data is specified and form the basis for a data-based evaluation of studies, for
+example.
+
+To collect information, MEx uses a mixed approach: In addition to manually compiling, we
+also automatically extract information from various primary sources managed by the RKI
+departments. To model this, we rely on the W3C recommendations
+`DCAT <https://www.w3.org/TR/vocab-dcat-2/>`_ and
+`PROV-O <http://www.w3.org/TR/prov-o/>`_.
+
+Fields
+------
+
+.. jsonschema:: ../mex/model/fields/identifier.json
+.. jsonschema:: ../mex/model/fields/link.json
+.. jsonschema:: ../mex/model/fields/text.json
+
+Entities
+--------
+
+.. jsonschema:: ../mex/model/entities/access-platform.json
+.. jsonschema:: ../mex/model/entities/activity.json
+.. jsonschema:: ../mex/model/entities/bibliographic-resource.json
+.. jsonschema:: ../mex/model/entities/consent.json
+.. jsonschema:: ../mex/model/entities/contact-point.json
+.. jsonschema:: ../mex/model/entities/distribution.json
+.. jsonschema:: ../mex/model/entities/organization.json
+.. jsonschema:: ../mex/model/entities/organizational-unit.json
+.. jsonschema:: ../mex/model/entities/person.json
+    :encoding: utf8
+.. jsonschema:: ../mex/model/entities/primary-source.json
+.. jsonschema:: ../mex/model/entities/resource.json
+.. jsonschema:: ../mex/model/entities/variable-group.json
+.. jsonschema:: ../mex/model/entities/variable.json
+
+Concepts
+--------
+
+.. jsonschema:: ../mex/model/entities/concept-scheme.json
+.. jsonschema:: ../mex/model/entities/concept.json
+
+Available concepts (aka vocabularies) soon. In the meantime, see
+https://github.com/robert-koch-institut/mex-model/tree/main/mex/model/vocabularies
diff --git a/mex/model/entities/access-platform.json b/mex/model/entities/access-platform.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/access-platform#/identifier",
     "$id": "https://mex.rki.de/schema/entities/access-platform",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A technical system or service that provides access to distributions or resources.",

diff --git a/mex/model/entities/activity.json b/mex/model/entities/activity.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/activity#/identifier",
     "$id": "https://mex.rki.de/schema/entities/activity",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "An activity carried out by RKI. This may be a research activity, such as a funded project, or a task that RKI performs under federal law.  Activities provide useful context information for resources.",

diff --git a/mex/model/entities/bibliographic-resource.json b/mex/model/entities/bibliographic-resource.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/bibliographic-resource#/identifier",
     "$id": "https://mex.rki.de/schema/entities/bibliographic-resource",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A book, article, or other documentary resource.",

diff --git a/mex/model/entities/concept-scheme.json b/mex/model/entities/concept-scheme.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/concept-scheme#/identifier",
     "$id": "https://mex.rki.de/schema/entities/concept-scheme",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "properties": {

diff --git a/mex/model/entities/concept.json b/mex/model/entities/concept.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/concept#/identifier",
     "$id": "https://mex.rki.de/schema/entities/concept",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "properties": {

diff --git a/mex/model/entities/consent.json b/mex/model/entities/consent.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/consent#/identifier",
     "$comment": "In our case, the 'specified process or activity' is the publication of contact details of a person (e.g. name and email address) in our metadata catalog. ",
     "$id": "https://mex.rki.de/schema/entities/consent",
     "$schema": "http://json-schema.org/draft/2020-12/schema",

diff --git a/mex/model/entities/contact-point.json b/mex/model/entities/contact-point.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/contact-point#/identifier",
     "$id": "https://mex.rki.de/schema/entities/contact-point",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A mail address, where a group of people has access to.",

diff --git a/mex/model/entities/distribution.json b/mex/model/entities/distribution.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/distribution#/identifier",
     "$id": "https://mex.rki.de/schema/entities/distribution",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A specific representation of a dataset. A dataset might be available in multiple serializations that may differ in various ways, including natural language, media-type or format, schematic organization, temporal and spatial resolution, level of detail or profiles (which might specify any or all of the above) ([DCAT, 2020-02-04](https://www.w3.org/TR/2020/REC-vocab-dcat-2-20200204/)).",

diff --git a/mex/model/entities/organization.json b/mex/model/entities/organization.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/organization#/identifier",
     "$id": "https://mex.rki.de/schema/entities/organization",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "Represents a collection of people organized together into a community or other social, commercial or political structure. The group has some common purpose or reason for existence which goes beyond the set of people belonging to it and can act as an Agent. Organizations are often decomposable into hierarchical structures ([The Organization Ontology, 2014-01-16](http://www.w3.org/TR/2014/REC-vocab-org-20140116/)).",

diff --git a/mex/model/entities/organizational-unit.json b/mex/model/entities/organizational-unit.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/organizational-unit#/identifier",
     "$id": "https://mex.rki.de/schema/entities/organizational-unit",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "An Organization such as a department or support unit which is part of some larger Organization and only has full recognition within the context of that Organization. In particular the unit would not be regarded as a legal entity in its own right.",

diff --git a/mex/model/entities/person.json b/mex/model/entities/person.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/person#/identifier",
     "$id": "https://mex.rki.de/schema/entities/person",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A person ([FOAF, 2004-05-01](http://xmlns.com/foaf/0.1/)). This class comprises real persons who live or are assumed to have lived ([CIDOC CRM, version 7.1.1](https://cidoc-crm.org/html/cidoc_crm_v7.1.1.html)).",

diff --git a/mex/model/entities/primary-source.json b/mex/model/entities/primary-source.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/primary-source#/identifier",
     "$id": "https://mex.rki.de/schema/entities/primary-source",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A collection of information, that is managed and curated by an RKI unit and lists activities and/or resources.",

diff --git a/mex/model/entities/resource.json b/mex/model/entities/resource.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/resource#/identifier",
     "$id": "https://mex.rki.de/schema/entities/resource",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "A defined piece of information or collection of information on Public Health, that has been generated as part of a (research) activity at the RKI or to comply with a (federal) law or regulation that applies to the RKI.",

diff --git a/mex/model/entities/variable-group.json b/mex/model/entities/variable-group.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/variable-group#/identifier",
     "$id": "https://mex.rki.de/schema/entities/variable-group",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": "The grouping of variables according to a certain aspect, e.g. how the information is modelled in the primary source.",

diff --git a/mex/model/entities/variable.json b/mex/model/entities/variable.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/entities/variable#/identifier",
     "$id": "https://mex.rki.de/schema/entities/variable",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "description": " Variables are defined for the data-based evaluation of investigations (e.g. studies). A variable is characterized by its data type (e.g. integer, string, date) and value range. The variable can be either quantitative or qualitative, i.e. the value range can take numerical or categorical values.",

diff --git a/mex/model/fields/identifier.json b/mex/model/fields/identifier.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/fields/identifier",
     "$id": "https://mex.rki.de/schema/fields/identifier",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "examples": [

diff --git a/mex/model/fields/link.json b/mex/model/fields/link.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/fields/link",
     "$id": "https://mex.rki.de/schema/fields/link",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "properties": {

diff --git a/mex/model/fields/text.json b/mex/model/fields/text.json
@@ -1,4 +1,5 @@
 {
+    "$$target": "/schema/fields/text",
     "$id": "https://mex.rki.de/schema/fields/text",
     "$schema": "http://json-schema.org/draft/2020-12/schema",
     "properties": {