[SCHEMA] Add metadata term files

pdf_build_src/process_markdowns.py

@@ -445,6 +445,7 @@ def process_macros(duplicated_src_dir_path):
 
             # Replace code snippets in the text with their outputs
             matches = re.findall("({{.*?}})", contents)
+            matches = re.findall(re.compile("({{.*?}})", re.DOTALL), contents)
             for m in matches:
                 # Remove macro delimiters to get *just* the function call
                 function_string = m.strip("{} ")

src/02-common-principles.md

@@ -517,14 +517,19 @@ Note that if a field name included in the data dictionary matches a column name
 then that field MUST contain a description of the corresponding column,
 using an object containing the following fields:
 
-| **Key name** | **Requirement level** | **Data type**                           | **Description**                                                                                                 |
-| ------------ | --------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
-| LongName     | OPTIONAL              | [string][]                              | Long (unabbreviated) name of the column.                                                                        |
-| Description  | RECOMMENDED           | [string][]                              | Description of the column.                                                                                      |
-| Levels       | RECOMMENDED           | [object][] of [strings][]               | For categorical variables: An object of possible values (keys) and their descriptions (values).                 |
-| Units        | RECOMMENDED           | [string][]                              | Measurement units. SI units in CMIXF formatting are RECOMMENDED (see [Units](./02-common-principles.md#units)). |
-| TermURL      | RECOMMENDED           | [string][]                              | URL pointing to a formal definition of this type of data in an ontology available on the web.                   |
-| HED          | OPTIONAL              | [object][] of [strings][] or [string][] | Hierarchical Event Descriptor (HED) information, see: [Appendix III](./99-appendices/03-hed.md) for details.    |
+{{ MACROS___make_metadata_table(
+   {
+        "LongName": "OPTIONAL",
+        "Description": (
+            "RECOMMENDED",
+            "The description of the column.",
+        ),
+        "Levels": "RECOMMENDED",
+        "Units": "RECOMMENDED",
+        "TermURL": "RECOMMENDED",
+        "HED": "OPTIONAL",
+   }
+) }}
 
 Please note that while both `Units` and `Levels` are RECOMMENDED, typically only one
 of these two fields would be specified for describing a single TSV file column.
@@ -767,10 +772,4 @@ to suppress warnings or provide interpretations of your file names.
 
 [derived-dataset-description]: 03-modality-agnostic-files.md#derived-dataset-and-pipeline-description
 
-[string]: https://www.w3schools.com/js/js_json_syntax.asp
-
-[strings]: https://www.w3schools.com/js/js_json_syntax.asp
-
-[object]: https://www.json.org/json-en.html
-
 [deprecated]: ./02-common-principles.md#definitions

src/03-modality-agnostic-files.md

@@ -14,20 +14,22 @@ Templates:
 The file `dataset_description.json` is a JSON file describing the dataset.
 Every dataset MUST include this file with the following fields:
 
-| **Key name**       | **Requirement level** | **Data type**            | **Description**                                                                                                                                                                                                                                       |
-|--------------------|-----------------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Name               | REQUIRED              | [string][]               | Name of the dataset.                                                                                                                                                                                                                                  |
-| BIDSVersion        | REQUIRED              | [string][]               | The version of the BIDS standard that was used.                                                                                                                                                                                                       |
-| HEDVersion         | RECOMMENDED           | [string][]               | If HED tags are used: The version of the HED schema used to validate HED tags for study.                                                                                                                                                              |
-| DatasetType        | RECOMMENDED           | [string][]               | The interpretation of the dataset. MUST be one of `"raw"` or `"derivative"`. For backwards compatibility, the default value is `"raw"`.                                                                                                               |
-| License            | RECOMMENDED           | [string][]               | The license for the dataset. The use of license name abbreviations is RECOMMENDED for specifying a license (see [Appendix II](./99-appendices/02-licenses.md)). The corresponding full license text MAY be specified in an additional `LICENSE` file. |
-| Authors            | OPTIONAL              | [array][] of [strings][] | List of individuals who contributed to the creation/curation of the dataset.                                                                                                                                                                          |
-| Acknowledgements   | OPTIONAL              | [string][]               | Text acknowledging contributions of individuals or institutions beyond those listed in Authors or Funding.                                                                                                                                            |
-| HowToAcknowledge   | OPTIONAL              | [string][]               | Text containing instructions on how researchers using this dataset should acknowledge the original authors. This field can also be used to define a publication that should be cited in publications that use the dataset.                            |
-| Funding            | OPTIONAL              | [array][] of [strings][] | List of sources of funding (grant numbers).                                                                                                                                                                                                           |
-| EthicsApprovals    | OPTIONAL              | [array][] of [strings][] | List of ethics committee approvals of the research protocols and/or protocol identifiers.                                                                                                                                                             |
-| ReferencesAndLinks | OPTIONAL              | [array][] of [strings][] | List of references to publications that contain information on the dataset. A reference may be textual or a [URI][uri].                                                                                                                               |
-| DatasetDOI         | OPTIONAL              | [string][]               | The Digital Object Identifier of the dataset (not the corresponding paper). DOIs SHOULD be expressed as a valid [URI][uri]; bare DOIs such as `10.0.2.3/dfjj.10` are [DEPRECATED][deprecated].                                                        |
+{{ MACROS___make_metadata_table(
+   {
+      "Name": "REQUIRED",
+      "BIDSVersion": "REQUIRED",
+      "HEDVersion": "RECOMMENDED",
+      "DatasetType": "RECOMMENDED",
+      "License": "RECOMMENDED",
+      "Authors": "OPTIONAL",
+      "Acknowledgements": "OPTIONAL",
+      "HowToAcknowledge": "OPTIONAL",
+      "Funding": "OPTIONAL",
+      "EthicsApprovals": "OPTIONAL",
+      "ReferencesAndLinks": "OPTIONAL",
+      "DatasetDOI": "OPTIONAL",
+   }
+) }}
 
 Example:
 
@@ -69,10 +71,12 @@ In addition to the keys for raw BIDS datasets,
 derived BIDS datasets include the following REQUIRED and RECOMMENDED
 `dataset_description.json` keys:
 
-| **Key name**   | **Requirement level** | **Data type**            | **Description**                                                                                                                                                                      |
-|----------------|-----------------------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| GeneratedBy    | REQUIRED              | [array][] of [objects][] | Used to specify provenance of the derived dataset. See table below for contents of each object.                                                                                      |
-| SourceDatasets | RECOMMENDED           | [array][] of [objects][] | Used to specify the locations and relevant attributes of all source datasets. Valid keys in each object include `URL`, `DOI` (see [URI][uri]), and `Version` with [string][] values. |
+{{ MACROS___make_metadata_table(
+   {
+      "GeneratedBy": "REQUIRED",
+      "SourceDatasets": "RECOMMENDED",
+   }
+) }}
 
 Each object in the `GeneratedBy` list includes the following REQUIRED, RECOMMENDED
 and OPTIONAL keys:
@@ -394,16 +398,8 @@ code organization of these scripts at the moment.
 
 <!-- Link Definitions -->
 
-[objects]: https://www.json.org/json-en.html
-
 [object]: https://www.json.org/json-en.html
 
 [string]: https://www.w3schools.com/js/js_json_syntax.asp
 
-[strings]: https://www.w3schools.com/js/js_json_syntax.asp
-
-[array]: https://www.w3schools.com/js/js_json_arrays.asp
-
 [uri]: ./02-common-principles.md#uniform-resource-indicator
-
-[deprecated]: ./02-common-principles.md#definitions