[SCHEMA] Add metadata term files

pdf_build_src/process_markdowns.py

@@ -445,6 +445,7 @@ def process_macros(duplicated_src_dir_path):
 
             # Replace code snippets in the text with their outputs
             matches = re.findall("({{.*?}})", contents)
+            matches = re.findall(re.compile("({{.*?}})", re.DOTALL), contents)
             for m in matches:
                 # Remove macro delimiters to get *just* the function call
                 function_string = m.strip("{} ")

src/02-common-principles.md

@@ -517,6 +517,20 @@ Note that if a field name included in the data dictionary matches a column name
 then that field MUST contain a description of the corresponding column,
 using an object containing the following fields:
 
+{{ MACROS___make_metadata_table(
+   {
+        "LongName": "OPTIONAL",
+        "Description": (
+            "RECOMMENDED",
+            "The description of the column.",
+        ),
+        "Levels": "RECOMMENDED",
+        "Units": "RECOMMENDED",
+        "TermURL": "RECOMMENDED",
+        "HED": "OPTIONAL",
+   }
+) }}
+
 | **Key name** | **Requirement level** | **Data type**                           | **Description**                                                                                                 |
 | ------------ | --------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
 | LongName     | OPTIONAL              | [string][]                              | Long (unabbreviated) name of the column.                                                                        |

src/03-modality-agnostic-files.md

@@ -14,6 +14,23 @@ Templates:
 The file `dataset_description.json` is a JSON file describing the dataset.
 Every dataset MUST include this file with the following fields:
 
+{{ MACROS___make_metadata_table(
+   {
+      "Name": "REQUIRED",
+      "BIDSVersion": "REQUIRED",
+      "HEDVersion": "RECOMMENDED",
+      "DatasetType": "RECOMMENDED",
+      "License": "RECOMMENDED",
+      "Authors": "OPTIONAL",
+      "Acknowledgements": "OPTIONAL",
+      "HowToAcknowledge": "OPTIONAL",
+      "Funding": "OPTIONAL",
+      "EthicsApprovals": "OPTIONAL",
+      "ReferencesAndLinks": "OPTIONAL",
+      "DatasetDOI": "OPTIONAL",
+   }
+) }}
+
 | **Key name**       | **Requirement level** | **Data type**            | **Description**                                                                                                                                                                                                                                       |
 |--------------------|-----------------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Name               | REQUIRED              | [string][]               | Name of the dataset.                                                                                                                                                                                                                                  |
@@ -69,6 +86,13 @@ In addition to the keys for raw BIDS datasets,
 derived BIDS datasets include the following REQUIRED and RECOMMENDED
 `dataset_description.json` keys:
 
+{{ MACROS___make_metadata_table(
+   {
+      "GeneratedBy": "REQUIRED",
+      "SourceDatasets": "RECOMMENDED",
+   }
+) }}
+
 | **Key name**   | **Requirement level** | **Data type**            | **Description**                                                                                                                                                                      |
 |----------------|-----------------------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | GeneratedBy    | REQUIRED              | [array][] of [objects][] | Used to specify provenance of the derived dataset. See table below for contents of each object.                                                                                      |

src/04-modality-specific-files/03-electroencephalography.md

@@ -86,6 +86,12 @@ be specified.
 
 Generic fields MUST be present:
 
+{{ MACROS___make_metadata_table(
+   {
+      "TaskName": "REQUIRED",
+   }
+) }}
+
 | **Key name** | **Requirement level** | **Data type** | **Description**                                                                                                                                                                                                                                                                                                                                                                    |
 | ------------ | --------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | TaskName     | REQUIRED              | [string][]    | Name of the task. No two tasks should have the same name. The task label included in the file name is derived from this TaskName field by removing all non-alphanumeric (`[a-zA-Z0-9]`) characters. For example `TaskName` `"faces n-back"` will correspond to task label `facesnback`. A RECOMMENDED convention is to name resting state task using labels beginning with `rest`. |
@@ -94,6 +100,21 @@ SHOULD be present: For consistency between studies and institutions, we
 encourage users to extract the values of these fields from the actual raw data.
 Whenever possible, please avoid using ad hoc wording.
 
+{{ MACROS___make_metadata_table(
+   {
+      "InstitutionName": "RECOMMENDED",
+      "InstitutionAddress": "RECOMMENDED",
+      "Manufacturer": "RECOMMENDED",
+      "ManufacturersModelName": "RECOMMENDED",
+      "SoftwareVersions": "RECOMMENDED",
+      "TaskDescription": "RECOMMENDED",
+      "Instructions": "RECOMMENDED",
+      "CogAtlasID": "RECOMMENDED",
+      "CogPOID": "RECOMMENDED",
+      "DeviceSerialNumber": "RECOMMENDED",
+   }
+) }}
+
 | **Key name**           | **Requirement level** | **Data type** | **Description**                                                                                                                                                                                                                   |
 | ---------------------- | --------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | InstitutionName        | RECOMMENDED           | [string][]    | The name of the institution in charge of the equipment that produced the measurements.                                                                                                                                            |
@@ -109,6 +130,15 @@ Whenever possible, please avoid using ad hoc wording.
 
 Specific EEG fields MUST be present:
 
+{{ MACROS___make_metadata_table(
+   {
+      "EEGReference": "REQUIRED",
+      "SamplingFrequency": ("REQUIRED", "The sampling frequency of data channels that deviate from the main sampling frequency SHOULD be specified in the `channels.tsv` file."),
+      "PowerLineFrequency": "REQUIRED",
+      "SoftwareFilters": "REQUIRED",
+   }
+) }}
+
 | **Key name**       | **Requirement level** | **Data type**                        | **Description**                                                                                                                                                                                                                                                                                                                                                       |
 | ------------------ | --------------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | EEGReference       | REQUIRED              | [string][]                           | General description of the reference scheme used and (when applicable) of location of the reference electrode in the raw recordings (for example, `"left mastoid"`, `"Cz"`, `"CMS"`). If different channels have a different reference, this field should have a general description and the channel specific reference should be defined in the `channels.tsv` file. |
@@ -118,6 +148,27 @@ Specific EEG fields MUST be present:
 
 SHOULD be present:
 
+{{ MACROS___make_metadata_table(
+   {
+      "CapManufacturer": "RECOMMENDED",
+      "CapManufacturersModelName": "RECOMMENDED",
+      "EEGChannelCount": "RECOMMENDED",
+      "ECGChannelCount": "RECOMMENDED",
+      "EMGChannelCount": "RECOMMENDED",
+      "EOGChannelCount": "RECOMMENDED",
+      "MiscChannelCount": "RECOMMENDED",
+      "TriggerChannelCount": "RECOMMENDED",
+      "RecordingDuration": "RECOMMENDED",
+      "RecordingType": "RECOMMENDED",
+      "EpochLength": "RECOMMENDED",
+      "EEGGround": "RECOMMENDED",
+      "HeadCircumference": "RECOMMENDED",
+      "EEGPlacementScheme": "RECOMMENDED",
+      "HardwareFilters": "RECOMMENDED",
+      "SubjectArtefactDescription": "RECOMMENDED",
+   }
+) }}
+
 | **Key name**               | **Requirement level** | **Data type**                        | **Description**                                                                                                                                                                                                                                                                                                                        |
 | -------------------------- | --------------------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | CapManufacturer            | RECOMMENDED           | [string][]                           | Name of the cap manufacturer (for example, `"EasyCap"`).                                                                                                                                                                                                                                                                               |
@@ -352,12 +403,30 @@ head.
 
 General fields:
 
+{{ MACROS___make_metadata_table(
+   {
+      "IntendedFor": (
+         "OPTIONAL",
+         "This identifies the MRI or CT scan associated with the electrodes, "
+         "landmarks, and fiducials.",
+      )
+   }
+) }}
+
 | **Key name** | **Requirement level** | **Data type** | **Description**                                                                  |
 | ------------ | --------------------- | ------------- | -------------------------------------------------------------------------------- |
 | IntendedFor  | OPTIONAL              | [string][]    | Relative path to associate the electrodes, landmarks and fiducials to an MRI/CT. |
 
 Fields relating to the EEG electrode positions:
 
+{{ MACROS___make_metadata_table(
+   {
+      "EEGCoordinateSystem": "REQUIRED",
+      "EEGCoordinateUnits": "REQUIRED",
+      "EEGCoordinateSystemDescription": 'RECOMMENDED, but REQUIRED if `EEGCoordinateSystem` is `"Other"`',
+   }
+) }}
+
 | **Key name**                   | **Requirement level**                                           | **Data type** | **Description**                                                                                                                                                                                                                                                            |
 | ------------------------------ | --------------------------------------------------------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | EEGCoordinateSystem            | REQUIRED                                                        | [string][]    | Defines the coordinate system for the EEG sensors. See [Appendix VIII](../99-appendices/08-coordinate-systems.md) for a list of restricted keywords for coordinate systems. If `"Other"`, provide definition of the coordinate system in `EEGCoordinateSystemDescription`. |
@@ -366,6 +435,16 @@ Fields relating to the EEG electrode positions:
 
 Fields relating to the position of fiducials measured during an EEG session/run:
 
+{{ MACROS___make_metadata_table(
+   {
+      "FiducialsDescription": "OPTIONAL",
+      "FiducialsCoordinates": "RECOMMENDED",
+      "FiducialsCoordinateSystem": "RECOMMENDED",
+      "FiducialsCoordinateUnits": "RECOMMENDED",
+      "FiducialsCoordinateSystemDescription": 'RECOMMENDED, but REQUIRED if `FiducialsCoordinateSystem` is `"Other"`',
+   }
+) }}
+
 | **Key name**                         | **Requirement level**                                                 | **Data type**            | **Description**                                                                                                                                                                                                                                                                                                                                                 |
 | ------------------------------------ | --------------------------------------------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | FiducialsDescription                 | OPTIONAL                                                              | [string][]               | Free-form text description of how the fiducials such as vitamin-E capsules were placed relative to anatomical landmarks, and how the position of the fiducials were measured (for example, both with Polhemus and with T1w MRI).                                                                                                                                |
@@ -376,6 +455,15 @@ Fields relating to the position of fiducials measured during an EEG session/run:
 
 Fields relating to the position of anatomical landmark measured during an EEG session/run:
 
+{{ MACROS___make_metadata_table(
+   {
+      "AnatomicalLandmarkCoordinates": "RECOMMENDED",
+      "AnatomicalLandmarkCoordinateSystem": ("RECOMMENDED", "Preferably the same as the `EEGCoordinateSystem`."),
+      "AnatomicalLandmarkCoordinateUnits": "RECOMMENDED",
+      "AnatomicalLandmarkCoordinateSystemDescription": 'RECOMMENDED, but REQUIRED if `AnatomicalLandmarkCoordinateSystem` is `"Other"`',
+   }
+) }}
+
 | **Key name**                                  | **Requirement level**                                                          | **Data type**            | **Description**                                                                                                                                                                                                                                                                                                                                                          |
 | --------------------------------------------- | ------------------------------------------------------------------------------ | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
 | AnatomicalLandmarkCoordinates                 | RECOMMENDED                                                                    | [object][] of [arrays][] | Key:value pairs of the labels and 3-D digitized position of anatomical landmarks, interpreted following the `AnatomicalLandmarkCoordinateSystem` (for example, `{"NAS": [12.7,21.3,13.9], "LPA": [5.2,11.3,9.6], "RPA": [20.2,11.3,9.1]}`). Each array MUST contain three numeric values corresponding to x, y, and z axis of the coordinate system in that exact order. |