Fixes to Metapath syntax in constraints (#1012)

* Some adjustments to metapath statements in constraints to fix syntax errors in invalid metapaths.
* removing unneeded cardinality
* updating to latest metaschema to sync with schema changes
* Adjusting metaschema schematron to avoid complaining about indexes on is-unique constraints.
* Restoring missing metaschema transform.
* Fixing broken links in docs.

build/ci-cd/README.md

@@ -1,30 +1,33 @@
 # Continuous Integration and Continuous Deployment (CI/CD)
 
 This GitHub repository uses a CI/CD process that provides automation for:
+
 - Validation of contributed content and models.
 - Generation of documentation, content converters, and content in alternate formats.
 - Methods to update the [project website](https://pages.nist.gov/OSCAL).
 
 Using a CI/CD process ensures that all artifacts in [pull requests](https://github.com/usnistgov/OSCAL/blob/master/CONTRIBUTING.md) and in the [master branch](https://github.com/usnistgov/OSCAL) are valid and usable.
 
-# Provided Scripts
+## Provided Scripts
 
 The primary CircleCI supported CI/CD workflow, ```build```, is uses a series of bash scripts. The following scripts are used in this workflow for all PRs that are submitted:
 
 1. [run-all.sh](run-all.sh): Runs all of the scripts in workflow order.
-1. [validate-metaschema.sh](validate-metaschema.sh): Ensures that all [metaschema](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema) are valid according to the [metaschema XML schema](https://github.com/usnistgov/metaschema/blob/master/toolchains/oscal-m2/lib/metaschema.xsd).
+1. [validate-metaschema.sh](validate-metaschema.sh): Ensures that all [metaschema](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema) are valid according to the [metaschema XML schema](https://github.com/usnistgov/metaschema/blob/master/toolchains/xslt-M4/validate/metaschema.xsd).
 1. [generate-schema.sh](generate-schema.sh): Generates [XML schema](https://github.com/usnistgov/OSCAL/tree/master/xml/schema) and [JSON schema](https://github.com/usnistgov/OSCAL/tree/master/json/schema) files for each OSCAL model based on their [respective metaschemas](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema).
 1. [validate-content.sh](validate-content.sh): Validates OSCAL content in the repository's [src](https://github.com/usnistgov/OSCAL/tree/master/src) against the content's respective OSCAL model and format.
 1. [generate-content-converters.sh](generate-content-converters.sh): Generates [JSON to XML](https://github.com/usnistgov/OSCAL/tree/master/xml/convert) and [XML to JSON](https://github.com/usnistgov/OSCAL/tree/master/json/convert) content conversion scripts for each OSCAL model based on their [respective metaschemas](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema).
 1. [copy-and-convert-content.sh](copy-and-convert-content.sh): Copies selective content from the [src/content](https://github.com/usnistgov/OSCAL/tree/master/src/content) directory, and generates instance of this content in alternate OSCAL formats using the appropriate content converters. Note __This script requires that the process have write access to the Git repository branch that the CI/CD is operating on to work completely.__
 1. [validate-content-conversion-round-trips.sh](validate-content-conversion-round-trips.sh): Performs round trip validations of the content from XML->JSON->XML to confirm that the conversions are lossless and that resulting files comply with their schemas.
 
-Additional scripts are provided that generate content for the OSCAL project website, supported by the CircleCI ```build``` workflow on the ```master``` branch.
+    Additional scripts are provided that generate content for the OSCAL project website, supported by the CircleCI ```build``` workflow on the ```master``` branch.
+
 1. [generate-model-documentation.sh](generate-model-documentation.sh): This is used as part of the site generation pipeline to build HTML documentation for the OSCAL models.
 
-# Running the Scripts Locally
+## Running the Scripts Locally
 
 The bash scripts used in the CI/CD workflow can be run locally in the correct environment. In addition to a bash shell, to run these scripts the following must be installed:
+
 - [xmlint](http://xmlsoft.org/xmllint.html)
 - [Java](https://www.java.com/en/) 8.0 or greater
 - [Apache Maven](https://maven.apache.org/) 3.6.1 or greater
@@ -36,123 +39,119 @@ Additionally, the following packages need to be installed.
 
 NodeJS Packages:
 
-```
+```bash
 npm install -g ajv-cli
 ```
 
 Python Packages:
 
-```
+```bash
 pip install lxml
 ```
 
 The Saxon jar file needs to be retrieved. This can be two ways:
 
-1) You can set the environment variable ```SAXON_VERSION``` to the Saxon version to use. This must correspond to a valid Saxon-HE version in [Maven Central](https://search.maven.org/artifact/net.sf.saxon/Saxon-HE). Maven will be used to automatically download the Saxon-HE jar dependency.
+1. You can set the environment variable ```SAXON_VERSION``` to the Saxon version to use. This must correspond to a valid Saxon-HE version in [Maven Central](https://search.maven.org/artifact/net.sf.saxon/Saxon-HE). Maven will be used to automatically download the Saxon-HE jar dependency.
 
-For example:
+    For example:
 
-```
-export SAXON_VERSION=9.9.1-3
-```
+    ```bash
+    export SAXON_VERSION=9.9.1-3
+    ```
 
-2) You can manually download any variant of Saxon (e.g., HE, ) from [Saxonica](https://www.saxonica.com/download/java.xml) and set the ```SAXON_HOME``` environment variable to the location of the Saxon JAR.
+2. You can manually download any variant of Saxon (e.g., HE, ) from [Saxonica](https://www.saxonica.com/download/java.xml) and set the ```SAXON_HOME``` environment variable to the location of the Saxon JAR.
 
-```
+```bash
 export SAXON_HOME=~/saxon-jar-directory
 ```
 
 Finally, the Schematron Skeleton needs to be retrieved. The following commands can be used to download it.
 
-```
+```bash
 export SCHEMATRON_HOME=~/schematron-skeleton
 git clone --depth 1 --no-checkout https://github.com/Schematron/schematron.git "$SCHEMATRON_HOME"
 cd "$SCHEMATRON_HOME"
 git checkout master -- trunk/schematron/code
 ```
 
-## Further installation notes
+### Further installation notes
 
 Setting up to run in WSL (Windows subsystem for Linux) avoids many Windows-related configuration issues, but requires the entire environment be available from within the subsystem. The easiest way to accomplish this is to install the components into the subsystem directly.
 
 To check whether xmllint is installed:
 
-```
+```bash
 xmllint -version
 ```
 
 To install it
 
-```
+```bash
 sudo apt install libxml2-utils
 ```
 
 To install Java:
 
-```
+```bash
 sudo apt install openjdk-11-jre-headless
 ```
 
 To install NodeJS:
 
-```
+```bash
 sudo apt install nodejs
 ```
 
 And npm (Node package manager, separate from NodeJS):
 
-```
+```bash
 sudo apt install npm
 ```
 
-
 To install AJV (nodeJS JSON validator)
 
-```
+```bash
 npm install -g ajv-cli
 ```
 
-
 or if you want to install globally (for all users):
 
-```
+```bash
 sudo npm install -g ajv-cli
 ```
 
 Ubuntu comes with Python. To check the version:
 
-```
+```bash
 python3 -V
 ```
 
 Scripts were tested with version 3.7.3 but 3.6.8 (the default in Ubuntu under WSL) should work.
 
 Along with Python you also need a library, for which you need pip (a Python installer):
 
-```
+```bash
 sudo apt install python-pip
 pip install lxml
 
 ```
 
-
 To install Maven:
 
-```
+```bash
 sudo apt install maven
 ```
 
 Then run Maven to acquire Saxon:
 
-
-```
+```bash
 export SAXON_VERSION=9.9.1-3
 mvn org.apache.maven.plugins:maven-dependency-plugin:2.10:get -DartifactId=Saxon-HE -DgroupId=net.sf.saxon -Dversion=$SAXON_VERSION
 ```
 
 You will also need a copy of the ISO Schematron skeleton.
 
-```
+```bash
 mkdir $HOME/oscal-oss
 export SCHEMATRON_HOME=$HOME/oscal-oss/git-schematron
 git clone --depth 1 --no-checkout https://github.com/Schematron/schematron.git "$SCHEMATRON_HOME"
@@ -164,12 +163,12 @@ Finally, export instructions in the preceding must also be copied into .bashrc s
 
 prettyson
 
-```
+```bash
 sudo npm install -g prettyjson
 ```
 
 jq
 
-```
+```bash
 sudo apt-get install jq
 ```

src/metaschema/oscal_assessment-common_metaschema.xml

@@ -66,10 +66,10 @@
             <enum value="objective"></enum>
             <enum value="assessment"></enum>
          </allowed-values>
-         <has-cardinality target="part[@name='objective']" min-occurs="0" max-occurs="1" />
+         <has-cardinality target="part[@name='objective']" max-occurs="1" />
          <has-cardinality target="part[@name='assessment']/prop[@name='method']" min-occurs="1" max-occurs="1" />
          <has-cardinality target="part[@name='assessment']/part[@name='objects']" min-occurs="1" max-occurs="1" />
-         <has-cardinality target="part[@name='objective']/prop[@name='method-id']" min-occurs="1" max-occurs="unbounded" />
+         <has-cardinality target="part[@name='objective']/prop[@name='method-id']" min-occurs="1" />
       </constraint>
    </define-assembly>
    <define-assembly name="assessment-method">
@@ -157,7 +157,7 @@
                <field ref="remarks" in-xml="WITH_WRAPPER" min-occurs="0" max-occurs="1"/>
             </model>
             <constraint>
-               <is-unique name="unique-step-responsible-role" target="responsible-role">
+               <is-unique id="unique-step-responsible-role" target="responsible-role">
                   <key-field target="@role-id"/>
                   <remarks>
                      <p>Since multiple <code>party-uuid</code> entries can be provided, each role-id must be referenced only once.</p>
@@ -190,7 +190,7 @@
             <enum value="EXAMINE">The process of reviewing, inspecting, observing, studying, or analyzing one or more assessment objects (i.e., specifications, mechanisms, or activities).</enum>
             <enum value="TEST">The process of exercising one or more assessment objects (i.e., activities or mechanisms) under specified conditions to compare actual with expected behavior.</enum>
          </allowed-values>
-         <is-unique name="unique-activity-responsible-role" target="responsible-role">
+         <is-unique id="unique-activity-responsible-role" target="responsible-role">
             <key-field target="@role-id"/>
             <remarks>
                <p>Since <code>responsible-role</code> associates multiple <code>party-uuid</code> entries with a single <code>role-id</code>, each role-id must be referenced only once.</p>
@@ -331,7 +331,7 @@
 -->               <field ref="remarks" in-xml="WITH_WRAPPER" min-occurs="0" max-occurs="1"/>
             </model>
             <constraint>
-               <is-unique name="unique-associated-activity-responsible-role" target="responsible-role">
+               <is-unique id="unique-associated-activity-responsible-role" target="responsible-role">
                   <key-field target="@role-id"/>
                   <remarks>
                      <p>Since <code>responsible-role</code> associates multiple <code>party-uuid</code> entries with a single <code>role-id</code>, each role-id must be referenced only once.</p>
@@ -687,7 +687,7 @@
                      <field ref="remarks" in-xml="WITH_WRAPPER"/>
                   </model>
                   <constraint>
-                     <is-unique name="unique-ssp-uses-component-responsible-party" target="responsible-party">
+                     <is-unique id="unique-ssp-uses-component-responsible-party" target="responsible-party">
                         <key-field target="@role-id"/>
                         <remarks>
                            <p>Since <code>responsible-party</code> associates multiple <code>party-uuid</code> entries with a single <code>role-id</code>, each role-id must be referenced only once.</p>
@@ -700,7 +700,7 @@
          </define-assembly>
       </model>
       <constraint>
-         <is-unique name="unique-ssp-assessment-assets-component" target="component">
+         <is-unique id="unique-ssp-assessment-assets-component" target="component">
             <key-field target="@uuid"/>
             <remarks>
                <p>Since multiple assessment <code>component</code> entries can be provided, each component must have a unique <code>uuid</code>.</p>
@@ -887,18 +887,18 @@
          </define-field>
          <field ref="remarks" in-xml="WITH_WRAPPER" min-occurs="0" max-occurs="1"/>
       </model>
-      <constraint>
-         <!-- TODO: review these and figure out where these go -->
-<!--         <allowed-values target="origin/@type" allow-other="no">
-            <!-\- CHANGED: "tool" to "******" -\->
+<!--      <constraint>
+         <!-\- TODO: review these and figure out where these go -\->
+         <allowed-values target="origin/@type" allow-other="no">
+            <!-\\- CHANGED: "tool" to "******" -\\->
             <enum value="tool">An assessment tool, defined in the assets section of the assessment plan or results.</enum>
             <enum value="test-method">A test method defined in the assessment-activities section of the assessment plan or results.</enum>
             <enum value="task">A task defined in the schedule of the assessment plan or results.</enum>
             <enum value="included-activity">An assessment activity defined in the assessment-activities section of the assessment plan or results.</enum>
             <enum value="other">The UUID points elsewhere in the this file or an imported file.</enum>
          </allowed-values>
--->      </constraint>
-   </define-assembly>
+      </constraint>
+-->   </define-assembly>
 
    <define-assembly name="origin">
       <formal-name>Origin</formal-name>
@@ -994,7 +994,7 @@
          <field ref="remarks" in-xml="WITH_WRAPPER" min-occurs="0" max-occurs="1"/>
       </model>
       <constraint>
-         <is-unique name="unique-ssp-related-task-responsible-party" target="responsible-party">
+         <is-unique id="unique-ssp-related-task-responsible-party" target="responsible-party">
             <key-field target="@role-id"/>
             <remarks>
                <p>Since <code>responsible-party</code> associates multiple <code>party-uuid</code> entries with a single <code>role-id</code>, each role-id must be referenced only once.</p>
@@ -1194,7 +1194,7 @@
                      <allowed-values target="prop/@name" allow-other="yes">
                         <enum value="type">The type of remediation tracking entry. Can be multi-valued.</enum>
                      </allowed-values>
-                     <allowed-values target="prop/[@name='type']/@value" allow-other="yes">
+                     <allowed-values target="prop[@name='type']/@value" allow-other="yes">
                         <enum value="vendor-check-in">Contacted vendor to determine the status of a pending fix to a known vulnerability.</enum>
                         <enum value="status-update">Information related to the current state of response to this risk.</enum>
                         <enum value="milestone-complete">A significant step in the response plan has been achieved.</enum>
@@ -1587,12 +1587,6 @@
                </assembly>
                <field ref="remarks" in-xml="WITH_WRAPPER" min-occurs="0" max-occurs="1"/>
             </model>
-            <constraint>
-               <allowed-values target="origin/@type" allow-other="no">
-                  <enum value="party">The UUID of the person or organization who made the recommendation</enum>
-                  <enum value="tool">The UUID of the tool that made the recommendation</enum>
-               </allowed-values>
-            </constraint>
          </define-assembly>
          <assembly ref="task" min-occurs="0" max-occurs="unbounded">
             <group-as name="tasks" in-json="ARRAY"/>

src/metaschema/oscal_assessment-plan_metaschema.xml

@@ -67,13 +67,13 @@
                     <field ref="remarks" in-xml="WITH_WRAPPER" min-occurs="0" max-occurs="1"/>
                 </model>
                 <constraint>
-                    <is-unique name="unique-ap-local-definitions-component" target="component">
+                    <is-unique id="unique-ap-local-definitions-component" target="component">
                         <key-field target="@uuid"/>
                         <remarks>
                             <p>Since multiple <code>component</code> entries can be provided, each component must have a unique <code>uuid</code>.</p>
                         </remarks>
                     </is-unique>
-                    <is-unique name="unique-ap-local-definitions-user" target="user">
+                    <is-unique id="unique-ap-local-definitions-user" target="user">
                         <key-field target="@uuid"/>
                         <remarks>
                             <p>A given <code>uuid</code> must be assigned only once to a user.</p>

src/metaschema/oscal_assessment-results_metaschema.xml

@@ -141,13 +141,13 @@
           </assembly>
         </model>
         <constraint>
-          <is-unique name="unique-ar-local-definitions-component" target="component">
+          <is-unique id="unique-ar-local-definitions-component" target="component">
             <key-field target="@uuid"/>
             <remarks>
               <p>Since multiple <code>component</code> entries can be provided, each component must have a unique <code>uuid</code>.</p>
             </remarks>
           </is-unique>
-          <is-unique name="unique-ar-local-definitions-user" target="user">
+          <is-unique id="unique-ar-local-definitions-user" target="user">
             <key-field target="@uuid"/>
             <remarks>
               <p>A given <code>uuid</code> must be assigned only once to a user.</p>
@@ -179,7 +179,7 @@
           </assembly>
         </model>
         <constraint>
-          <is-unique name="unique-ar-attestation-responsible-party" target="responsible-party">
+          <is-unique id="unique-ar-attestation-responsible-party" target="responsible-party">
             <key-field target="@role-id"/>
             <remarks>
               <p>Since <code>responsible-party</code> associates multiple <code>party-uuid</code> entries with a single <code>role-id</code>, each role-id must be referenced only once.</p>

src/metaschema/oscal_catalog_metaschema.xml

@@ -45,9 +45,6 @@
                         </remarks>
                   </assembly>
             </model>
-            <constraint>
-
-            </constraint>
             <remarks>
                   <p>Catalogs may use one or more <code>group</code> objects to subdivide the control contents of a catalog.</p>
                   <p>An OSCAL catalog model provides a structured representation of control information.</p>

src/metaschema/oscal_complete_metaschema.xml

@@ -3,7 +3,7 @@
 <!-- OSCAL GRAND UNIFIED MEGALOMETASCHEMA -->
 <!-- validate with XSD and Schematron (linked) -->
 <!DOCTYPE METASCHEMA [
-   <!ENTITY allowed-values-control-group-property-name SYSTEM "shared-constraints/allowed-values-control-group-property-name.ent">
+   <!ENTITY allowed-values-control-group-property-name SYSTEM "./shared-constraints/allowed-values-control-group-property-name.ent">
 ]>
 <METASCHEMA xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0">
       <schema-name>OSCAL Unified Model of Models</schema-name>