Profile resolution unit tests and specification updates #1219

wendellpiez · 2022-04-28T20:53:52Z

Committer Notes

Includes WIP unit tests for end-to-end profile resolution unit tests, with bindings to current draft specification.

This PR will address #1076.

Includes:

New copy of XML Profile Resolution spec, to be merged
- Includes markup for test bindings - each requirement can be assigned one or more test examples (I/O)
Updates to SpecML (schema, CSS, XSLTs)
Tooling, including autogeneration of XSpec files from SpecML
- These XSpecs can be executed, and index test examples to requirements
Examples of functional tests showing correct functionality (in the XSLT profile resolver in this branch)

Still to do (for this or a later PR):

Finalizing updates to schema and presentation XSLT
Cleanup and any outstanding merges
Finish the unit tests! now about 20% done (see [EPIC] Update XSLT profile resolver to current specification #1076)

All Submissions:

Have you selected the correct base branch per Contributing guidance?
Have you set "Allow edits and access to secrets by maintainers
"?
Have you checked to ensure there aren't other open Pull Requests for the same update/change?
Have you squashed any non-relevant commits and commit messages? [instructions]
Do all automated CI/CD checks pass?

Changes to Core Features:

Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your core changes, as applicable?
Have you included examples of how to use your new feature(s)?
Have you updated all OSCAL website and readme documentation affected by the changes you made? Changes to the OSCAL website can be made in the docs/content directory of your branch.

…irectly from SpecML source

…pdates

…ields (emended 1.0.2 syntax)

…d readme.md

… unit testing

* Updates to tests including whitespace scrub test * Adding oXygen project * Integration work on specs and unit tests / XSLT and Schematron * Profile resolution testing infrastructure, plus some tests * Some test files with expected results * Updated and improved XSpec generation and tests - now driven directly from SpecML source * Adjusted gitignore; extended spec with more bindings; illustrator XSLT * Added XSLT producing Markdown summary of requirements w/ examples; updates * Added a small utility for converting JSON "prop" fields to "props" fields (emended 1.0.2 syntax) * Updated to Profile Resolution spec (with example files now tagged) and readme.md * Updated profile resolution (spec) readme with clarification regarding unit testing

@david-waltermire-nist

* Profile resolution unit tests and specification updates (#1219) * Updates to tests including whitespace scrub test * Adding oXygen project * Integration work on specs and unit tests / XSLT and Schematron * Profile resolution testing infrastructure, plus some tests * Some test files with expected results * Updated and improved XSpec generation and tests - now driven directly from SpecML source * Adjusted gitignore; extended spec with more bindings; illustrator XSLT * Added XSLT producing Markdown summary of requirements w/ examples; updates * Added a small utility for converting JSON "prop" fields to "props" fields (emended 1.0.2 syntax) * Updated to Profile Resolution spec (with example files now tagged) and readme.md * Updated profile resolution (spec) readme with clarification regarding unit testing * Adding XSLT provisionally to produce a unit testing harness after @david-waltermire-nist model, with example and schema; also cleanup * Removed working file included by accident * More unit tests with edits to spec for tracking. * Catching up various outstanding profile resolution work * Small update to oXygen setup * Update metaschema submodule to existing commit. * Move up Metaschema submodule Prior work on pushing up the XSpec dependency and pulling in other file relocation improvements were merged and it appears I pushed the wrong commit into this branch and inadvertently led to more confusion where builds failed in current version of #1754. * XSpec test suites script & CI (#1754) * XSpec test suites script & CI * Helper script to build and tag the OSCAL env * Move up Metaschema submodule Prior work on pushing up the XSpec dependency and pulling in other file relocation improvements were merged and it appears I pushed the wrong commit into this branch and inadvertently led to more confusion where builds failed in current version of #1754. * Save XSpec test summary + pull appropriate tag * Escape TEST_SUITE in XSpec runner Co-authored-by: A.J. Stein <[email protected]> * Update test script to address shellcheck guidance * Grab branch from GHA * Fix for builds on the M1 --------- Co-authored-by: A.J. Stein <[email protected]> * Fixed logic for when a tag does not exist --------- Co-authored-by: Wendell Piez <[email protected]> Co-authored-by: A.J. Stein <[email protected]>

@david-waltermire-nist

* Profile resolution unit tests and specification updates (usnistgov#1219) * Updates to tests including whitespace scrub test * Adding oXygen project * Integration work on specs and unit tests / XSLT and Schematron * Profile resolution testing infrastructure, plus some tests * Some test files with expected results * Updated and improved XSpec generation and tests - now driven directly from SpecML source * Adjusted gitignore; extended spec with more bindings; illustrator XSLT * Added XSLT producing Markdown summary of requirements w/ examples; updates * Added a small utility for converting JSON "prop" fields to "props" fields (emended 1.0.2 syntax) * Updated to Profile Resolution spec (with example files now tagged) and readme.md * Updated profile resolution (spec) readme with clarification regarding unit testing * Adding XSLT provisionally to produce a unit testing harness after @david-waltermire-nist model, with example and schema; also cleanup * Removed working file included by accident * More unit tests with edits to spec for tracking. * Catching up various outstanding profile resolution work * Small update to oXygen setup * Update metaschema submodule to existing commit. * Move up Metaschema submodule Prior work on pushing up the XSpec dependency and pulling in other file relocation improvements were merged and it appears I pushed the wrong commit into this branch and inadvertently led to more confusion where builds failed in current version of usnistgov#1754. * XSpec test suites script & CI (usnistgov#1754) * XSpec test suites script & CI * Helper script to build and tag the OSCAL env * Move up Metaschema submodule Prior work on pushing up the XSpec dependency and pulling in other file relocation improvements were merged and it appears I pushed the wrong commit into this branch and inadvertently led to more confusion where builds failed in current version of usnistgov#1754. * Save XSpec test summary + pull appropriate tag * Escape TEST_SUITE in XSpec runner Co-authored-by: A.J. Stein <[email protected]> * Update test script to address shellcheck guidance * Grab branch from GHA * Fix for builds on the M1 --------- Co-authored-by: A.J. Stein <[email protected]> * Fixed logic for when a tag does not exist --------- Co-authored-by: Wendell Piez <[email protected]> Co-authored-by: A.J. Stein <[email protected]>

@david-waltermire-nist

* Profile resolution unit tests and specification updates (usnistgov#1219) * Updates to tests including whitespace scrub test * Adding oXygen project * Integration work on specs and unit tests / XSLT and Schematron * Profile resolution testing infrastructure, plus some tests * Some test files with expected results * Updated and improved XSpec generation and tests - now driven directly from SpecML source * Adjusted gitignore; extended spec with more bindings; illustrator XSLT * Added XSLT producing Markdown summary of requirements w/ examples; updates * Added a small utility for converting JSON "prop" fields to "props" fields (emended 1.0.2 syntax) * Updated to Profile Resolution spec (with example files now tagged) and readme.md * Updated profile resolution (spec) readme with clarification regarding unit testing * Adding XSLT provisionally to produce a unit testing harness after @david-waltermire-nist model, with example and schema; also cleanup * Removed working file included by accident * More unit tests with edits to spec for tracking. * Catching up various outstanding profile resolution work * Small update to oXygen setup * Update metaschema submodule to existing commit. * Move up Metaschema submodule Prior work on pushing up the XSpec dependency and pulling in other file relocation improvements were merged and it appears I pushed the wrong commit into this branch and inadvertently led to more confusion where builds failed in current version of usnistgov#1754. * XSpec test suites script & CI (usnistgov#1754) * XSpec test suites script & CI * Helper script to build and tag the OSCAL env * Move up Metaschema submodule Prior work on pushing up the XSpec dependency and pulling in other file relocation improvements were merged and it appears I pushed the wrong commit into this branch and inadvertently led to more confusion where builds failed in current version of usnistgov#1754. * Save XSpec test summary + pull appropriate tag * Escape TEST_SUITE in XSpec runner Co-authored-by: A.J. Stein <[email protected]> * Update test script to address shellcheck guidance * Grab branch from GHA * Fix for builds on the M1 --------- Co-authored-by: A.J. Stein <[email protected]> * Fixed logic for when a tag does not exist --------- Co-authored-by: Wendell Piez <[email protected]> Co-authored-by: A.J. Stein <[email protected]>

wendellpiez marked this pull request as draft April 28, 2022 20:54

wendellpiez marked this pull request as ready for review May 10, 2022 12:37

david-waltermire force-pushed the develop branch from 6587e35 to 51741c7 Compare May 17, 2022 12:13

wendellpiez requested a review from david-waltermire as a code owner May 17, 2022 12:13

wendellpiez added 15 commits May 31, 2022 12:42

Updates to tests including whitespace scrub test

333ed6a

Adding oXygen project

0180b50

Integration work on specs and unit tests / XSLT and Schematron

8fad06d

Profile resolution testing infrastructure, plus some tests

8be991b

Renamed current XSpec

4c0289f

Some test files with expected results

eb657af

Updates and improvements to XSpec generation and tests - now driven d…

14a1d0b

…irectly from SpecML source

Adding unit test starter set

0554705

Adjusted gitignore; extended spec with more bindings; illustrator XSLT

8595f37

slowly building out more tests

00e4ff4

Adding XSLT producing Markdown summary of requirements w/ examples; u…

a3bdc85

…pdates

Adding a small utility for converting JSON "prop" fields to "props" f…

9c49480

…ields (emended 1.0.2 syntax)

Cleanup

51114f5

Updates to Profile Resolution spec (with example files now tagged) an…

507f5cd

…d readme.md

Updated profile resolution (spec) readme with clarification regarding…

819367c

… unit testing

wendellpiez force-pushed the profile-resolution-update-v1d branch from f2aedd0 to 819367c Compare May 31, 2022 16:56

david-waltermire changed the base branch from develop to feature-profile-resolution-unittesting May 31, 2022 16:56

david-waltermire merged commit f5353c4 into usnistgov:feature-profile-resolution-unittesting May 31, 2022

david-waltermire deleted the profile-resolution-update-v1d branch May 31, 2022 16:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Profile resolution unit tests and specification updates #1219

Profile resolution unit tests and specification updates #1219

wendellpiez commented Apr 28, 2022

Profile resolution unit tests and specification updates #1219

Profile resolution unit tests and specification updates #1219

Conversation

wendellpiez commented Apr 28, 2022

Committer Notes

All Submissions:

Changes to Core Features: