Update XSLT M4 data converters to latest Metaschema design #212
Comments
4 tasks
9 tasks
wendellpiez
added a commit
to wendellpiez/metaschema
that referenced
this issue
Jul 6, 2022
…edge cases (array items of SINGLETON_OR_ARRAY groups)
8 tasks
wendellpiez
added a commit
to wendellpiez/metaschema
that referenced
this issue
Jul 6, 2022
…edge cases (array items of SINGLETON_OR_ARRAY groups)
8 tasks
|
Isn't #51 closable when this Issue is closable? |
Closed
3 tasks
david-waltermire
pushed a commit
to wendellpiez/metaschema
that referenced
this issue
Jul 14, 2022
…edge cases (array items of SINGLETON_OR_ARRAY groups)
david-waltermire
pushed a commit
that referenced
this issue
Jul 14, 2022
* Composition unit tests now valid to updated schemas; added (missing) tests.
* Adding XSpec for schema generation; nominal schema target examples
* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing
* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md
8 tasks
david-waltermire
added a commit
that referenced
this issue
Jul 14, 2022
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
Repository owner
moved this from In Progress
to Done
in NIST OSCAL Work Board
Jul 14, 2022
david-waltermire
added a commit
that referenced
this issue
Dec 7, 2022
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 9, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
david-waltermire
added a commit
that referenced
this issue
Mar 9, 2023
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
david-waltermire
added a commit
that referenced
this issue
Mar 29, 2023
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
david-waltermire
added a commit
that referenced
this issue
Mar 29, 2023
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
User Story:
As an OSCAL user, I expect to be able to continue to use the standalone (XSLT-based) converters for OSCAL content (JSON to and from XML) as I have in the past.
The converter stylesheets are produced programmatically from metaschema sources using the XSLT M4 pipeline. With respect to generating converters (at least), this pipeline needs to continue to work as the metaschema format evolves - for latest changes see https://github.com/usnistgov/metaschema/tree/feature-metaschema-relocation-plus-enhancements.
An upgraded converter generator may not resolve all known bugs in data conversion (such as #1151), but its converters will function at least as well as the converters they replace - more importantly with the latest metaschemas with enhanced syntax and features.
Goals:
The XSLT-M4 converter generation must function correctly using the updated Metaschema model represented by
metaschema.xsdin #197. "Correctly" means producing converters with the same capabilities, or better.After this refurbishment (including demonstration and testing) we will have a PR ready for the Metaschema branch (behind PR #197) , which when merged and included into the OSCAL repository, will able to serve its release-time requirements for both converter generation (publishing the OSCAL data converters) and for applying the generated converters to OSCAL data (for publication).
A secondary goal (optional, if appropriate) would be to be able to demonstrate some OSCAL-based format conversions (in both directions) under XSpec. (This will entail installing into XSpec some level of support for testing against JSON.)
Note: since the 1.1 OSCAL Schemas (XSD and JSON) will be generated using the oscal-cli-java toolset, they do not have to be produced by the Metaschema M4 pipeline. This capability is a a nice-to-have only.
Dependencies:
None at present.
Acceptance Criteria
The text was updated successfully, but these errors were encountered: