Update tests to support corrected JSON Schema Generation PR

[aj-stein-nist]

Committer Notes

This PR amends existing tests and creates new ones to expand JSON Schema generation coverage given root cause analysis and review provided by community member @kylelaker in #360.

Specifically, json-schema-gen/json-value_flag_constrained-open_metaschema.xml and the related test in json-schema-gen.xspec will fail if run against current develop code (before merging #360) and will pass once run against the #360 branch (against the original commit as specific in most current PR feedback; I have not included that code to make both PRs mergeable).

As-is, the test will fail with current develop as the integration test will actually return this output, which is incorrect.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://csrc.nist.gov/ns/metaschema-tests/0.1/json-value_flag_constrained-open-schema.json",
  "$comment": "JSON value testing mini metaschema: JSON Schema",
  "type": "object",
  "definitions": {
    "json-value_flag_constrained-open-json-value_flag_constrained-open:root": {
      "title": "Root",
      "description": "Root root root.",
      "$id": "#assembly_json-value_flag_constrained-open_root",
      "type": "object",
      "properties": {
        "constrained-open": {
          "$ref": "#field_json-value_flag_constrained-open_constrained-open"
        }
      },
      "additionalProperties": false
    },
    "json-value_flag_constrained-open-json-value_flag_constrained-open:constrained-open": {
      "title": "Constrained open",
      "description": "Because allow-other is 'yes' no enumeration can be given in the JSON Schema.",
      "$id": "#field_json-value_flag_constrained-open_constrained-open",
      "type": "object",
      "properties": {
        "some": {
          "title": "Some Flag",
          "description": "Some flag some flag some flag.",
          "allOf": [
            {
              "$ref": "#/definitions/StringDatatype"
            },
            {
              "enum": [
                "one",
                "two",
                "three",
                "four"
              ]
            }
          ]
        },
        "token-value": {
          "$ref": "#/definitions/TokenDatatype"
        }
      },
      "required": [
        "token-value"
      ],
      "additionalProperties": false
    },
    "StringDatatype": {
      "description": "A non-empty string with leading and trailing whitespace disallowed. Whitespace is: U+9, U+10, U+32 or [ \n\t]+",
      "type": "string",
      "pattern": "^\\S(.*\\S)?$"
    },
    "TokenDatatype": {
      "description": "A non-colonized name as defined by XML Schema Part 2: Datatypes Second Edition. https://www.w3.org/TR/xmlschema11-2/#NCName.",
      "type": "string",
      "pattern": "^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$"
    }
  },
  "properties": {
    "root": {
      "$ref": "#assembly_json-value_flag_constrained-open_root"
    }
  },
  "required": [
    "root"
  ],
  "additionalProperties": false,
  "maxProperties": 1
}

The correct output expected is like the target:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://csrc.nist.gov/ns/metaschema-tests/0.1/json-value_flag_constrained-open-schema.json",
  "$comment": "JSON value testing mini metaschema: JSON Schema",
  "type": "object",
  "definitions": {
    "json-value_flag_constrained-open-json-value_flag_constrained-open:root": {
      "title": "Root",
      "description": "Root root root.",
      "$id": "#assembly_json-value_flag_constrained-open_root",
      "type": "object",
      "properties": {
        "constrained-open": {
          "$ref": "#field_json-value_flag_constrained-open_constrained-open"
        }
      },
      "additionalProperties": false
    },
    "json-value_flag_constrained-open-json-value_flag_constrained-open:constrained-open": {
      "title": "Constrained open",
      "description": "Because allow-other is 'yes' no enumeration can be given in the JSON Schema.",
      "$id": "#field_json-value_flag_constrained-open_constrained-open",
      "type": "object",
      "properties": {
        "some": {
          "title": "Some Flag",
          "description": "Some flag some flag some flag.",
          "anyOf": [
            {
              "$ref": "#/definitions/StringDatatype"
            },
            {
              "enum": [
                "one",
                "two",
                "three",
                "four"
              ]
            }
          ]
        },
        "token-value": {
          "$ref": "#/definitions/TokenDatatype"
        }
      },
      "required": [
        "token-value"
      ],
      "additionalProperties": false
    },
    "StringDatatype": {
      "description": "A non-empty string with leading and trailing whitespace disallowed. Whitespace is: U+9, U+10, U+32 or [ \n\t]+",
      "type": "string",
      "pattern": "^\\S(.*\\S)?$"
    },
    "TokenDatatype": {
      "description": "A non-colonized name as defined by XML Schema Part 2: Datatypes Second Edition. https://www.w3.org/TR/xmlschema11-2/#NCName.",
      "type": "string",
      "pattern": "^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$"
    }
  },
  "properties": {
    "root": {
      "$ref": "#assembly_json-value_flag_constrained-open_root"
    }
  },
  "required": [
    "root"
  ],
  "additionalProperties": false,
  "maxProperties": 1
}

The additional flag tests do not include the same number of tests because, as this developer fatefully learned last week after some unfortunate trial and error, allowed-values for flag elements do not have an explicit @target attribute in their syntax. Only fields can have an explicit @target and for good reason.

All Submissions:

• Have you followed the guidelines in our Contributing document?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Have you squashed any non-relevant commits and commit messages? [instructions]
• Do all automated CI/CD checks pass?

Changes to Core Features:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?
• Have you included examples of how to use your new feature(s)?
• Have you updated all website](https://pages.nist.gov/metaschema) and readme documentation affected by the changes you made? Changes to the website can be made in the website/content directory of your branch.

[aj-stein-nist]

@JustKuzya and @wendellpiez I had to enhance some of the tests and noticed the "sort of" and "narrow" cases for field enumerations (not flags) regressed back to not working correctly. I also included test data. I can walk everyone through this but for review it would be good to do the following.

1. Run test-suite/metaschema-xspec/json-schema-gen/json-schema-gen.xspec first by opening it in Oxygen to run the detected scenario. (Ensure the transforms now generate the target JSON Schema as desired)
2. Ensure any example JSON file with PASS in the name is valid and any with FAIL in the name correctly throw validation errors when configuring the similarly named target schema with Oxygen's validation menu.

I will soon create a wiki entry.

[aj-stein-nist]

@wendellpiez Dave asked in today's call that we get your review before we merge this. Do we need to talk on Monday and walk through it together or you can review it without me?

[wendellpiez]

Let's talk, pretty much at your convenience. Thanks!

Main question being not the PR but the corresponding patch in the new repo https://github.com/usnistgov/metaschema-xslt.

[JustKuzya]

Can we follow up after or precede the daily status meeting? DC From: Wendell Piez ***@***.***> Sent: Monday, June 26, 2023 9:09 AM To: usnistgov/metaschema ***@***.***> Cc: Cousin, Dmitry A. (Fed) ***@***.***>; Mention ***@***.***> Subject: Re: [usnistgov/metaschema] Update tests to support corrected JSON Schema Generation PR (PR #380) Let's talk, pretty much at your convenience. Thanks! - Reply to this email directly, view it on GitHub<#380 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACS6IVXL4YSBOAYBRNJVQN3XNGCX3ANCNFSM6AAAAAAY7XRTAQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[aj-stein-nist]

Can we follow up after or precede the daily status meeting? DC

@JustKuzya we can catch up but it seems I have a meeting after standup so perhaps we can find another time?

[JustKuzya]

OK, you seem to have the busiest calendar today - suggest the time DC From: A.J. Stein ***@***.***> Sent: Monday, June 26, 2023 9:24 AM To: usnistgov/metaschema ***@***.***> Cc: Cousin, Dmitry A. (Fed) ***@***.***>; Mention ***@***.***> Subject: Re: [usnistgov/metaschema] Update tests to support corrected JSON Schema Generation PR (PR #380) Can we follow up after or precede the daily status meeting? DC @JustKuzya<https://github.com/JustKuzya> we can catch up but it seems I have a meeting after standup so perhaps we can find another time? - Reply to this email directly, view it on GitHub<#380 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACS6IVWQ525LARTY57RNLYTXNGEOBANCNFSM6AAAAAAY7XRTAQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[wendellpiez]

Nice work, Kyle and AJ both/especially

[david-waltermire]

Many thanks to @aj-stein-nist and reviewers.

[JustKuzya]

The changes seem to work.

• I would have prefered the WIKI (or additional Readme with tests) to have more clarification on the tests purpose, workflows tested, and expected results for each test we add.
• Also, would be nice to separate Manual testing WIKI and Automated/Terminal testing WIKI/Readme guidance