diff --git a/jsonschema-core.md b/jsonschema-core.md index 393e8acd..698e2c7e 100644 --- a/jsonschema-core.md +++ b/jsonschema-core.md @@ -30,10 +30,11 @@ and interaction control of JSON data. This specification defines JSON Schema core terminology and mechanisms, including pointing to another JSON Schema by reference, dereferencing a JSON Schema reference, specifying the dialect being used, specifying a dialect's -vocabulary requirements, and defining the expected output. +vocabulary requirements, and defining terms. Other specifications define the vocabularies that perform assertions about -validation, linking, annotation, navigation, and interaction. +validation, linking, annotation, navigation, and interaction as well as output +formats. ## Conventions and Terminology @@ -2130,613 +2131,92 @@ Omitting this keyword has the same assertion behavior as an empty schema. ## Output Formatting {#output} -JSON Schema is defined to be platform-independent. As such, to increase -compatibility across platforms, implementations SHOULD conform to a standard -validation output format. This section describes the minimum requirements that -consumers will need to properly interpret validation results. +In order to foster increased usability and interoperability, implementations +SHOULD adhere to well-defined output formats. -### Format +Because JSON Schema has multiple uses cases, and those uses cases have different +intended consumers, this specification defers the details of any output formats +to other documents. Implementations are encouraged to support multiple output +formats as required by their target user base. -JSON Schema output is defined using the JSON Schema data instance model as -described in [Instance Data Model](#data-model). Implementations MAY deviate -from this as supported by their specific languages and platforms, however it -is RECOMMENDED that the output be convertible to the JSON format defined herein -via serialization or other means. +The scope of this section, therefore, is limited to defining common terms that +SHOULD be used in JSON Schema output specifications in order to align the +vernacular across differing formats. Output specifications which use this +information MUST use this terminology to describe it. Conversely, output +specifications which use these terms MUST maintain their meaning. -### Output Formats +### Evaluation path -This specification defines three output formats. See the "Output Structure" -section for the requirements of each format. - -- *Flag*: A boolean which simply indicates the overall validation result with no - further details. -- *List*: Provides validation information in a flat list structure. -- *Hierarchical*: Provides validation information in a hierarchical structure that -follows the evaluation paths generated while processing the schema. - -An implementation MUST provide the "flag" format and SHOULD provide at least one -of the "list" or "hierarchical" formats. Implementations SHOULD specify in their -documentation which formats they support. - -### Minimum Information - -Beyond the simplistic "flag" output, additional information is useful to aid in -debugging a schema or instance. Each sub-result SHOULD contain the information -contained within this section at a minimum. - -A single object that contains all of these components is considered an output -unit. - -Implementations MAY elect to provide additional information. - -#### Evaluation path - -The evaluation path to the schema object that produced the output unit. The -value MUST be expressed as a JSON Pointer, and it MUST include any by-reference -applicators such as `$ref` or `$dynamicRef`.[^13] - -[^13]: The schema may not actually have a value at the location indicated by -this pointer. It is provided as an indication of the traversal path only. +The evaluation path is the set of keys, starting from the schema root, through +which evaluation passes to reach the schema object that produced a specific +result. The value MUST be expressed as a JSON Pointer, and it MUST include any +by-reference applicators such as `$ref` or `$dynamicRef`. ``` /properties/width/$ref/allOf/1 ``` -Note that this pointer may not be resolvable by the normal JSON Pointer process -due to the inclusion of these by-reference applicator keywords. +Note that this pointer may not be resolvable on the root schema by the normal +JSON Pointer process. It is intended as an indication of the traversal path +only. -The JSON key for this information is "evaluationPath". +When represented in JSON, the key for this information MUST be "evaluationPath". -#### Schema Location +### Schema Location -The absolute, dereferenced location of the schema object that produced the -output unit. The value MUST be expressed using the canonical IRI of the relevant -schema resource plus a JSON Pointer fragment that indicates the schema object -that produced the output. It MUST NOT include by-reference applicators such as -`$ref` or `$dynamicRef`.[^14] - -[^14]: Note that "absolute" here is in the sense of "absolute filesystem path" -(meaning the complete location) rather than the "absolute-IRI" terminology from -RFC 3987 (meaning with scheme and without fragment). Schema locations will have -a fragment in order to identify the specific schema object. +The schema location is the canonical URI of the schema object plus a JSON +Pointer fragment indicating the subschema that produced a result. In contrast +with the evaluation path, the schema location MUST NOT include by-reference +applicators such as `$ref` or `$dynamicRef`. ``` https://example.com/schemas/common#/$defs/allOf/1 ``` -The JSON key for this information is "schemaLocation". - -#### Instance Location - -The location of the JSON value within the instance being validated. The value -MUST be expressed as a JSON Pointer. - -The JSON key for this information is "instanceLocation". - -#### Errors - -Any errors produced by the validation. This property MUST NOT be included if the -validation was successful. The value for this property MUST be an object where -the keys are the names of keywords and the values are the error message produced -by the associated keyword. - -If the subschema itself is producing the error, that error MUST be listed with -an empty string key.[^15] - -[^15]: Although there may be other cases where a subschema can produce an error, -the most common case is the `false` schema. In cases like these, there is no -keyword that produces the error, so there is nothing to use as a key. Thus the -empty string is used instead. - -The specific wording for the message is not defined by this specification. -Implementations will need to provide this. - -The JSON key for this information is "errors". - -#### Annotations - -Any annotations produced by the evaluation. This property MUST NOT be included -if the validation result of the containing subschema was unsuccessful. - -The value for this property MUST be an object where the keys are the names of -keywords and the values are the annotations produced by the associated keyword. - -Each keyword defines its own annotation data type (e.g. `properties` produces a -list of keywords, whereas `title` produces a string). - -The JSON key for this information is "annotations". - -#### Dropped Annotations - -Any annotations produced and subsequently dropped by the evaluation due to an -unsuccessful validation result of the containing subschema. This property MAY be -included if the validation result of the containing subschema was unsuccessful. -It MUST NOT be included if the local validation result of the containing -subschema was successful. - -Implementations that wish to provide these annotations MUST NOT provide them as -their default behavior. These annotations MUST only be included by explicitly -configuring the implementation to do so. - -The value for this property MUST be an object where the keys are the names of -keywords and the values are the annotations produced by the associated keyword. - -Each keyword defines its own annotation data type (e.g. `properties` produces a -list of keywords, whereas `title` produces a string). - -The JSON key for this information is "droppedAnnotations". - -#### Results from Subschemas - -Evaluation results generated by applying a subschema to the instance or a child -of the instance. Keywords which have multiple subschemas (e.g. `anyOf`) will -generally generate an output unit for each subschema. In order to accommodate -potentially multiple results, the value of this property MUST be an array of -output units, even if only a single output unit is produced. - -For "list", this property will appear only at the root output unit and will hold -all output units in a flat list. - -For "hierarchical", this property will contain results in a tree structure where -each output unit may itself have further nested results. +### Instance Location -The sequence of output units within this list is not specified and MAY be -determined by the implementation. Sets of output units are considered equivalent -if they contain the same units, in any order. +The instance location is the location of the JSON value within the root instance +being validated. The value MUST be expressed as a JSON Pointer. -The JSON key for these additional results is "details". +### Errors -### Output Structure +Errors are textual representations of individual validation failures, often +intended for human consumers. This specification contains no requirements for +the content of these errors. -The output MUST be an object containing a boolean property named "valid". When -additional information about the result is required, the output MUST also -contain "details" as described below. +Output specifications which include errors SHOULD be written such that the +sources (schema and instance) of a given error is easily identifiable and SHOULD +use the terms defined by this document to do so. -valid: a boolean value indicating the overall validation success or failure +### Annotations -details: the collection of results produced by subschemas +Many keywords are defined to produce annotations, whether intended for +inter-keyword communication (e.g. between `properties` and +`unevaluatedProperties`) or for application consumption (e.g. `title` or +`readOnly`). Annotation values may be of any type and are defined by the +keywords that produced them. -For these examples, the following schema and instances will be used. +Output specifications which include annotations SHOULD be written such that they +can be easily associated with the data defined in {{collect}} and SHOULD use the +terms defined by this document to do so. -```jsonschema -{ - "$schema": "https://json-schema.org/draft/next/schema", - "$id": "https://json-schema.org/schemas/example", - "type": "object", - "title": "root", - "properties": { - "foo": { - "allOf": [ - { "required": [ "unspecified-prop" ] }, - { - "type": "object", - "title": "foo-title", - "properties": { - "foo-prop": { - "const": 1, - "title": "foo-prop-title" - } - }, - "additionalProperties": { "type": "boolean" } - } - ] - }, - "bar": { - "$ref": "#/$defs/bar" - } - }, - "$defs": { - "bar": { - "type": "object", - "title": "bar-title", - "properties": { - "bar-prop": { - "type": "integer", - "minimum": 10, - "title": "bar-prop-title" - } - } - } - } -} -``` +### Dropped Annotations -```json "Failing instance" -{ - "foo": {"foo-prop": "not 1", "other-prop": false}, - "bar": {"bar-prop": 2} -} -``` - -```json "Passing instance" -{ - "foo": { - "foo-prop": 1, - "unspecified-prop": true - }, - "bar": {"bar-prop": 20} -} -``` - -The failing instance will produce the following errors: - -- The value at `/foo` evaluated at `/properties/foo/allOf/0` by following the - path `/properties/foo/allOf/0` by the `required` keyword is missing the - property `unspecified-prop`. -- The value at `/foo/foo-prop` evaluated at - `/properties/foo/allOf/1/properties/foo-prop` by following the path - `/properties/foo/allOf/1/properties/foo-prop` by the `const` keyword is not - the constant value 1. -- The value at `/bar/bar-prop` evaluated at `/$defs/bar/properties/bar-prop` by - following the path `/properties/bar/$ref/properties/bar-prop` by the `type` - keyword is not a number.[^16][^17] - -[^16]: `minimum` doesn't produce an error because it only operates on instances -that are numbers. - -[^17]: Note that the error message wording as depicted in the examples below is -not a requirement of this specification. Implementations SHOULD craft error -messages tailored for their audience or provide a templating mechanism that -allows their users to craft their own messages. - -The passing instance will produce the following annotations: - -- The keyword `title` evaluated at the instance root by the root schema will produce - `"root"`. -- The keyword `properties` evaluated at instance root by the root schema will produce - `["foo", "bar"]`. -- The keyword `title` evaluated at `/properties/foo` by following the path - `/properties/foo` will produce `"foo-title"`. -- The keyword `properties` evaluated at `/properties/foo/allOf/1` by following - the path `/properties/foo/allOf/1` will produce `["foo-prop"]`. -- The keyword `additionalProperties` evaluated at `/properties/foo/allOf/1` by - following the path `/properties/foo/allOf/1` will produce - `["unspecified-prop"]`. -- The keyword `title` evaluated at `/properties/foo/allOf/1/properties/foo-prop` - by following the path `/properties/foo/allOf/1/properties/foo-prop` will - produce `"foo-prop-title"`. -- The keyword `title` evaluated at `/$defs/bar` by following the path - `/properties/bar/$ref` will produce `"bar-title"`. -- The keyword `properties` evaluated at `/$defs/bar` by following the path - `/properties/var/$ref` will produce `["bar-prop"]`. -- The keyword `title` evaluated at `/$defs/bar/properties/bar-prop` by following - the path `/properties/bar/$ref/properties/bar-prop` will produce - `"bar-prop-title"`. - -#### Flag - -In the simplest case, merely the boolean result for the "valid" valid property -needs to be fulfilled. For this format, all other information is explicitly -omitted. - -```json -{ - "valid": false -} -``` - -Because no errors or annotations are returned with this format, it is -RECOMMENDED that implementations use short-circuiting logic to return failure or -success as soon as the outcome can be determined. For example, if an `anyOf` -keyword contains five subschemas, and the second one passes, there is no need to -check the other three. The logic can simply return with success. - -#### List - -The "List" structure is a flat list of output units contained within a root -output unit. - -The root output unit contains "valid" for the overall result and "details" for -the list of specific results. All other information is explicitly omitted from -the root output unit. If the root schema produces errors or annotations, then -the output node for the root MUST be present within the root output unit's -"details" list with those errors or annotations. - -Output units which do not contain errors or annotations SHOULD be excluded from -this format, however implementations MAY choose to include them for -completeness. - -```json "Failing results" -{ - "valid": false, - "details": [ - { - "valid": false, - "evaluationPath": "/properties/foo/allOf/0", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", - "instanceLocation": "/foo", - "errors": { - "required": "Required properties [\"unspecified-prop\"] were not present" - } - }, - { - "valid": false, - "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", - "instanceLocation": "/foo/foo-prop", - "errors": { - "const": "Expected \"1\"" - } - }, - { - "valid": false, - "evaluationPath": "/properties/bar/$ref/properties/bar-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", - "instanceLocation": "/bar/bar-prop", - "errors": { - "minimum": "2 is less than or equal to 10" - } - } - ] -} -``` - -```json "Passing results" -{ - "valid": true, - "details": [ - { - "valid": true, - "evaluationPath": "", - "schemaLocation": "https://json-schema.org/schemas/example#", - "instanceLocation": "", - "annotations": { - "title": "root", - "properties": [ - "foo", - "bar" - ] - } - }, - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", - "instanceLocation": "/foo", - "annotations": { - "title": "foo-title", - "properties": [ - "foo-prop" - ], - "additionalProperties": [ - "unspecified-prop" - ] - } - }, - { - "valid": true, - "evaluationPath": "/properties/bar/$ref", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", - "instanceLocation": "/bar", - "annotations": { - "title": "bar-title", - "properties": [ - "bar-prop" - ] - } - }, - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", - "instanceLocation": "/foo/foo-prop", - "annotations": { - "title": "foo-prop-title" - } - }, - { - "valid": true, - "evaluationPath": "/properties/bar/$ref/properties/bar-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", - "instanceLocation": "/bar/bar-prop", - "annotations": { - "title": "bar-prop-title" - } - } - ] -} -``` - -#### Hierarchical - -The "Hierarchical" structure is a tree structure that follows the evaluation -path during the validation process. Typically, it will resemble the schema as if -all referenced schemas were inlined in place of their associated by-reference -keywords. - -All output units are included in this format. - -The location properties of the root output unit MAY be omitted. - -```json "Failing results (errors)" -{ - "valid": false, - "evaluationPath": "", - "schemaLocation": "https://json-schema.org/schemas/example#", - "instanceLocation": "", - "details": [ - { - "valid": false, - "evaluationPath": "/properties/foo", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", - "instanceLocation": "/foo", - "details": [ - { - "valid": false, - "evaluationPath": "/properties/foo/allOf/0", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", - "instanceLocation": "/foo", - "errors": { - "required": "Required properties [\"unspecified-prop\"] were not present" - } - }, - { - "valid": false, - "evaluationPath": "/properties/foo/allOf/1", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", - "instanceLocation": "/foo", - "droppedAnnotations": { - "properties": [ "foo-prop" ], - "title": "foo-title" - }, - "details": [ - { - "valid": false, - "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", - "instanceLocation": "/foo/foo-prop", - "errors": { - "const": "Expected \"1\"" - }, - "droppedAnnotations": { - "title": "foo-prop-title" - } - }, - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1/additionalProperties", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", - "instanceLocation": "/foo/other-prop" - } - ] - } - ] - }, - { - "valid": false, - "evaluationPath": "/properties/bar", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", - "instanceLocation": "/bar", - "details": [ - { - "valid": false, - "evaluationPath": "/properties/bar/$ref", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", - "instanceLocation": "/bar", - "droppedAnnotations": { - "properties": [ "bar-prop" ], - "title": "bar-title" - }, - "details": [ - { - "valid": false, - "evaluationPath": "/properties/bar/$ref/properties/bar-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", - "instanceLocation": "/bar/bar-prop", - "errors": { - "minimum": "2 is less than or equal to 10" - }, - "droppedAnnotations": { - "title": "bar-prop-title" - } - } - ] - } - ] - } - ] -} -``` - -```json "Passing results (annotations)" -{ - "valid": true, - "evaluationPath": "", - "schemaLocation": "https://json-schema.org/schemas/example#", - "instanceLocation": "", - "annotations": { - "title": "root", - "properties": [ - "foo", - "bar" - ] - }, - "details": [ - { - "valid": true, - "evaluationPath": "/properties/foo", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", - "instanceLocation": "/foo", - "details": [ - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/0", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", - "instanceLocation": "/foo" - }, - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", - "instanceLocation": "/foo", - "annotations": { - "title": "foo-title", - "properties": [ - "foo-prop" - ], - "additionalProperties": [ - "unspecified-prop" - ] - }, - "details": [ - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", - "instanceLocation": "/foo/foo-prop", - "annotations": { - "title": "foo-prop-title" - } - }, - { - "valid": true, - "evaluationPath": "/properties/foo/allOf/1/additionalProperties", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", - "instanceLocation": "/foo/unspecified-prop" - } - ] - } - ] - }, - { - "valid": true, - "evaluationPath": "/properties/bar", - "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", - "instanceLocation": "/bar", - "details": [ - { - "valid": true, - "evaluationPath": "/properties/bar/$ref", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", - "instanceLocation": "/bar", - "annotations": { - "title": "bar-title", - "properties": [ - "bar-prop" - ] - }, - "details": [ - { - "valid": true, - "evaluationPath": "/properties/bar/$ref/properties/bar-prop", - "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", - "instanceLocation": "/bar/bar-prop", - "annotations": { - "title": "bar-prop-title" - } - } - ] - } - ] - } - ] -} -``` +A dropped annotation is any annotation produced and subsequently dropped by the +evaluation due to an unsuccessful validation result of the containing subschema. +This information MAY be included if the validation result of the containing +subschema was unsuccessful. It MUST NOT be included if the local validation +result of the containing subschema was successful. -#### Output validation schemas +As the intended purpose for including these annotations is debugging, +implementations that wish to provide dropped annotations SHOULD NOT provide them +as their default behavior. Dropped annotations SHOULD only be included when the +implementation is explicitly configured to do so or if the implementation is +specifically intended to be used as a debugging tool. -For convenience, JSON Schema has been provided to validate output generated by -implementations. Its IRI is: . +Output specifications which include dropped annotations SHOULD be written such +that they can be easily associated with the data defined in {{collect}} and +SHOULD use the terms defined by this document to do so. ## Security Considerations {#security} diff --git a/jsonschema-validation-output-machines.md b/jsonschema-validation-output-machines.md new file mode 100644 index 00000000..398162bb --- /dev/null +++ b/jsonschema-validation-output-machines.md @@ -0,0 +1,655 @@ +# A Specification for Machine-Readable Output for JSON Schema Validation and Annotation + +JSON Schema is defined to be platform-independent. As such, to increase +compatibility across platforms, implementations SHOULD conform to a standard +validation output format. This specification describes the minimum requirements +for machine consumers to properly interpret validation results. + +## Table of Contents + +## Schema Identifiers + +The output defined in this specification requires that the evaluation root be +defined with an absolute IRI, i.e. using the `$id` keyword. In the event an +absolute IRI has not been defined, the implementation MUST generate one. + +There are no requirements on the form of IRI itself, except that it MUST be +absolute. + +## Textual Format and Encoding + +JSON Schema output is defined using the JSON Schema data instance model as +described in [JSON Schema](#json-schema) "Instance Data Model". Implementations +MAY deviate from this in their internal modelling, as supported by their +specific languages and platforms, however it is RECOMMENDED that the output be +convertible to the JSON format defined herein via serialization or other means. + +## Minimum Information + +Beyond the simplistic "flag" output, additional information is useful to aid in +debugging evaluation of an instance by a schema. + +The output of a subschema validation is considered an "output unit." The +contents of each output unit is specified by this section. + +Each output unit MUST contain the [validation result](#validation-result) for +the associated subschema as well as the following information defined by +[JSON Schema](#json-schema) "Output Formatting": + +- Evaluation Path +- Schema Location +- Instance Location + +The following information MAY be included conditionally: + +- When subschema validation has succeeded + - Annotations +- When subschema validation has failed + - Errors + - Dropped Annotations + +Implementations MAY elect to provide additional information. + +### Validation Result {#validation-result} + +The validation result is a boolean that indicates whether the local instance +passed validation by the local subschema. + +The JSON key for these additional results is `valid`. + +### Results from Subschemas + +Evaluation results generated by applying a subschema to the instance or a child +of the instance. Keywords which have multiple subschemas (e.g. `anyOf`) will +generally generate an output unit for each subschema. In order to accommodate +potentially multiple results, the value of this property MUST be an array of +output units, even if only a single output unit is produced. + +For "list", this property will appear only at the root output unit and will hold +all output units in a flat list. + +For "hierarchical", this property will contain results in a tree structure where +each output unit may itself have further nested results. + +The sequence of output units within this list is not specified and MAY be +determined by the implementation. Sets of output units are considered equivalent +if they contain the same units, in any order. + + + +The JSON key for these additional results is `details`. + +## Output Structure {#output-structure} + +This specification defines three output formats. + +- **Flag** - A boolean which simply indicates the overall validation result with + no further details. +- **List** - Provides validation information in a flat list structure. +- **Hierarchical** - Provides validation information in a hierarchical structure + that follows the evaluation paths generated while processing the schema. + +An implementation MUST provide the "flag" format and SHOULD provide at least one +of the "list" or "hierarchical" formats. Implementations SHOULD specify in their +documentation which formats they support. + +For these examples, the following schema and instances will be used. + +```jsonschema "Example Schema" +{ + "$schema": "https://json-schema.org/draft/next/schema", + "$id": "https://json-schema.org/schemas/example", + "type": "object", + "title": "root", + "properties": { + "foo": { + "allOf": [ + { "required": ["unspecified-prop"] }, + { + "type": "object", + "title": "foo-title", + "properties": { + "foo-prop": { + "const": 1, + "title": "foo-prop-title" + } + }, + "additionalProperties": { "type": "boolean" } + } + ] + }, + "bar": { "$ref": "#/$defs/bar" } + }, + "$defs": { + "bar": { + "type": "object", + "title": "bar-title", + "properties": { + "bar-prop": { + "type": "integer", + "minimum": 10, + "title": "bar-prop-title" + } + } + } + } +} +``` + +```json "Failing Instance" +{ + "foo": { "foo-prop": "not 1", "other-prop": false }, + "bar": { "bar-prop": 2 } +} +``` + +```json "Passing Instance" +{ + "foo": { + "foo-prop": 1, + "unspecified-prop": true + }, + "bar": { "bar-prop": 20 } +} +``` + +The failing instance will produce the following errors: + +- The instance value at `/foo` + evaluated at `/properties/foo/allOf/0` + by following the path `/properties/foo/allOf/0` + by the `required` keyword + is missing the property `unspecified-prop`. +- The value at `/foo/foo-prop` + evaluated at `/properties/foo/allOf/1/properties/foo-prop` + by following the path `/properties/foo/allOf/1/properties/foo-prop` + by the `const` keyword + is not the constant value 1. +- The value at `/bar/bar-prop` + evaluated at `/$defs/bar/properties/bar-prop` + by following the path `/properties/bar/$ref/properties/bar-prop` + by the `type` keyword + is not a number. + + + + +The passing instance will produce the following annotations: + +- The keyword `title` + evaluated at `` + by following the path `` + will produce `"root"`. +- The keyword `properties` + evaluated at `` + by following the path `` + will produce `["foo", "bar"]`. +- The keyword `title` + evaluated at `/properties/foo` + by following the path `/properties/foo` + will produce `"foo-title"`. +- The keyword `properties` + evaluated at `/properties/foo/allOf/1` + by following the path `/properties/foo/allOf/1` + will produce `["foo-prop"]`. +- The keyword `additionalProperties` + evaluated at `/properties/foo/allOf/1` + by following the path `/properties/foo/allOf/1` + will produce `["unspecified-prop"]`. +- The keyword `title` + evaluated at `/properties/foo/allOf/1/properties/foo-prop` + by following the path `/properties/foo/allOf/1/properties/foo-prop` + will produce `"foo-prop-title"`. +- The keyword `title` + evaluated at `/$defs/bar` + by following the path `/properties/bar/$ref` + will produce `"bar-title"`. +- The keyword `properties` + evaluated at `/$defs/bar` + by following the path `/properties/var/$ref` + will produce `["bar-prop"]`. +- The keyword `title` + evaluated at `/$defs/bar/properties/bar-prop` + by following the path `/properties/bar/$ref/properties/bar-prop` + will produce `"bar-prop-title"`. + +### Flag + +In the simplest case, merely the boolean result for the `valid` valid property +needs to be fulfilled. For this format, all other information is explicitly +omitted. + +```json "Flag Results" +{ + "valid": false +} +``` + +Because no errors or annotations are returned with this format, it is +RECOMMENDED that implementations use short-circuiting logic to return failure or +success as soon as the outcome can be determined. For example, if an `anyOf` +keyword contains five subschemas, and the second one passes, there is no need to +check the other three. The logic can simply return with success. + +### List + +The "list" structure is a flat list of output units contained within a root +output unit. + +The root output unit contains `valid` for the overall result and `details` for +the list of specific results. All other information is explicitly omitted from +the root output unit. If the root schema produces errors or annotations, then +the output node for the root MUST be present within the root output unit's +`details` list with those errors or annotations. + +Output units which do not contain errors or annotations SHOULD be excluded from +this format, however implementations MAY choose to include them for +completeness. + +```json "Failing Results" +{ + "valid": false, + "details": [ + { + "valid": false, + "evaluationPath": "/properties/foo/allOf/0", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", + "instanceLocation": "/foo", + "errors": { + "required": "Required properties [\"unspecified-prop\"] were not present" + } + }, + { + "valid": false, + "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", + "instanceLocation": "/foo/foo-prop", + "errors": { + "const": "Expected \"1\"" + } + }, + { + "valid": false, + "evaluationPath": "/properties/bar/$ref/properties/bar-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", + "instanceLocation": "/bar/bar-prop", + "errors": { + "minimum": "2 is less than or equal to 10" + } + } + ] +} +``` + +```json "Passing Results" +{ + "valid": true, + "details": [ + { + "valid": true, + "evaluationPath": "", + "schemaLocation": "https://json-schema.org/schemas/example#", + "instanceLocation": "", + "annotations": { + "title": "root", + "properties": [ + "foo", + "bar" + ] + } + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", + "instanceLocation": "/foo", + "annotations": { + "title": "foo-title", + "properties": [ + "foo-prop" + ], + "additionalProperties": [ + "unspecified-prop" + ] + } + }, + { + "valid": true, + "evaluationPath": "/properties/bar/$ref", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "instanceLocation": "/bar", + "annotations": { + "title": "bar-title", + "properties": [ + "bar-prop" + ] + } + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", + "instanceLocation": "/foo/foo-prop", + "annotations": { + "title": "foo-prop-title" + } + }, + { + "valid": true, + "evaluationPath": "/properties/bar/$ref/properties/bar-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", + "instanceLocation": "/bar/bar-prop", + "annotations": { + "title": "bar-prop-title" + } + } + ] +} +``` + +### Hierarchical + +The "Hierarchical" structure is a tree structure that follows the evaluation +path during the validation process. Typically, it will resemble the schema as if +all referenced schemas were inlined in place of their associated by-reference +keywords. + +All output units are included in this format. + +The location properties of the root output unit MAY be omitted. + +```json "failing Results +{ + "valid": false, + "evaluationPath": "", + "schemaLocation": "https://json-schema.org/schemas/example#", + "instanceLocation": "", + "details": [ + { + "valid": false, + "evaluationPath": "/properties/foo", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", + "instanceLocation": "/foo", + "details": [ + { + "valid": false, + "evaluationPath": "/properties/foo/allOf/0", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", + "instanceLocation": "/foo", + "errors": { + "required": "Required properties [\"unspecified-prop\"] were not present" + } + }, + { + "valid": false, + "evaluationPath": "/properties/foo/allOf/1", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", + "instanceLocation": "/foo", + "droppedAnnotations": { + "properties": [ "foo-prop" ], + "title": "foo-title" + }, + "details": [ + { + "valid": false, + "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", + "instanceLocation": "/foo/foo-prop", + "errors": { + "const": "Expected \"1\"" + }, + "droppedAnnotations": { + "title": "foo-prop-title" + } + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/additionalProperties", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", + "instanceLocation": "/foo/other-prop" + } + ] + } + ] + }, + { + "valid": false, + "evaluationPath": "/properties/bar", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", + "instanceLocation": "/bar", + "details": [ + { + "valid": false, + "evaluationPath": "/properties/bar/$ref", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "instanceLocation": "/bar", + "droppedAnnotations": { + "properties": [ "bar-prop" ], + "title": "bar-title" + }, + "details": [ + { + "valid": false, + "evaluationPath": "/properties/bar/$ref/properties/bar-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", + "instanceLocation": "/bar/bar-prop", + "errors": { + "minimum": "2 is less than or equal to 10" + }, + "droppedAnnotations": { + "title": "bar-prop-title" + } + } + ] + } + ] + } + ] +} +``` + +```json "Passing Results +{ + "valid": true, + "evaluationPath": "", + "schemaLocation": "https://json-schema.org/schemas/example#", + "instanceLocation": "", + "annotations": { + "title": "root", + "properties": [ + "foo", + "bar" + ] + }, + "details": [ + { + "valid": true, + "evaluationPath": "/properties/foo", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo", + "instanceLocation": "/foo", + "details": [ + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/0", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/0", + "instanceLocation": "/foo" + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1", + "instanceLocation": "/foo", + "annotations": { + "title": "foo-title", + "properties": [ + "foo-prop" + ], + "additionalProperties": [ + "unspecified-prop" + ] + }, + "details": [ + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/properties/foo-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/properties/foo-prop", + "instanceLocation": "/foo/foo-prop", + "annotations": { + "title": "foo-prop-title" + } + }, + { + "valid": true, + "evaluationPath": "/properties/foo/allOf/1/additionalProperties", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/foo/allOf/1/additionalProperties", + "instanceLocation": "/foo/unspecified-prop" + } + ] + } + ] + }, + { + "valid": true, + "evaluationPath": "/properties/bar", + "schemaLocation": "https://json-schema.org/schemas/example#/properties/bar", + "instanceLocation": "/bar", + "details": [ + { + "valid": true, + "evaluationPath": "/properties/bar/$ref", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar", + "instanceLocation": "/bar", + "annotations": { + "title": "bar-title", + "properties": [ + "bar-prop" + ] + }, + "details": [ + { + "valid": true, + "evaluationPath": "/properties/bar/$ref/properties/bar-prop", + "schemaLocation": "https://json-schema.org/schemas/example#/$defs/bar/properties/bar-prop", + "instanceLocation": "/bar/bar-prop", + "annotations": { + "title": "bar-prop-title" + } + } + ] + } + ] + } + ] +} +``` + +## Output validation schemas + +For convenience, JSON Schema has been provided to validate output generated by +implementations. Its IRI is: + +## Filtering + +For various reasons, including human readability and performance during +evaluation, a user may desire to omit certain details from the output. To +address this, implementations MAY provide filtering output information. + +This section defines several RECOMMENDED approaches to filtering, however +implementations MAY choose to provide these features in a way that suits them +and their users best. + +For those implementations which support filtering behaviors, the behaviors MUST +be configurable and disabled by default. + +### Annotation Filtering + +Collecting and storing annotations can require increasing amounts of system +resources as evaluation proceeds. The strain on systems can be reduced by only +collecting those annotations which users care about. + +For annotation filtering, implementations SHOULD provide a mechanism which +allows users to configure either the set annotations they would like to keep or +the set they would like to ignore. + +For a "keep" list, an implementation MUST include in the output only those +annotations which are configured. + +For an "ignore" list, an implementation MUST NOT include in the output those +annotations which are configured. + +#### Annotations Required for Evaluation + +Some implementations may be annotation-driven. That is, the evaluation of some +keywords depends on the annotation results of other keywords. For example, which +properties are seen by `additionalProperties` can be determined by looking at +the annotation results of `properties` and `patternProperties`. + +For these implementations, the annotation results of these dependent keywords +must still be collected for proper evaluation. However, these annotations MUST +NOT be included in the output unless they are configured accordingly. + +### Output Unit Pruning + +In many cases, the output structures defined herein produce output units for +more than is necessary to identify evaluation issues. Implementations SHOULD +provide a mechanism which removes unimportant output units from the final +structure in order to highlight those units which impact the final result. + +Reasons to omit output units may include, but are not limited to: + +- Child output units whose validation result does not impact the validation + result of the parent. For example, a subschema of an `anyOf` which has a false + validation result when there exists a sibling subschema with a true validation + result. +- Child output units which have a true validation result but contain no + annotations. + +Output units which include annotations MUST NOT be pruned. + +Implementations which provide this behavior SHOULD provide configuration +mechanisms appropriate for their users' needs. + +## References + +### Normative References + +#### [RFC2119] {#rfc2119} + +Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, +RFC 2119, DOI 10.17487/RFC2119, March 1997, +<>. + +#### [RFC3986] {#rfc3986} + +Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier +(URI): Generic Syntax", STD 66, RFC 3986, DOI 10.17487/RFC3986, January 2005, +<>. + +#### [RFC3987] {#rfc3987} + +Duerst, M. and M. Suignard, "Internationalized Resource Identifiers (IRIs)", RFC +3987, DOI 10.17487/RFC3987, January 2005, +<>. + +#### [RFC6901] {#rfc6901} + +Bryan, P., Ed., Zyp, K., and M. Nottingham, Ed., "JavaScript Object Notation +(JSON) Pointer", RFC 6901, DOI 10.17487/RFC6901, April 2013, +<>. + +#### [RFC8259] {#rfc8259} + +Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", +STD 90, RFC 8259, DOI 10.17487/RFC8259, December 2017, +<>. + +#### [json-schema] {#json-schema} + +Wright, A., Andrews, H., Hutton, B., and G. Dennis, "JSON Schema: A Media Type +for Describing JSON Documents", Work in Progress, Internet-Draft, +draft-bhutton-json-schema-01, June 2022, +<>. diff --git a/package.json b/package.json index 3e806283..e5e2a18f 100644 --- a/package.json +++ b/package.json @@ -6,9 +6,10 @@ "main": "index.js", "scripts": { "lint": "eslint build/", - "build": "npm run build-core && npm run build-validation", + "build": "npm run build-core && npm run build-validation && npm run build-output", "build-core": "node build/build.js < jsonschema-core.md > jsonschema-core.html", - "build-validation": "node build/build.js < jsonschema-validation.md > jsonschema-validation.html" + "build-validation": "node build/build.js < jsonschema-validation.md > jsonschema-validation.html", + "build-output": "node build/build.js < jsonschema-validation-output-machines.md > jsonschema-validation-output-machines.html" }, "license": "MIT", "dependencies": {