Proposal: "jsonpointer" type #23
Comments
|
@handrews @awwright @Relequestual any thoughts on this? |
|
@erosb can you give a more specific use case for this? I understand how the proposal works, but I cannot think of a situation where I would need this sort of indirect specification. Also, the value you have given for |
|
What @handrews said. |
|
@erosb I'm going to go ahead and submit a PR for json-schema-org/json-schema-spec#109 that will include jsonpointer as a format, because even if this proposal (#141) is accepted, not all jsonpointers are used to refer back into the same instance or to apply any further validation. Just wanted to warn you- when you see a PR for json-schema-org/json-schema-spec#109 that's not me dismissing this issue. I still would need to see some real use cases for this one (#141) before I could support it, though. |
|
Okey. These 2 proposals can even live together, json-schema-org/json-schema-spec#109 for untyped references and this for typed ones. I will get back to here with some examples when I will have some more time. |
Example for using typed JSON PointersLets consider a json format which represents 3D objects in a scenegraph and collisions of elements. So it has 2 root properties:
Each individual collision should be represented with a list of JSON pointers, pointing to the colliding objects in the scene. So generally speaking, JSON is basically designed to represent hierarchical (tree) data structures. Once the data model becomes a graph and not a tree, the most robust solution to express (possibly circular) references is using JSON Pointers. This proposal (#141) helps in making these pointers typed. So in draftv4 there is not even a way to make syntactical restrictions on a string to verify that it is a JSON Pointer, and this is the problem being solved by So as an analogy, having only untyped pointers is like working with Final thoughts:
|
It's actually quite important. "/settings/width" would be {"format": "jsonpointer"} while "#/settings/width" would be {"format": "urireference"}, with the media type of the referenced resource determining whether the fragment can be interpreted as a JSON Pointer or not. Either type could have constraints on the thing to which it is pointing, although with a URI Reference the constraint can already be described through the "profile" media type parameter and/or link, and the "describedBy" link. In that case you are relying on the referenced resource to provide the correct schema. |
|
To repeat a relevant response to your comment in json-schema-org/json-schema-spec#150 RFC 6901 does not define a representation of a JSON Pointer as a URI fragment. It defines how to encode a JSON Pointer in a URI fragment. (specifically UTF8 + percent encoding). That is just a URI fragment, and the interpretation of the fragment as a JSON Pointer is controlled by the media type of the document (which is always how fragments are interpreted). If you want to use a URI with a JSON Pointer fragment, that is just a "uri" or "uriref" format. It is not, by itself, a JSON Pointer. It would be just as reasonable to propose a reference type (with For URI References, you need to define the base URI (is it the same base used for Also keep in mind that most uses of JSON Pointer from within a schema actually need Relative JSON Pointers for the same reason that most uses of URIs are actually URI References: when re-using a bit of schema you may not know where the schema is being used, but you more often know or have defined as sub-schemas what other schemas or instances exist relative to the point at which the schema is used. |
|
@handrews the following came into my mind when reading your comment above: what about defining these references as
If What do you think? |
|
@erosb that seems like a reasonable approach. I'm not entirely sold on whether the use case is in-scope for JSON Schema, but I need to look at it a bit more. Right now I'm focusing on trying to wrap up Draft 06 so I'll probably come back to this once that's sorted. |
|
I just realized that an obvious use case for this proposal is in the meta-schema, placing restrictions on what {
"definitions": {
"jsonReference": {
"properties": {
"$ref": {
"type": "reference",
"format": "uriref",
"referredSchema": {"$ref": "#"}
}
}
}
}
}Note In issue json-schema-org/json-schema-spec#98 (comment) I just noted that this type would be useful for defining the {
"definitions": {
"validationProperties": {
"enum": ["type", "multipleOf", "maximum", ...]
},
"nonValidationSchema": {
"allOf": [
{"$ref": "#"},
{"propertyNames": {"not": {"$ref": "#/definitions/validationProperties"}}},
{
"properties": {
"$ref": {
"referredSchema": "#/definitions/nonValidationSchema"
}
}
}
]
}
},
"properties": {
"$use": {
"type": "object",
"properties": {
"source": {"$ref": "#"},
"with": {"$ref": "#/definitions/nonValidationSchema"}
}
}
}
} |
|
I'm glad to see you start liking it :) Hopefully I will have some time to work on this after Christmas. |
|
"type" is used as basic JSON type, with "integer" being the only exclusion... Maybe instead of proliferating the exclusions we should indeed consider dropping "integer" and use format for it? @erosb I don't understand how |
|
If it's an alternative to $data proposal, it seems an unnecessary complex and confusing alternative. But it seems like it is something else... I don't think I understand your use-case. |
@handrews you can't really bend the standard for the purpose of writing meta-schemas... Users write schemas, meta-schemas serve the standard, enforce it, not the other way around... |
|
Back to your question @epoberezkin :
Regarding unnecessary complexity: if you don't want to specify the schema of the referred value, then you can still create untyped references like (or even no "referredSchema" key at all) For more detailed explanation please read this comment above. It describes the generic usecase I had in mind through an other example. Thanks. |
|
@erosb I don't understand how example in the issue (pointers within data instances - which I don't understand the use case for) correlate to the comment you linked to - which seems to be explaining some real use case but can be covered with the existing vocabulary. Leaving aside the fact that you are using type keyword, that has a different meaning, I still don't understand. So far it seems very artificial. Can you please post some simplified real life example of a data/schema with and without this suggestion and explain what are you achieving with this proposal that you cannot already achieve? Is it essentially defining meta-schema for the referred schema at the point of $ref (that's what @handrews seems to be thinking)? But it's not what you have in the beginning of the issue. Lost... |
No, one lower layer of abstraction. It is defining the schema of the referred value , pointed by the reference. Of course in case this construct is used in the meta-schema, it becomes "defining meta-schema for the referred schema at the point of $ref", as you wrote (everything is on a higher abstraction layer when we talk about meta-schema). So the significant difference between the |
@epoberezkin ???? I'm not sure what you think I'm doing but I'm pretty sure I'm not doing that. I had asked for a use case and was skeptical about such existing, and then realized that we could make use of this feature in the meta-schema to help describe the existing specification. I'm not saying that we have to add this because it would be useful in the meta-schema, so I'm not sure where "bending" comes in here. I'm just saying that if we added this, then there would be a use case for it already in the meta-schema. Does that make sense? The fact that I used the meta-schema for an example doesn't give it any more weight than if I used some other random schema from out in the wild. |
|
Ok sorry :) |
@erosb But why do you need to have a referred value in data? We have doubts even about $data which is less abstract - referring to data from the schema and using it during validation (that seems to have quite a few use cases). You are not only suggesting that we should refer to the data from the data, but also to be able to validate such references. Even if there is a use case, I think it is very confusing to use "type" keyword for it - we'll need to come up with a different syntax if we agree that there is a sufficiently wide use case... It took me a lot of effort to understand what is the intention. It could be keyword $datapointer that has schema as it's value - what's the point of having two keywords where one is sufficient, particularly when one of them has different meaning completely? In any case, I still don't understand the use case... |
|
@epoberezkin no worries. |
@epoberezkin Does the use case of using it to describe Once we agree on whether that is a valid example we can then talk about what sort of other use cases might exist, but I want to make sure we're on the same page with that example first. |
|
I think I understand now, having re-read the examples by @handrews and @erosb (about collisions). I don't think these validation scenarios should be part of the spec - they seem very specialised. They also may cause extra validation of the same data - first as the data in its own location and secondly in the referred location. Validators already handle validation of referred schemas, likewise a particular implementation of scene/collision may do regex/format based validation of json pointer based on the conventions in the tree data structure and just check that they are not empty when they are resolved in application - otherwise they will be resolved and validated twice if the proposed feature is used. It's been pointed out many times that JSON-schema domain is structural data validation and it should not be concerned with semantic data validation that belongs in applications. This idea seems very much like a semantic validation. I think it can be an interesting standard extension (I think I can make a custom validation keyword for Ajv quite easily), but I don't think it should be in the validation standard core. I need to sleep on it :) |
|
"type" right now only refers to the six core primitive types defined in JSON. While This can be a new keyword, however. Also note JSON Schema doesn't provide any mechanisms for checking data consistency, only structure. There's no way to make one piece of data be valid or invalid depending on the existence/value of another. |
That is completely untrue: {
"oneOf": [
{"properties": {"foo": {"const": 10}, "bar": {"type": "integer"}}},
{"properties": {"foo": {"not": {"const": 10}}, "bar": {"type": "string"}}}
]
}requires "bar" to be an integer if "foo" is 10 and a string otherwise. |
|
@handrews I mean, in the sense the you can use "oneOf" yeah, you can. But you can't generalize that example to say "for any number |
|
Not exactly. It makes piece of data valid or invalid based on the existence of its property, not some other data. |
|
I think we need some general approach of adoption of new keywords... Similarly to how HTML/CSS/DOM is extended. The feature is implemented as experimental in some browsers, then based on the feature usage it may be adopted by other browsers and only after it it may become a standard. I would like us to have a similar approach. Implement the keyword in some validators, try using it, then add it to the spec if it is indeed useful for many people. Otherwise it can forever stay an experimental feature. It may be a separate issue, but it applies to this keyword as well. It is undoubtedly needed for some users (at least one). The question is whether there is a sufficient number of users who need it and whether it should be a part of the spec. So let's add it to some validators as an optional extension, try using it, see how many people use it and then decide. Debating it is not particularly productive... |
|
With regards to the syntax I suggest |
|
Also it seems like it should expect non-encoded absolute or relative JSON pointer, without any # character - it is not URI, it is a pointer to the current data instance, so there is no reason to use hash-fragment of URI (as there is nowhere else to point to but the current |
|
Chiming in on this issue to say that the feature being proposed here is extremely useful and important. The discussion seems to have died down, and most of that discussion was focused on syntax and confusion about the use case. Somehow the simple message got buried in this, and I'd like to try to bring it back into focus, even if this won't be considered until a later release. The general use case is straightforward: In defining a schema for some type of JSON content, that content includes a property whose value is essentially a reference to a value that occurs elsewhere. That referent value may be in the same JSON resource that contains the reference, or in some other addressable JSON resource, and the syntax should allow for both of those variants. The key requirement is this: I need to be able to specify a schema for the referenced value. It's not sufficient to say that the reference itself is a string that looks, syntactically speaking, like a valid JSON Pointer, or JSON Reference, or whatever syntax we deem appropriate. Semantically, the reference must resolve to a value, and the value must conform to the schema that I specify. The OpenAPI specification has many examples of this. It defines a Reference Object as an object with a single Reference Objects are used in many places. For example, an Operation contains a Here's an example of a valid OpenAPI 3.0 document: The
How would we specify the JSON schema for OpenAPI 3.0, to show that it expects each element of the We can use JSON Schema itself allows What's being proposed here is to provide a similar facility, one meta-level down. Just as its useful for schema designers to reuse components via typed references, it's also useful for other JSON formats to have that same kind of modular reuse by reference. And its useful for us to fully describe and validate those other JSON formats, including their typed references, using JSON Schema. The JSON schema for OpenAPI is really incomplete without this. We have to document the expected types of So... we can debate syntax, terminology, priorities, and how best to position this proposal and reconcile it with others. But it should not be hard to understand what's being proposed here. And there really shouldn't be any doubt as to whether it's important and valuable for JSON Schema users. |
|
@tedepstein thanks- it's good to get another real-world use case. I'm going to have to go back through all of this, and may just try to distill it down and re-file it given the numerous digressions and confusions. And also since we've revived Relative JSON Pointer there are now at least three different types of "pointers" (URIs, JSON Pointers, Relative JSON Pointers). |
|
I agree with @tedepstein, such feature would be very helpful to define comprehensive schema for complex or schematic values (like OpenAPI or AsyncAPI). I'm in favor of format-based syntax with string {
"type": "string",
"format":"json-pointer", // "relative-json-pointer", "uri-reference"
"target": {
"required": ["myProperty"]
}
}
{
"type": "string",
"format": "uri-reference"
} |
|
Is there any use case for explicitly following pointers that isnt resolved by supporting {
"raw": { "n": 5 },
"count": { "$ref": "#/raw/n" }
}{
"$followReferences": true, // For backwards compatibility? Not sure if this is necessary
"properties": {
"count": { "type": "integer" }
}
}Making an extra property would be more work for developers, users and implementors, and the only benefit I see is that the reference could be 9 characters shorter. NOTE: Looking into the history of this, it seems like json-schema-org/json-schema-spec#279 might be relevant, and the reasons it was shot down might apply to any processing of pointers in the input (something about it being okay in the schema because preprocessing?). Until I've updated this, take it with a grain of salt |
|
This would be better handled with an extension keyword than a new value for |
Introduction
While the JSON Schema specification itself utilizes JSON Pointers, it has only a very basic support for letting schema authors specify a schema which expects JSON documents to use such JSON Pointers. Currently, when a schema author wants to describe a JSON Pointer, the only thing he/she can do is defining the schema with
"type":"string"and probably adding a regex restriction which mandates the string to be a syntactically valid json pointer.In practical usecases it is useful to be able to describe the schema of the referred JSON value. This proposal targets these usecases.
Purpose
The purpose of this new type is to let schema authors expect that the document being validated contains a JSON Pointer, and this pointer denotes a value which conforms to the restrictions of a schema defined by the schema author.
Syntax
Example schema:
{ "properties": { "ptrToNum": { "type": "jsonpointer", "referredSchema": { "type": "number" } } } }Example valid document against the above schema:
{ "ptrToNum" : "#/settings/width", "settings" : { "width" : 4 } }Example invalid document against the above schema:
{ "ptrToNum" : "#/settings/width", "settings" : { "width" : [ 4 ] } }Validation
If a schema instance has a
"type" : "jsonpointer"property , then it must also have a"referredSchema"key and a then an implementation should perform the following validation steps:{"type" : "string"}schema"referredSchema"Notes
"referredSchema"keyword , maybe something more expressive can be betterThe text was updated successfully, but these errors were encountered: