application/schema-instance+json

[handrews]

Using application/json as the instance data type imposes several limitations on JSON [Hyper-]Schema. While it is essential that application/schema+json works with plain application/json instances, defining an optional instance media type to enable additional features would make several things much easier. Specifically:

• Using URI fragments with instances (application/json does not define any fragment resolution semantics), which would help with "anchor" (Hyper-schema: Link subject/anchor #140).
• Using media type parameters to connect instances to schemas and conduct content negotiation based on the schema, e.g. choosing one of several versions of a given schema.

We have at times discussed proposing registering a media type relation for application/json as part of this project, and I'm sure the idea of JSON Pointer fragments for application/json have been raised several times. But given the popularity of JSON, these things would probably have to clear a high bar, and would no doubt take quite a bit of time.

Defining an optional application/instance+json would let us manage these proposals within our own process. If they seem likely to be accepted for application/json that could still be done. There is also some precedent for this approach: JSON-LD defines application/ld+json but also provides mechanisms to interpret application/json or other +json media types as JSON-LD via a link header. We already do more or less the same thing, but as noted in #222 there are concerns with our existing approach.

Finally, my plan to get around these various issues was to define a custom vendor media type for my API. I would rather not do that if possible. And even if I have another reason to use a custom +json media type, having application/schema-instance+json [EDIT: thanks to @Relequestual for suggesting schema-instance instead of just instance) would allow me to reference it as the basis for the media type, and then only need to define additional / varying semantics. This would further encourage consistency in design and implementation, particularly for hypermedia.

Tagging @dret, @dlax, @awwright

[handrews]

Goals for application/schema-instance+json

• Define clear and useful fragment semantics
• For content negotiation, asking for application/schema-instance+json ensures a programmatically identifiable schema in the response
• Define a media type parameter for identifying that applicable schema(s), which can also be used in content negotation
• Define a media type parameter for identifying versions of relevant schema(s), also for content negotiation (this may be the same parameter, with versioning incorporated into the schema URI in some way defined by the schema/instance authors)

Goals may also include:

• Define a standard mechanism to reference schemas and schema versions from within the document
• Define a standard mechanism to associate plain name URI fragments with specific positions within the document

Unlike the first set of goals, these additional goals would introduce keywords that would make no sense if interpreted as standard application/json. However, the same is true of the "@" keywords used in JSON-LD instances that use application/json as the media type. So again, there is precedent here.

[jdesrosiers]

I like this idea. However, I don't think it belongs as part of the Hyper-Schema spec. I'd like to see it as separate project. Also, "instance" is a much too generic name :-)

[handrews]

@jdesrosiers it would most likely go in the core spec, because it exists as a pair with application/schema+json. I don't think it makes sense on its own. Maybe application/schema-instance+json. [EDIT: changed to schema-instance throughout] I just used "instance" because that's what we call them everywhere in the specs.

(Oh, I see I put the hypermedia label on this- yeah, that's a bit misleading)

[dlax]

I've beed thinking about this a bit and how it'd help solving the problem of context resource identification for anchor. Clearly, only considering the "define clear and useful fragment semantics" goal, I am a bit worried that we'd need to recommend (or even suggest) the use of a new dedicated media type: this seems too much to me. On the other hand, I wonder if it wouldn't be sufficient to suggest the use of a "profile" link header (RFC6906) on the instance's resource that'd advertise for a fragment semantics on top of JSON (e.g. Link: <https://www.rfc-editor.org/rfc/rfc6901.txt> rel="profile" for JSON pointer). It seems very appropriate to me but maybe I'm missing something.

[awwright]

We could look into defining these parameters for application/json, but until then defining a media type tied to JSON Schema seems reasonable.

Or at least easier for us, but is it easier and better for the end users?

[handrews]

@awwright good question about end users! I may be able to get some validation on that in the next six months, so having a proposal to work with in the next draft would be useful (although I could always just do it on my own).

I think it will be easier because I have occasionally run into situations where the link target is properly identified with a fragment. It's hard to explain that that's not possible with application/json.

I suspect that presenting application/schema+json and application/instance+json as a pair will make it easier on users than if we were proposing application/instance+json as its own thing. As a pair, they will motivate each other quite clearly.

Obviously this issue needs to sit open for a while before moving to a PR, but I might come up with an outline of how I would fit it in and put it here to see if it feels like something users would understand.

[dret]

On 2017-08-27 15:47, Henry Andrews wrote: I /think/ it will be easier because I have occasionally run into situations where the link target is properly identified with a fragment. It's hard to explain that that's not possible with |application/json|.

fragment identifier semantics are optionally defined per media type https://tools.ietf.org/html/rfc6838#section-4.11 JSON does not have any: https://tools.ietf.org/html/rfc7159#section-11

[handrews]

@dret yes that is exactly one of the main reasons I want to define application/instance+json. Because I need fragments, and application/json does not support them.

And you and I and @awwright understand why media types control fragment syntax, but it's something that many people find counter-intuitive. Easier to just say "use this and it will work".

[dlax]

And you and I and @awwright understand why media types control fragment syntax, but it's something that many people find counter-intuitive. Easier to just say "use this and it will work".

Acknowledging that media types control fragment syntax, is defining a new media type the only way to declare a fragment syntax on an existing media type? I thought that a profile relation could be used for that, am I wrong?

[handrews]

I thought that a profile relation could be used for that, am I wrong?

@dlax, since @dret wrote the profile RFC he can give an authoritative reply, but my interpretation is that it does not help here.

Profiles add semantics to a media type without changing any that already exist. application/json does not support fragments. However, it could register a fragment syntax at some future date. Then any profile that attempted to define a fragment syntax would be in conflict with the base media type.

Additionally, application/json does not allow for a profile media type parameter (see also #222), substantially limiting the utility of that approach.

It is possible to add fragment support and media type parameters to an existing media type, but as a widely deployed media type which serves as the basis for many +json structured extensions, I suspect the bar for doing so will be very high.

If we want to attempt that, I would expect that if we can show the utility of our approach with a successful application/instance+json media type (which is fully under our own control), our proposals would be more likely to be considered.

[dret]

On 2017-08-28 08:32, Henry Andrews wrote: I thought that a profile relation could be used for that, am I wrong? @dlax <https://github.com/dlax>, since @dret <https://github.com/dret> wrote the |profile| RFC he can give an authoritative reply, but my interpretation is that it does not help here.

a profile cannot change a media type. i think. ;-) well, given that a profile is intended to communicate additional semantics of a media type, one could argue that this could include fragment identification semantics. to be honest, that's not something i ever thought of when the RFC was written. so all i can say is that the profile RFC neither allows nor disallows this, and my gut feeling is that there might be unintended side effects if you allowed profiles to reach this deeply into the question of what a media type is about.

Profiles add semantics to a media type without changing any that already exist. |application/json| does not support fragments. However, it could register a fragment syntax at some future date. Then any profile that attempted to define a fragment syntax would be in conflict with the base media type.

exactly. that's one of the obvious side effects that would need to be considered.

Additionally, |application/json| does not allow for a |profile| media type parameter (see also #222 <#222>), substantially limiting the utility of that approach.

there is an update of JSON in the pipeline. but i have given up on trying to convince the community that a profile media type parameter would be useful. feel free to try again! if you do let me know and i will try to provide some support. https://tools.ietf.org/html/draft-ietf-jsonbis-rfc7159bis

It is possible to add fragment support and media type parameters to an existing media type, but as a widely deployed media type which serves as the basis for many |+json| structured extensions, I suspect the bar for doing so will be very high.

yes. but because there is an update in the pipeline, that bar may be a bit more attainable.

[handrews]

@dret interesting about the JSON update. Is there an actual issue tracker somewhere or is all discussion exclusively on the mailing list?

[dret]

On 2017-08-29 01:20, Henry Andrews wrote: @dret <https://github.com/dret> interesting about the JSON update. Is there an actual issue tracker somewhere or is all discussion exclusively on the mailing list?

welcome to the wonderful land of tech beards and email only discussions!

[handrews]

OK, I think from conversations here and offline, it's clear that there's enough interest to make it worth writing a PR for draft-07.

I am going to take the minimal approach and only define the parameter for content negotiation and the JSON Pointer fragment identifier syntax. These are things that can only be through a media type, so we have no real alternatives (I don't have the time and energy to sell the application/json folks on this, especially not when the only interface is listserv FFS).

If there is clear interest through feedback, we may consider the features that would depend on in-document structure (e.g. reserving "$schema" in instances). If appropriate, we can file new issues for those. However, I am excluding them from this issue.

[handrews]

Merged #405.