Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PROCESS.md documenting the new SDLC #1348

Closed
wants to merge 3 commits into from
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions PROCESS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# JSON Schema Specification Development Process
This document formally defines the process used to evolve the JSON Schema
specification as of the first stable release in 2023. It applies to the core
specification and the specifications for any dialects and vocabularies
maintained by the JSON Schema Org. It does not necessarily apply to everything
the JSON Schema Org maintains. For example, media type registration follows the
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved
IETF process. Certain components used within JSON Schema, such as Relative JSON
Pointer, may also follow a separate process.

## Canonical URLs
There MUST be a stable canonical URL for referencing any specification that
follows this process. If the spec is made available from any other URLs, they
SHOULD redirect to the canonical URL. If the canonical URL is changed in the
future, all previous canonical URLs MUST remain accessible as redirects to the
current URL.

## Mutable Spec
There is only one version of the specification and it can be updated at any time
as long as those changes don't violate the compatibility rules identified by the
stability model. Bug fixes, clarifications, and other non-functional spec
updates can be merged at any time even for stable features. A change-log blog
post SHOULD be prepared quarterly to give visibility to any changes that were
merged in the last quarter.

## The Stability Model
Every feature in the spec has a release status. It's either evolving, stable, or
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved
deprecated. Flags are used to show the status a feature is in. If a feature
doesn't have a flag, it's considered stable. If it has a year flag (such as
`2024`) that's the year the feature reached stability. The other flags indicate
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're considering flagless features as stable, is there another indicator as to when the feature reached stability? Or does "doesn't have a flag" only pertain to the status flag? Do we expect all stable features to have a year flag? If one doesn't is that just an admin error that needs to be fixed?

Maybe it's easier to require that features have flags so that if they don't it's something to be fixed. Having an allowance for not having a flag seems to raise a lot of questions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A part of the spec can only be unflagged if it's a stable feature that was introduced in the original stable release.

My thinking was that it would be too noisy to flag everything in the spec. It would be hard to see 2024 flags in a sea of 2023 flags. The first stable release wouldn't have any 2023 flags because it's unambiguous when those features were introduced since 2023 is the first release. Stable flags would start to be added for 2024. That means that the number of flags in the spec should be fairly small and therefore stand out to readers.

I'll revise to clarify the purpose of unflagged features, but let's continue the discussion and get some more opinions if you still dislike the idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we keep the "unflagged implies stable" idea, and instead of flagging the year for future features, we have a newly-stable flag that identifies features that are new for the year's release. Then the next year, those newly-stable flags are cleared from those features so that the new newly-stable features stand out.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your changes look fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss it at the next OCWM

a feature that is not yet stable or is deprecated.

### Champions
Any feature that is evolving SHOULD have a champion. The champion is responsible
for shepherding the feature through the process to into stable status. The
champion doesn't need to do all the work, they're just responsible for making
sure the work gets done. The champion role MAY be shared by more than one
person. An evolving feature without a champion SHOULD be removed from the spec.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to see that first requirement as a MUST.
I guess it was framed this way to allow for a change of champion should an existing champion want to step down or vanish?
I'd much rather be stricter here and spell out even a workflow for changing champions and putting a feature "on hold" or some such until a champion is found/appointed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fully expect champions to disappear on occasion. In that case, a new champion would be named or the feature would be abandoned and removed. The "SHOULD" is to give us some leeway if for some reason we can't or don't want to assign a champion right away, but also don't want to remove the feature. It's not a likely scenario. I'm fine with calling it a MUST if we want to.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather call it a MUST. As for remaining in the spec, I think that would need to be a case by case basis.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If having a champion is a MUST, that means if a feature loses its champion for some reason, our only choices are to assign a new champion or remove it from the spec. If we don't remove it from the spec, then we are in violation of the MUST. If you want spec removal to be on a case-by-case basis rather than a MUST remove, we need to go with "SHOULD" to give us that option.


### Proposals
New features start out as proposals. Proposals are informal GitHub Issues or
Discussions and can be submitted by anyone.

All proposals MUST be backward compatible with the current stable specification.
Features that could be problematic for forward compatibility are highly
discouraged, but may be considered if the feature is optional and disabled by
default.

**Promotion Checklist:**
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved
[] Consensus among JSON Schema Org core contributors that the proposal is worth
pursuing
[] The proposal has a champion
[] A PR for an entry for in the `STAGE-0` registry has been approved
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved

### STAGE-0
`STAGE-0` features are tracked in the `STAGE-0` Registry on the website. This
registry lists all `STAGE-0` features. Each entry SHOULD include the following
checklist as well as an informal description with at least one example and any
relevant links.

This stage requires at least two implementations to implement the feature. The
purpose of these implementations is to identify any potential implementation
issues that might lead to low adoption such as difficulty of implementation or
performance implications. Therefore, these implementations don't need to be
publicly available or have real world use at this point.

**Promotion Checklist:**
[] There is general consensus among the core contributors for adding the feature
to the spec
[] Tests are available in the test suite
[] Two implementations have implemented the feature and pass the tests suite
[] A blog post has been published introducing the feature
[] A PR for the spec has been approved that completely specifies the feature and
removes the feature from the `STAGE-0` registry.

### STAGE-1
`STAGE-1` features are included to the specification, but flagged as `STAGE-1`
jdesrosiers marked this conversation as resolved.
Show resolved Hide resolved
and are not subject to the compatibility requirements of "stable" features. They
can be added at any time as long as it meets all criteria for `STAGE-1` status.

At this stage, the feature should be seeing real world use. These features MAY
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved
change or be removed all together based on what we learn from real world usage,
but anything more than small changes should be extremely rare. Any necessary
feature experimentation and evolution should have be done in `STAGE-0`.
jdesrosiers marked this conversation as resolved.
Show resolved Hide resolved

Implementers are encouraged to implement `STAGE-1` features, but are not
expected to maintain support for previous versions if they change. Users who
choose to use these features should be prepared for the possibility of changes
and be willing to update their schemas if a change does occur.

A `STAGE-1` feature can be promoted to `STAGE-2` at any time. Ideally a
feature should stay in `STAGE-1` for about one year, but may stay longer if we
don't see it used enough in the wild.

**Promotion Checklist:**
[] There is general consensus that the feature has been proven to be a good
addition to JSON Schema and is unlikely to change.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...general consensus...

Determined by whom?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoever the JSON Schema Org active contributors happen to be at the time. Maybe at some point that will be a specific and official group of people like a TSC, but for now it's squishy concept. For now I'll make the text more specific. We can update the wording when/if we have an official group we can name.

[] We see the feature being used successfully in the wild and not generating a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, then that needs to be defined.

lot of community support issues that could indicate a problem.

### STAGE-2
`STAGE-2` features are in the last stages of becoming stable. Changes are
unlikely, but backward incompatible changes are still allowed in extreme cases.
Users can be reasonably sure that the feature won't change, but can't expect all
implementations to support the feature yet.

`STAGE-2` is a staging area for features that are ready to be declared stable,
but are waiting for the next release or don't yet have enough implementations.
When we see the vast majority of actively maintained implementations support
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved
this feature, it will be promoted to stable in the next release. Implementers
SHOULD support `STAGE-2` features so they aren't out-of-date when the next
release happens.

Being in `STAGE-2` isn't a guarantee that a feature will be declared stable in
the next release. If we don't see that the vast majority of actively maintained
implementation have implemented the feature, it may stay in `STAGE-2` another
year to allow implementations to catch up.

`STAGE-2` features can only be promoted to stable as part of a release.

**Promotion Checklist:**
[] The vast majority of actively maintained implementations support the feature
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we need to define...
"vast majority"
"activitly maintained"
"support the feature"

As an aside, I'm concerned thinking about "what if implementers don't implement?" But I think we should investigate further sponsoring implementation work at some point, which could aid this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"vast majority"

see #1348 (comment)

"actively maintained"

Yep. The more I think about it, the more I agree that this needs more definition. For example, an implementation may be active for bug fixes, but don't intent to add new features. Should that count? This probably needs discussion and probably doesn't need to be decided now as it will be a while before we have to make this determination.

"support the feature"

The meaning of those words seem pretty clear to me. We could explicitly say it has to pass all the tests. Is there more nuance that I'm missing?

I think we should investigate further sponsoring implementation work at some point

Sure. I think that speaks to increasing the pool of "actively maintained" implementations.


### Stable
Any part of the specification that doesn't have stability flag or has a year
stability flag such as `2024` is considered stable. This flag indicates the year
a feature was declared stable. Any changes to stable features MUST be backward
compatible and not introduce problems for forward compatibility. Stable features
are never removed, but may be deprecated.

### Deprecated
Any feature in the specification that is flagged in the form `DEPRECATED-{YEAR}`
is considered deprecated as of the year specified. Implementations SHOULD still
support these features to maintain backward compatibility. Deprecated features
will never be removed from the spec, but schema authors SHOULD migrate away from
using them as implementations may begin dropping support for these features at
some point.

A feature can only be deprecated as part of a release.

## Release Process
Releases occur on Jan 1 every year. Each release is identified with the
four-digit year of when the release occurred. Releases never introduce anything
new. The only change should be updating `STAGE-2` flags to `{YEAR}` flags for
the features that are promoted to stable that year, or adding adding deprecation
flags for any features that are deprecated that year.

Implementations that express support for a particular release MUST support all
features that are stable as of that release.

A snapshot of the spec will be taken of the stable parts of the spec and made
available as a PDF on the website at.
gregsdennis marked this conversation as resolved.
Show resolved Hide resolved

## Meta-Schemas
Meta-schemas associated with the specification(s) maintained under this process
are subject to the same compatibility rules as the specification. They can be
updated at any time, but changes MUST be backward compatible and allow for
forward compatibility with future schemas.
Comment on lines +169 to +172
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some questions/ comments here.

Are we going to maintain meta-schemas attached to individual release snapshots?
Are we going to have a "current" meta-schema which contains all keywords, even stage-0?
Are we going to have a "current" meta-schema which per stage? For example, all stage-1 and above or all stage-2 and above?

If we have a "current" meta-schema, which includes all of the stages, how can we update meta-schemas yet maintain backwards compatiblity?
This becomes a very specific problem if we disallow unknown keywords by default.

What if we wanted to update the meta-schemas to use a new keyword? That wouldn't be backwards compatible.

I think backwards compatible meta-schemas only makes sense for a given release.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just like the spec, there is one set of mutable meta-schemas that describe the current state of the specification. Because of backward and forward compatibility constraints, no matter what release of the spec the schema is written for, the latest meta-schemas is all you need.

Are we going to maintain meta-schemas attached to individual release snapshots?

We could have snapshots for reference like with the spec, but there wouldn't be reason to actually use those snapshots for anything other than historical reference.

Are we going to have a "current" meta-schema which contains all keywords, even stage-0?
Are we going to have a "current" meta-schema which per stage? For example, all stage-1 and above or all stage-2 and above?

The meta-schemas should include anything that's in the spec. That includes STAGE-1, STAGE-2, and deprecated.

If we have a "current" meta-schema, which includes all of the stages, how can we update meta-schemas yet maintain backwards compatiblity?

Meta-schema changes should be backward compatible because spec changes are backward compatible.

This becomes a very specific problem if we disallow unknown keywords by default.

What if we wanted to update the meta-schemas to use a new keyword? That wouldn't be backwards compatible.

I was thinking that the standard dialect meta-schema would allow additional keywords. If a keyword appears in the meta-schema but is not supported by the implementation, the implementation should raise an error. It's not something the meta-schema can know, so I consider it a runtime constraint the implementation needs to deal with, not the meta-schema.

However, meta-schemas could disallow additional keywords and it shouldn't hurt anything. If an implementation is using an older version of the meta-schema that doesn't have a newer keyword used by the schema, then the implementation also wouldn't have support for the keyword. The result is always that you get an error that the implementation doesn't know the keyword.

Either way, there shouldn't be a backward or forward compatibility issue and whether we choose to allow additional keywords in the meta-schema is not relevant to process, so we can discuss that when we get to writing the meta-schemas.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect there to be some discussion about meta-schema lifecycle that may end up as a bigger conversation. This was the kind of thing I thought might be best to get into after the initial merge. Like I said, I don't expect merging this document to mean it's "done". I expect it to be a first draft that we will iterate on.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a number of assertions in your response which I don't agree with, but I don't think here's the right place to capture this discussion.

I expect there to be some discussion about meta-schema lifecycle that may end up as a bigger conversation. This was the kind of thing I thought might be best to get into after the initial merge. Like I said, I don't expect merging this document to mean it's "done". I expect it to be a first draft that we will iterate on.

I think we need to discuss how we can break up the SDLC into more discreet chunks for further discussion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's exactly why I proposed merging this as a work-in-progress and opening issues to discuss the parts that need more discussion. Holding this up until we decide every detail makes it more difficult to have those discussions.


Meta-schema URIs SHOULD not change once published, but if they do, the old URI
jdesrosiers marked this conversation as resolved.
Show resolved Hide resolved
MUST remain available and redirect to the new URI.

Meta-schemas are considered non-normative. It's sometimes not possible or not
practical for the meta-schema to describe every constraint in the specification.