Add an meta property that identifies schema properties as deprecated

[cfraser]

What are the schemas that are affected by the issue

All schemas

Proposal

As our schemas change over time we will need a way to communicate that a property should no longer be used. For example, when a schema is being used to describe the schema in data storage the property should continue to be available so that data may be queried, but consumers and implementers should be alerted to the fact that the property should not longer be used to store data.

This requirement is similar to the schema level meta:status that was introduced in #161

[lrosenthol]

@cfraser this seems to fly in the face of our designs for XDM, where once a schema is defined as stable, it can't change. We will certainly have new versions and extensions, but the original remains.

[cfraser]

Thanks for the feedback @lrosenthol!

I believe this proposal is complimentary to #161 as it allows a schema to evolve over time while affording the ability to be backwards compatible. I expect this will be an important feature that extensions will leverage as they learn more about how their schemas are being used and need to make breaking changes to the schema while still supporting data that has already been ingested.

[kstreeter]

I feel like we need to allow our meta:status property to be placed both on the root schema, as well as placed on individual properties. We would of course have to document very specifically what the allowed combinations mean, but there are several cases where we probably need this:

• As Chris describes, we want to deprecate a single property in a stable schema. We can't remove it (as @lrosenthol mentioned that is the definition of 'stable') but we want to make it clear it shouldn't be used for new applications. We may also want to add some kind of meta:supercededBy tag for cases that a new property is intended to replace the old.

• We want to add a new, experimental property to an existing stable schema. This is actually ambiguous right now: if I add a new property to a stable schema, it automatically becomes stable, without ever going through the experimental->stabilizing->stable flow all the other properties went through.

My thoughts are that we should allow individual properties to be annotated with status after a schema becomes stable. For new properties, this would be temporary (once they are stable, too we remove the annotation) but for deprecated fields they would remain until the whole schema is deprecated.

[lrosenthol]

@kstreeter I think we need a big discussion on this as this was not the original design or plan for XDM and goes against some of the key philosophies that Oliver has been pushing on ACP

[trieloff]

For the record: I think both of @kstreeter's scenarios are sensible, but we should see deprecation always as a weapon of last resort. Therefore I'd suggest we leave the meta:status=deprecated option for properties until we have the concrete need to deprecate a property (which will definitely cause a big discussion – including the question "why didn't we do it right in the first place")

I'd still introduce it for the meta:status=experimental, so that we can flag new, in-progress additions to stable schemas.

[reivaxy]

I support the need for "meta:status": "deprecated" on fields (not sure about other status values, didn't find the wiki page listing them).

Use case:
My favorite PM wants a field to be renamed. If I just change it in XDM, it will break ACS where it is used, until ACS is updated, released and deployed. If I just add a field with the new name, the old one will keep showing up in a field selection UI, where it should no longer be used.
The field name is a user-facing piece of data (technical user), we can't just change its title.

I assume this could be refused for schema stability reason, which is just an Adobe internal decision. But what about extension that customers make ? They may not want or may not be able to enforce such stability rules, and may need to mark fields obsolete.

As a side note, I'm already seeing fields bearing the "meta:status" with value "deprecated" in the "person" schema.

[cfraser]

@kstreeter Given that we've taken this approach with the recent changes in #498 can we close this discussion and document that this is the approved approach?

[lrosenthol]

@cfraser Yes, since we are now doing this with #498 - I would agree that we have (defacto) moved in this direction - though only with deprecated. No other values are defined for status at this time.

[cfraser]

We have defacto adopted this proposal.