Versioning

[larshp]

Change is constant, so over time, some fields in the JSON schemas might be moved/removed/added

Any considerations regarding versioning? eg. does a JSON file follow the old or new schema of an object

in abapGit we add https://github.com/abapGit-tests/VIEW/blob/main/src/zag_unit_testv.view.xml#L2 so that its possible for the code reading the file can make assumptions regarding the content

[albertmink]

We are aware of the versioning and we actually plan to provide the information via the JSON schema. For example

{
    "$schema": "https://raw.githubusercontent.com/SAP/abap-file-formats/v1.0/file-formats/clas/clas.json",
    "header": {
        "description": "Example class for ABAP File Formats",
        "masterLanguage": "EN",
        "abapLanguageVersion": "standardUnicode"
    }
}

For a release branch/tag v1.0 and a modified header in the JSON schema.

[larshp]

👍 I have something similar setup at https://github.com/abaplint/schema.abaplint.org

I think each object should be versioned independently, eg. if there are changes to INTF and not CLAS, I would expect CLAS to keep the old version number. Or we might end up in a situation where the schema number changes more often in the files than the actual content

[albertmink]

Hi Lars,

good point. So we most likely will adopt the versioning to

{
    "$schema": "https://raw.githubusercontent.com/SAP/abap-file-formats/main/file-formats/clas/clas.json",
    "header": {
        "description": "Example class for ABAP File Formats",
        "masterLanguage": "EN",
        "abapLanguageVersion": "standardUnicode"
    }
}

with kind of rolling tip to main branch as the default schema. Non up-to-date schema will be maintained on main branch, however indicated with a number, say clas-v.0.0.0.json, clas-v.0.0.1.json and others.

Note: Branching or tagging as mentionen in #14 does not apply to this type of versioning.

[schneidermic0]

I agree. The version of a schema should be schema-specific. Otherwise, the versions will heavily increase and we have to update the version number for object types that have not been changed at all.

@albertmink I am not sure whether we should need semantic versioning for each resource. E.g., ADT has just a version counter per resource that is increased whenever it is necessary due to incompatible changes. Compatible changes are not reflected in the version number there.

[larshp]

Possible Solutions:
A: version as part of serialized filename
B: version field in the JSON, and a conditional "super" schema

[larshp]

Simple conditional versioning, see https://github.com/larshp/json-versioning

Each version can be a definition, and then some ifs to choose the schema...

$schema is added in the json file for testing purposes, it should not be part of the serialized files

[larshp]

when #109 is done, we can add a small script that builds the super schema and pushes to each PR

[schneidermic0]

Just some notes related to versioning:

1. For plain text files (like .abap files), I think we don't need a versioning of the content, because we have just to convert the plain text
2. For structured files (like .json files), the file format might evolve over time and we need an indicator in which version the content was serialised/can be deserialised. We discussed following options to indicate such versions
   (a) Using field $schema similar to the approach to specify the version in JSON schema files. This would be the URI to the schema file (e.g, https://github.com/SAP/abap-file-formats/blob/main/file-formats/intf/intf-v1.json
   (b) Using field $version and indicate a version number (e.g., as simple integer).
   (c) Specify the version in the file name (e.g., if_my_interface.intf.v1.json).

I personally, don't prefer approach (c), because it leads to changes in the file name when the format version changes.

I am not aware of a standard way to indicate format versions in JSON files. If anybody knows a standard, please share it here. ;)

Approaches $schema (a) and $version (b) are basically similar. I am wondering why we can't use the same approach as JSON schemas. The URI could be used by tools as convenience to load the correct schema from the specified location. I see only following disadvantages compared to $version approach:

1. A full URI is harder to compare/analyze than a simple integer.
   Maybe, we could specify that only a specific part is important to specify the schema version (e.g., only intf-v1.json indicates the version in the URI https://github.com/SAP/abap-file-formats/blob/main/file-formats/intf/**intf-v1.json**) to make the version identifier stable.
2. The URI might change over time.
   We could check whether we would like to use a more stable location (e.g. storing the schemas at https://api.sap.com, https://www.schemastore.org/json/ or any other location.

[schneidermic0]

I have just seen that https://www.schemastore.org/json/ refers in some cases to schemas stored in GitHub repositories.

Example For Azure Pipelines the schema is stored at https://raw.githubusercontent.com/microsoft/azure-pipelines-vscode/v1.174.2/service-schema.json) and the schema specifies "$id": "https://github.com/Microsoft/azure-pipelines-vscode/blob/main/service-schema.json"

[albertmink]

Regarding tooling in VS Code:
If we decide to provide the schemas also via schemastore, then we could define an instance-schema matcher, see https://www.schemastore.org/api/json/catalog.json keyword fileMatch. This means, VS Code does automatically recognize the suitable JSON schema for validation/annotation and the keyword $schema could be optional.
Technically, this why the package.json as "NPM configuration file" and many other JSON based configurations are matched automatically.

[albertmink]

We decided to drop $schema from the JSON data, as suggested in #83.
At the same time we introduce a formatVersion field to account for versioning, see #231 and we adopt the JSON schema file name to reflect the version.

With this change, the data is independent of (possible) URI changes and the consumer has to link the data to the desired JSON schema.

[albertmink]

Technical question: Should the formatVersion rather be const than enum as by now?

Quote from JSON Schema: The const keyword is used to restrict a value to a single value and is new in draft07, see http://json-schema.org/understanding-json-schema/reference/generic.html#constant-values

We could simply replace the JSON schema line

abap-file-formats/file-formats/intf/intf-v1.json

Line 12 in 4798eb4

"enum": [ "1" ]

by

"const": "1"

Data has not to be adjusted.

[schneidermic0]

I think we can start with const, because one schema shall describe one version.

If we want to have a schema valid for multiple versions in the future, we could change it to enum. I would expect this as a compatible change to the schema.

[larshp]

the deprecation keyword could be used for the user to discover that there is a new version

but currently not implemented in vscode, microsoft/vscode-json-languageservice#87