Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Looking at the future json-schema drafts #3

Closed
sorinsarca opened this issue Apr 2, 2018 · 1 comment
Closed

Looking at the future json-schema drafts #3

sorinsarca opened this issue Apr 2, 2018 · 1 comment

Comments

@sorinsarca
Copy link
Member

Well, I have to say that I'm somehow disappointed. draft-08's focus is schema reuse
and I don't see any improvement. Here are some proposals, which will "help" reusability

Ok, now lets take it one by one, but first I'll define the reused schema:

{
    "$id": "user",
    "type": "object",
    "properties": {
        "name": {"type": "string"}
    },
    "required": ["name"],
    "additionalProperties": false
}

$merge & $patch

Extending it with $merge

{
    "$id": "extended-user",
    "$merge": {
        "source": {"$ref": "user"},
        "with": {
            "properties": {
                "age": {"type": "integer"}
            },
            "required": ["age"]
        }
    }
}

The result is

{
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"}
    },
    "required": ["age"],
    "additionalProperties": false
}

Yes, the name is no longer required because was overwritten by extended-user's $merge.
So it doesn't work how you expect. Wait! I could use as an allOf and put there the required.
And if the user also have an allOf? You guessed, the same problem, so it won't work.

Don't dispare, it's not all lost. Here comes $patch:

{
    "$id": "extended-user",
    "$patch": {
        "source": {"$ref": "user"},
        "with": [
            {"op": "add", "path": "/properties/age", "value": {
                "type": "integer"
            }},
            {"op": "add", "path": "/required", "value": "age"}
        ]
    }
}

Now, isn't that readable?

If you still think that it is readable, try adding three more properties to extended-user.
For a better taste just use a pinch of remove operation.
Now just do the same in other schemas and after two months you'll loose your time
trying to understand what that schema really does.
Probably you'll end up moving the properties from user to extended-user just because it is easier to read and debug.

$spread

Extending it with $spread

{
    "$id": "extended-user",
    "properties": {
        "age": {"type": "integer"},
        "$spread": [
            {"$ref": "user"}
        ]
    },
    "required": ["age"]
}

The result is

{
    "type": "object",
    "properties": {
        "age": {"type": "integer"},
        "name": {"type": "string"}
    },
    "required": ["age"]
}

Ok, in order to also make name required you have to manually add it.
Also you have to manually add additionalProperties. But what if user also have an allOf?

Yes, you guessed, it won't work.

unevaluatedProperties

This one is a little bit trickier. You can use it in conjunction with allOf, anyOf, oneOf.
The ideea is that unevaluatedProperties knows what properties from ***Of were checked.

Extending it with unevaluatedProperties

{
    "$id": "extended-user",
    "properties": {
        "age": {"type": "integer"}
    },
    "required": ["age"],
    "allOf": [
        {"$ref": "user"}
    ],
    "unevaluatedProperties": false
}

Well, it will not work because user has additionalProperties set to false.

Anyway, the working design for unevaluatedProperties is not in our case.
It cannot be used for extending, only for ignoring this or that but in a limited way, because it will produce unexpected behaviour eventually.

Here it is an example of usage (we don't care about user schema for now)

{
    "$id": "hip-hop",
    "properties": {
        "common": {"type": "string"}
    },
    "required": ["common"],
    "anyOf": [
        {
            "properties": {
                "foo": {"type": "integer"}
            },
            "required": ["foo"]
        },
        {
            "properties": {
                "bar": {"type": "string"},
                "baz": {"type": "number"}
            },
            "required": ["bar"]
        }
    ],
    "unevaluatedProperties": false
}

Examples of data

1 - valid (common & first item of anyOf)

{
    "common": "this is common and required",
    "foo": 1
}

2 - valid (common & second item of anyOf)

{
    "common": "this is common and required",
    "bar": "bar value",
    "baz": 1
}

3 - valid (common & second item of anyOf, baz is not required)

{
    "common": "this is common and required",
    "bar": "bar value"
}

4 - invalid (baz is not present in first item of anyOf and bar is not present in the second one)

{
    "common": "this is common and required",
    "foo": 1,
    "baz": 1
}

5 - ???

{
    "common": "this is common and required",
    "foo": 1,
    "bar": "bar value",
    "baz": 1
}

Well, in theory this should be valid. unevaluatedProperties must know what properties were checked.
But, if an implementation of json-schema decides to do some optimizations, this will not work anymore. In case of anyOf once you've found a valid schema it will not make sense to check remaining schemas.
So, if json-schema doesn't allow optimizations by design it means that apps using it will spend most of the time checking things that don't make sense checking.
And if this is the case, you better start typing if-else and forget about json-schema.

Epilogue

Maybe you saw, but we already added a new keyword $map (besides $filters and $vars)
for opis/json-schema to allow a simple extending.

$map solves the following problem:

Given a schema and an object, map the properties of the object to match the schema properties.

So, if the schema is:

{
    "$id": "example",
    "type": "object",
    "properties": {
        "foo": {"type": "string"},
        "bar": {"type": "number"}
    }
}

and the current object is

{
    "a": "value for foo",
    "b": 123
}

you can validate it using $ref and $map

{
    "$ref": "example",
    "$map": {
        "foo": {"$ref": "0/a"},
        "bar": {"$ref": "0/b"}
    }
}

Please note that inside $map (and $vars) the $ref property is a (relative) json pointer for
current object.

Here is how we can extend the user

{
    "$id": "extended-user",
    "type": "object",
    "properties": {
        "age": {"type": "integer"}
    },
    "required": ["age"],
    "allOf": [
        {
            "$ref": "user",
            "$map": {
                "name": {"$ref": "0/name"}
            }
        }
    ]
}

You can even have other property for name in your extended schema

{
    "$id": "extended-user",
    "type": "object",
    "properties": {
        "full-name": {"type": "string"},
        "age": {"type": "integer"}
    },
    "required": ["full-name", "age"],
    "allOf": [
        {
            "$ref": "user",
            "$map": {
                "name": {"$ref": "0/full-name"}
            }
        }
    ]
}

I know that this example is too simple, and it doesn't make sense to just validate again
if name is a string since you already checked full-name, but what if user schema
also contains an allOf? What if the constraints for name are more complex?

Here is a more complex example (using two base schemas for our extended-user schema)

{
    "$id": "user",
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "active": {"type": "boolean"},
        "required": ["name", "active"]
    },
    "allOf": [
        ... other checks for user
    ],
    "additionalProperties": false
}
{
    "$id": "user-permissions",
    "type": "object",
    "properties": {
        "realm": {
            "type": "string"
        },
        "permissions": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "enabled": {"type": "boolean"}
                },
                "required": ["name", "enabled"],
                "additionalProperties": false,
                "allOf": [
                    ... other checks for permission
                ]
            }
        }
    },
    "required": ["realm", "permissions"],
    "additionalProperties": false
}

Our extended user

{
    "$id": "extended-user",
    "type": "object",
    "properties": {
        "first-name": {"type": "string"},
        "last-name": {"type": "string"},
        "is-admin": {"type": "boolean"},
        "admin-permissions": {
            "type": "array",
            "items": {
                "enum": ["create", "read", "update", "delete"]
            }
        },
    },
    "required": ["first-name", "last-name", "is-admin", "admin-permissions"],
    "additionalProperties": false,
    "allOf": [
        {
            "$ref": "user",
            "$map": {
                "name": {"$ref": "0/last-name"},
                "active": true
            }
        },
        {
            "$ref": "user-permissions",
            "$map": {
                "realm": "administration",
                "permissions": {
                    "$ref": "0/admin-permissions",
                    "$each": {
                        "name": {"$ref": "0"},
                        "enabled": {"$ref": "2/is-admin"}
                    }
                }
            }
        }
    ]
}

So if the data for extended-user schema is

{
    "first-name": "Json-Schema",
    "last-name": "Opis",
    "is-admin": true,
    "admin-permissions": ["create", "delete"]
}

the mapped data provided to user schema (first item of allOf) will be

{
    "name": "Opis",
    "active": true
}

and the mapped data provided to user-permissions schema (second item of allOf) will be

{
    "realm": "administration",
    "permissions": [
        {
            "name": "create",
            "enabled": true
        },
        {
            "name": "delete",
            "enabled": true
        }
    ]
}

As you can see, with $map you can add only what properties you want, you can handle nested properties, you can provide default values, and you can even use $each to map arrays.

The advantage is that I can change extended-user schema however I want without touching user and user-permissions schemas.

And for validation this is verbose, clear and flexible.

Anyway, another method you can use to simplify your schemas is using $ref together with $vars.

Here is an example where $vars is handy

{
    "$id": "settings-type",
    "definitions": {
        "type-A": {
            ...
        },
        "type-B": {
            ...
        },
        ...
    }
}
{
    "type": "object",
    "properties": {
        "type": {"enum": ["A", "B", ...]},
        "settings": {
            "$ref": "settings-type#/definitions/type-{typeName}",
            "$vars": {
                "typeName": {"$ref", "1/type"}
            }
        }
    }
    "required": ["type", "settings"]
}

Without $vars you'll probably need an anyOf or oneOf which will be very slow.
But in this way you can add as many definitions as you want to settings-type schema,
or even better (and recommended), you can use different schema files for each type
and load only needed schemas, because it doesn't make sanse to load and check things that
will never change the final result.

@adjenks
Copy link

adjenks commented Mar 24, 2020

You may want to consider adding a feature to output the json-schema standard output format when the time is right:
http://json-schema.org/draft/2019-09/json-schema-core.html#rfc.section.10
I know it's a fairly new draft, but I'm looking forward to seeing it implemented in various places.
Looks like it was merged Dec 2018: json-schema-org/json-schema-spec#679

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants