Validate and infer JSON Schema validated data type: introducing type providers

[toomuchdesign]

What version of Ajv you are you using?
8

What problem do you want to solve?
Using ajv to validate and infer JSON Schema validated data type at the same time. Eg:

import Ajv from 'ajv'

const ajv = new Ajv()

const schema = {
  type: 'object',
  properties: {
    foo: { type: 'integer' },
    bar: { type: 'string' }
  },
  required: ['foo'],
  additionalProperties: false
} as const
const validate = ajv.compile(schema)
let data: unknown = { foo: 6 }

if (validate(data)) {
  // data type inferred from schema
  console.log('Validation ok', data)
} else {
  // validate is the usual AJV validate function
  console.log('Validation ko', validate.errors)
}

The point here is inferring schema types (with json-schema-to-ts) instead of separately providing them.

What do you think is the correct solution to problem?

There are several possible solutions but, as far as I know, json-schema-to-ts is the only existing library currently being able to infer TS types from a JSON schema.

In order not to bind ajv with json-schema-to-ts (as a npm dependency), I tried to replicate the type provider approach already taken in other libraries like Fastify.

I published a @toomuchdesign/ajv-type-provider-json-schema-to-ts package which is supposed to abstract the necessary bindings between ajv and json-schema-to-ts.

This is an example of the current api. Happy to review it based on your feedback:

import Ajv from 'ajv'
import { enhanceCompileWithTypeInference } from '@toomuchdesign/ajv-type-provider-json-schema-to-ts'

const ajv = new Ajv()
const compile = enhanceCompileWithTypeInference(ajv.compile.bind(ajv))

const schema = {
  type: 'object',
  properties: {
    foo: { type: 'integer' },
    bar: { type: 'string' }
  },
  required: ['foo'],
  additionalProperties: false
} as const

const validate = compile(schema)
let data: unknown = { foo: 6 }

if (validate(data)) {
  // data type inferred from schema
  console.log('Validation ok', data)
} else {
  // validate is the usual AJV validate function
  console.log('Validation ko', validate.errors)
}

If ajv maintainers decided to pick such approach, the type provider could be officially moved under ajv umbrella as an official extension.

ajv class might even expose a .withTypeProvider<TypeProvider>() method returning the class instance itself which does nothing but providing a type hook:

const ajv = new Ajv()
const typedAjv = ajv.withTypeProvider<JsonSchemaToTsProvider>()

// Use typedAjv as usual

Related issues:

• What needs to happen before we can get simultaneous TS inference and validation for JTD schemas? #1902
• [Q] Recommendations for Converting JSON Schema to TS Types in Ajv? #2339
• Verify is incorrectly typed for inline schemas #2360
• Add a method for generating JSON Schema from Ajv #1956
• Natively make ajv validators typeguards in TS (integrate with json-schema-to-ts) #2091
• Implement a JSONDataType to make typescript conversion on json schema #1792

Will you be able to implement it?

Yes.

• Source code: https://github.com/toomuchdesign/ajv-type-provider-json-schema-to-ts
• NPM package: https://www.npmjs.com/package/@toomuchdesign/ajv-type-provider-json-schema-to-ts

[jasoniangreen]

Hi @toomuchdesign, thanks for your contributions to the world of TS and JSON schema. I have only recently been added to the org, but I can share some thoughts based on my own experience and discussions with the author.

Firstly I am curious how json-schema-to-ts handles that fact that JSON Schema is MORE expressive than TypeScript? I have worked with systems that build json schema from types, which made sense to me as writing Types is easier, closer to the code and ultimately anything you can express in TS could be expressed as JSON Schema (ok, I'm not sure that's true, but for your usual API inputs/outputs). But I haven't done it the other way around before.

Secondly, from what I know about AJV and conversations I've had with EP, the general philosophy would be to leave tools like this as 3rd parties. Maintaining a library with such a huge and varied usage across the world is already a massive responsibility and every addition to that footprint, even if it is just the moving of a library under our umbrella, is a risk that has to be balanced with considerable benefits. It's easy enough for anyone to use your library as is without it being under the AJV umbrella.

[toomuchdesign]

Hi @jasoniangreen, thanks for sharing your thoughts!

Unfortunately I don't have a clear map of the existing gaps between JSON Schema, json-schema-to-ts and TS types. In the last few months I've been daily working with json-schema-to-ts inferring types from JSON Schemas created from real-world OpenAPI definitions and I can tell I haven't run into any major issue or shortcomings (it doesn't mean there aren't :) ).

Btw, json-schema-to-ts accepts a tiny set of options which can change the actual inferred type output.

I'd like to reiterate the main use case why I tried this type provider approach. I've found this thread started by json-schema-to-ts author where I had the impression that there was a basic misunderstanding among the participants.

This example comes straight from ajv documentation:

import Ajv, {JSONSchemaType} from "ajv"
const ajv = new Ajv()

interface MyData {
  foo: number
  bar?: string
}

const schema: JSONSchemaType<MyData> = {
  type: "object",
  properties: {
    foo: {type: "integer"},
    bar: {type: "string", nullable: true}
  },
  required: ["foo"],
  additionalProperties: false
}

// validate is a type guard for MyData - type is inferred from schema type
const validate = ajv.compile(schema)

MyData is a manually written TS type that user have to manually keep in sync with the actual JSON schema (schema) counterpart, in order for type inference to make sense. JSONSchemaType helps the user by raising a TS error in case schema and provided type don't match.

Consider the case where MyData is the return type of a big REST api returning tons of nested props.

Using a type provider allows getting rid of the manually written MyData type and promotes JSON Schema as the single source of truth for validation and type inference.

☝️ This is the main point that could be worth going into details. Deciding whether from ajv stand point inferring types of validated data brings value or not.

This sounds quite natural to me, since JSON schema and OpenAPI are meant to be an interchange formats between different realms. But other point of views are very welcome!

That said, I didn't mean to put any kind of pressure over ajv maintainers. This is actually the reason why I decided to publish this POC as a standalone package, externalising any implementation responsibility and allowing ajv not to depend on any specific type inference library. Other type providers could provide support for other libraries like zod, typebox or whatever.

Thanks for the amazing work done on this fundamental library!

[jasoniangreen]

Thanks for this informative post @toomuchdesign, yeah it is something I will definitely look more into. I can see what you mean by the misunderstanding of what JSONSchemaType is actually for, this is maybe something else to review to ensure docs are clear and maybe at the same time refer to external solutions for using JSON Schema to generate types.

I will leave this for a while in case it generates any other discussion, and as I said, when I get some time I definitely want to try it out along with json-schema-to-ts.