Revision 0.30.0

[sinclairzx81]

0.30.0

Overview

Revision 0.30.0 is a milestone revision for the TypeBox project. It is primarily focused on internal optimizations, refactoring work to reduce package and bundle sizes, enable increased modularity of internal sub modules (with some considerations given to future ESM publishing), renaming internal functions to address react native bundling issues and consolidating shared internal modules to reduce overall code overhead.

This revision also implements several new features, including new validation constraints for Array, new types for iterators, new utility types, a TypeScript code generation option for the compiler, enhancements made to modifiers and better options for TypeScript to TypeBox code translation. This revision also includes new examples including a transform type for handling IO encode an decode as well as a reference implementation for JSON Type Definition specification.

This revision includes breaking changes and some deprecations. It requires a minor semver revision.

Contents

• Enhancements
  • TypeScript Code Generation
  • Optional and Readonly
  • Iterator and AsyncIterator
  • Order Independent References
  • Value Submodules
  • Array Contains Constraint
  • Additional Utility Types
  • Reduced Package Size
  • TypeBox Codegen
• Examples
  • JSON Type Definition
  • Prototype Types
  • Transform Types
• Breaking
  • Extended Type Representation-Change
  • Modifier Symbol Deprecated
  • RegEx Renamed To RegExp
  • ValueErrorType Custom Renamed To Kind

TypeScript Code Generation

Revision 0.30.0 adds TypeScript code generation support to the TypeCompiler. By specifying the language option on the .Code() function, TypeBox will add type annotations to the compiled output. This functionality can be used to produce typed TS functions for projects that preference AOT compilation.

const Code = TypeCompiler.Code(Type.String(), {     // return function check(value: any): boolean {
  language: 'typescript'                            //   return (
})                                                  //     (typeof value === 'string')
                                                    //   )
                                                    // }

Optional and Readonly

Revision 0.30.0 deprecates the [Modifier] symbol and introduces two new symbols, [Readonly] and [Optional]. This change is carried out to simplify type inference as well as to simplify runtime mapping logic. This change should not implicate users leveraging the Type.* purely for type composition, however implementors using TypeBox for reflection and code generation should update to the new symbols.

// Revision 0.29.0
//
const A = Type.ReadonlyOptional(Type.Number())     // const A: TReadonlyOptional<TNumber> = {
                                                   //   type: 'number',
                                                   //   [TypeBox.Modifier]: 'ReadonlyOptional'
                                                   // }

const B = Type.Readonly(Type.Number())             // const B: TReadonly<TNumber> = {
                                                   //   type: 'number',
                                                   //   [TypeBox.Modifier]: 'Readonly'
                                                   // }

const C = Type.Optional(Type.Number())             // const C: TOptional<TNumber> = {
                                                   //   type: 'number',
                                                   //   [TypeBox.Modifier]: 'Optional'
                                                   // }

// Revision 0.30.0
//
const A = Type.ReadonlyOptional(Type.Number())     // const A: TReadonly<TOptional<TNumber>> = {
                                                   //   type: 'number',
                                                   //   [TypeBox.Readonly]: 'Readonly',
                                                   //   [TypeBox.Optional]: 'Optional'
                                                   // }

const B = Type.Readonly(Type.Number())             // const B: TReadonly<TNumber> = {
                                                   //   type: 'number',
                                                   //   [TypeBox.Readonly]: 'Readonly'
                                                   // }

const C = Type.Optional(Type.Number())             // const C: TOptional<TNumber> = {
                                                   //   type: 'number',
                                                   //   [TypeBox.Optional]: 'Optional'
                                                   // }

Iterator and AsyncIterator

Revision 0.30.0 adds the types Iterator and AsyncIterator. These types add to the existing non-validatable extended type set and can be used build callable generator functions. These types are written primarily to describe RPC network interfaces that return multiple values. Examples of which may include web socket streams or reading database result cursors over a network.

// Revision 0.30.0
//
const Enumerable = <T extends TSchema>(T: T) => Type.Function([
  Type.Number({ description: 'Start index' }),
  Type.Number({ description: 'End index' })
], Type.Iterator(T))

const EnumerableNumber = Enumerable(Type.Number())

const Range: Static<typeof EnumerableNumber> = function * (start: number, end: number) {
  for(let i = start; i < end; i++) yield i
}

const R = [...Range(10, 20)]                 // const R = [10, 11, 12, ..., 19]

Order Independent References

Revision 0.30.0 adds an overload for Ref to enable non order dependent type referencing. Prior to this revision, reference targets needed to be defined first before being referenced. Revision 0.30.0 lifts this restriction and allows referencing of "yet to be defined" targets through the use of typeof operator. This overload borrows on TypeScript's ability to derive type information irrespective of topological ordering.

This overload is implemented for "TypeScript to TypeBox" code generation utilities where TypeScript types are not guaranteed ordered in a runtime sorted fashion.

// Revision 0.29.0
//
const R = Type.Ref(T)                              // Error: T isn't defined yet

const T = Type.Object({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number()
}, { $id: 'T' })

// Revision 0.30.0
//
const R = Type.Ref<typeof T>('T')                  // Ok: infer from typeof T

const T = Type.Object({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number()
}, { $id: 'T' })

Value Submodules

Revision 0.30.0 carries out a number of refactorings for the Value.* modules to enable each submodule to be imported individually. These refactorings are support better "pay to play" library characteristics, allowing users to import only the submodules they need. This update also makes provisions for ESM publishing by removing internal namespaces.

The top level Value.* namespace will remain on all subsequent versions of TypeBox.

// Revision 0.29.0
//
import { Value } from '@sinclair/typebox/value'           // Value.* namespace

const A = Value.Create(Type.String())              

// Revision 0.30.0
//
import { Create } from '@sinclair/typebox/value/create'   // Only Create()

const A = Create(Type.String())                  

Array Contains Constraint

Revision 0.30.0 implements validation support for the contains keyword as well as the draft 2019-09 minContains and maxContains constraints on Array. Documentation on these constraints can be found https://json-schema.org/understanding-json-schema/reference/array.html#contains

// Revision 0.30.0
//
const T = Type.Array(Type.Number(), {
  contains: Type.Literal(1),
  minContains: 3,
  maxContains: 5
})

Value.Check(T, [1, 1, 1])                          // true - between 3 and 5 instances of 1
Value.Check(T, [1, 1, 1, 1, 1])                    // true - between 3 and 5 instances of 1
Value.Check(T, [0, 1, 1, 1, 1, 1])                 // true - between 3 and 5 instances of 1
Value.Check(T, [1, 1])                             // false - less than 3 instances of 1
Value.Check(T, [1, 1, 1, 1, 1, 1])                 // false - more than 5 instances of 1
Value.Check(T, [0])                                // false - no instances of 1

Additional Utility Types

Revision 0.30.0 adds the utility types Awaited, Uppercase, Lowercase, Capitalize, and Uncapitalize to the supported type set.

// Revision 0.30.0
const T1 = Type.Awaited(Type.Promise(Type.String()))  // const T1: TString

const T2 = Type.Uppercase(Type.Literal('hello'))      // const T2: TLiteral<'HELLO'>

const T3 = Type.Lowercase(Type.Literal('HELLO'))      // const T3: TLiteral<'hello'>

const T4 = Type.Capitalize(Type.Literal('hello'))     // const T4: TLiteral<'Hello'>

const T5 = Type.Uncapitalize(Type.Literal('HELLO'))   // const T5: TLiteral<'hELLO'>

A full list of TypeScript utility types can be found at this link.

Reduced Package Size

Revision 0.30.0 carries out several internal refactorings to reduce package and bundle sizes. This work is largely an ongoing process with provisional work carried out across type, value and compiler modules. Revision 0.30.0 manages to weigh in slightly less than Revision 0.29.0 with the additional functionality provided on the revision.

// Revision 0.29.0
//
┌──────────────────────┬────────────┬────────────┬─────────────┐
│       (index)        │  Compiled  │  Minified  │ Compression │
├──────────────────────┼────────────┼────────────┼─────────────┤
│ typebox/compiler     │ '130.3 kb' │ ' 58.2 kb' │  '2.24 x'   │
│ typebox/errors       │ '113.3 kb' │ ' 49.8 kb' │  '2.27 x'   │
│ typebox/system       │ ' 78.8 kb' │ ' 32.2 kb' │  '2.45 x'   │
│ typebox/value        │ '180.0 kb' │ ' 77.7 kb' │  '2.32 x'   │
│ typebox              │ ' 77.7 kb' │ ' 31.7 kb' │  '2.45 x'   │
└──────────────────────┴────────────┴────────────┴─────────────┘

// Revision 0.30.0
//
┌──────────────────────┬────────────┬────────────┬─────────────┐
│       (index)        │  Compiled  │  Minified  │ Compression │
├──────────────────────┼────────────┼────────────┼─────────────┤
│ typebox/compiler     │ '129.4 kb' │ ' 58.6 kb' │  '2.21 x'   │
│ typebox/errors       │ '111.6 kb' │ ' 50.1 kb' │  '2.23 x'   │
│ typebox/system       │ ' 76.5 kb' │ ' 31.7 kb' │  '2.41 x'   │
│ typebox/value        │ '180.7 kb' │ ' 79.3 kb' │  '2.28 x'   │
│ typebox              │ ' 75.4 kb' │ ' 31.3 kb' │  '2.41 x'   │
└──────────────────────┴────────────┴────────────┴─────────────┘

TypeBox Codegen

Revision 0.30.0 offers an external code generation API tool which can be used to programmatically convert TypeScript types into TypeBox types.

TypeBox-Code Project

import * as Codegen from '@sinclair/typebox-codegen'

const Code = Codegen.TypeScriptToTypeBox.Generate(`
  type T = { x: number, y: number, z: number }
`)

console.log(Code)

// Output:
//
// import { Type, Static } from '@sinclair/typebox'
// 
// type T = Static<typeof T>
// const T = Type.Object({
//   x: Type.Number(),
//   y: Type.Number(),
//   z: Type.Number()
// })

JSON Type Definition

Revision 0.30.0 includes a reference implementation for JSON Type Definition (RFC 8927). This specification is currently under consideration for inclusion in the TypeBox library as an alternative schema representation for nominal type systems. The implementation currently contains all types expressed in the JSON Type Definition spec, but omits constraints such and minimum and maximum values (which are not formally represented in the specification).

The implementation is offered as a single file which can be copied in to projects with TypeBox installed. This implementation may be enhanced over the next few revisions (with some potential to implement mapping types such as partial, required, omit, pick, keyof). This specification will be considered for inclusion under @sinclair/typebox/typedef if there is enough interest.

import { Type } from './typedef'                   // from: examples/typedef/typedef.ts

const T3 = Type.Struct({                           // const T3 = {
  x: Type.Float32(),                               //   properties: {
  y: Type.Float32(),                               //     x: { type: 'float32' },
  z: Type.Float32()                                //     y: { type: 'float32' },
})                                                 //     z: { type: 'float32' }
                                                   //   }
                                                   // }

const T2 = Type.Struct({                           // const T3 = {
  x: Type.Float32(),                               //   properties: {    
  y: Type.Float32()                                //     x: { type: 'float32' },
})                                                 //     y: { type: 'float32' }
                                                   //   }
                                                   // }

const U = Type.Union([                             // const U = {
  T3,                                              //   discriminator: 'type',
  T2                                               //   mapping: {
])                                                 //     0: {
                                                   //       properties: {
                                                   //         x: { type: 'float32' },
                                                   //         y: { type: 'float32' },
                                                   //         z: { type: 'float32' }
                                                   //       }
                                                   //     },
                                                   //     1: {
                                                   //       properties: {
                                                   //         x: { type: 'float32' },
                                                   //         y: { type: 'float32' }
                                                   //       }
                                                   //     }
                                                   //   }
                                                   // }

Prototype Types

Revision 0.30.0 renames Experimental types to Prototype types within the examples directory. Updates here include additional documentation and rationales for the existing types UnionOneOf, UnionEnum, Const, and includes two new types Evaluate and PartialDeep. These types are written as standalone modules and can be copied into a project for direct use. The TypeBox project is open to community discussions around the inclusion of these types in future revisions.

Transform Types

Revision 0.30.0 provides a reference implementation for Transform types. There has been some interest from users to offer combinators similar to Zod's .transform() function that permits remapping values during .parse() like operations. As TypeBox types do not have fluent combinators or a parse function (and are just JSON Schema objects), introducing similar functionality without augmenting types or implementing a .parse() on all types has proven to be particularily challenging.

The reference Transform implementation implements a workable design by augmenting TypeBox types with codec functions outside the type system. These functions allow values to be structurally encoded and decoded through the .parseLike() functions Encode() and Decode(). TypeBox adopts the io-ts perspective for value transformation, viewing the act of transforming values primarily the role of dedicated codec system. As much of this functionality is considered high level and above and beyond the type system, Transform types will not likely be added to TypeBox type system; but rather added as an optional import in later revisions.

import { Transform, Encode, Decode } from './transform'

const Timestamp = Transform(Type.Number(), {       // The Transform function wraps a TypeBox type with two codec
  decode: (value) => new Date(value),              // functions which implement logic to decode a received value
  encode: (value) => value.getTime(),              // (i.e. number) into a application type (Date). The encode
})                                                 // function handles the reverse mapping.

type N = Static<typeof N>                          // type N = { timestamp: number }
                                                   //
const N = Type.Object({                            // Transform types are to be used like any other type and will
  timestamp: Timestamp                             // infer as the original TypeBox type. For example, the type `N` 
})                                                 // above will infer as { timestamp: number } (as derived from 
                                                   // the TB type)



const D = Decode(N, { timestamp: 123 })            // const D = { timestamp: Date(123) }
                                                   //
                                                   // The Decode function accepts any type plus a value. The Decode 
                                                   // function return type will be that of the transforms decode() 
                                                   // return type (which is Date), with the second argument statically
                                                   // typed as N. This function acts as a kind of parse() that returns 
                                                   // the decoded type or throws on validation error.
                                               

const E = Encode(N, { timestamp: new Date(123) })  // const E = { timestamp: 123 }
                                                   //
                                                   // The encode function performs the inverse, accepting the
                                                   // decoded type { timestamp: Date } and re-encoding to the
                                                   // target type { timestamp: number }. This function will
                                                   // also throw on validation error.

Extended Type Representation Change

Revision 0.30.0 updates representations for all extended types. This change is made due to TypeBox's observed role as a general purpose JavaScript validation library as well as to deprecate support for extended type validation in Ajv which was only ever partially functional at best.

Attempts were made on Revision 0.25.0 to restructure extended types to provide Ajv hooks for custom type configuration. These hooks used the type property where { type: 'object', instanceOf: 'Type' } was used to configure schematics for JavaScript objects, and { type: 'null', typeOf: 'Type' } was used for JavaScript primitives. Despite these hooks, Ajv would still struggle with validation of primitive types (such as undefined), and for the types Function, Constructor and Promise; these were meaningless to Ajv and it did not make sense to try provide hooks for a validator that could not make use of them.

This change represents a move towards a formal specification to express pure JavaScript constructs which is partially under discussion within the runtime type community. This change will implicate the use of Uint8Array and Date objects when configuring for Ajv. A supplimentary fallback will be provided in the /examples directory using Type.Unsafe

// Revision 0.29.0
//
const T = Type.Date()                              // const T: TDate = { type: 'object', instanceOf: 'Date' }

const U = Type.Undefined()                         // const U: TUndefined = { type: 'null', typeOf: 'Undefined' }

// Revision 0.30.0
//
const T = Type.Date()                              // const T: TDate = { type: 'Date' }

const U = Type.Undefined()                         // const U: TUndefined = { type: 'undefined' }

RegEx Renamed To RegExp

Revision 0.30.0 marks Type.RegEx as deprecated but provides Type.RegExp as an alternative (matching the JavaScript RegExp type name). Additionally this type has also been moved from the Standard to Extended type set. The RegExp type will no longer considered part of the Standard type set due to JavaScript Regular Expressions supporting a wider range of symbols and control characeters than is supported by the ECMA262 subset used by the JSON Schema specification. Information on the ECMA262 subset supported by JSON Schema can be found at the following Url https://json-schema.org/understanding-json-schema/reference/regular_expressions.html

As Type.RegEx() is widely used, this function will be retained under the @deprecated annotation for the 0.30.0 revision.

// Revision 0.29.0

const T = Type.RegEx(/abc/) 

// Revision 0.30.0

const A = Type.RegEx(/abc/)                       // deprecation warning!

const B = Type.RegExp(/abc/)                      // Extended Type

const T = Type.String({ pattern: /abc/.source })  // Standard Type

For Unicode (UTF-16) support on 0.30.0, the recommendation is to continue using user defined formats.

import { Type, FormatRegistry } from '@sinclair/typebox'

FormatRegistry.Set('emoji', value => /<a?:.+?:\d{18}>|\p{Extended_Pictographic}/gu.test(value))

const T = Type.String({ format: 'emoji' })

Value.Check(T, '♥️♦️♠️♣️')                         // Ok

For information on configuring custom formats on Ajv, refer to https://ajv.js.org/guide/formats.html#user-defined-formats