-
Notifications
You must be signed in to change notification settings - Fork 534
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Tree: Add Schema export and compare APIs (#22733)
## Description See changeset for details. A realistic detailed usage example can be found in https://github.com/microsoft/FluidFramework/pull/22566/files#diff-b893cfcd2daa74fb9dd1a8b2ce47006faaaefcb7b83671f1413f63faf65257af showing how an actual app may use these APIs. Assuming this lands before the next release, the changeset could get a proper link to that code on main added as an example.
- Loading branch information
1 parent
0689b0e
commit 920a65f
Showing
28 changed files
with
553 additions
and
36 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| "fluid-framework": minor | ||
| "@fluidframework/tree": minor | ||
| --- | ||
| --- | ||
| "section": tree | ||
| --- | ||
|
|
||
| Add alpha API for snapshotting Schema | ||
|
|
||
| `extractPersistedSchema` can now be used to extra a JSON compatible representation of the subset of a schema that gets stored in documents. | ||
| This can be used write tests which snapshot an applications schema. | ||
| Such tests can be used to detect schema changes which could would impact document compatibility, | ||
| and can be combined with the new `comparePersistedSchema` to measure what kind of compatibility impact the schema change has. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| /*! | ||
| * Copyright (c) Microsoft Corporation and contributors. All rights reserved. | ||
| * Licensed under the MIT License. | ||
| */ | ||
|
|
||
| import type { ICodecOptions } from "../../codec/index.js"; | ||
| import { Compatibility, type TreeStoredSchema } from "../../core/index.js"; | ||
| import { | ||
| defaultSchemaPolicy, | ||
| encodeTreeSchema, | ||
| makeSchemaCodec, | ||
| } from "../../feature-libraries/index.js"; | ||
| // eslint-disable-next-line import/no-internal-modules | ||
| import type { Format } from "../../feature-libraries/schema-index/index.js"; | ||
| import type { JsonCompatible } from "../../util/index.js"; | ||
| import type { ImplicitFieldSchema } from "../schemaTypes.js"; | ||
| import { toStoredSchema } from "../toFlexSchema.js"; | ||
| import type { SchemaCompatibilityStatus } from "./tree.js"; | ||
| import { ViewSchema } from "./view.js"; | ||
|
|
||
| /** | ||
| * Dumps the "persisted" schema subset of the provided `schema` into a deterministic JSON-compatible, semi-human-readable, but unspecified format. | ||
| * | ||
| * @remarks | ||
| * This can be used to help inspect schema for debugging, and to save a snapshot of schema to help detect and review changes to an applications schema. | ||
| * | ||
| * This format may change across major versions of this package: such changes are considered breaking. | ||
| * Beyond that, no compatibility guarantee is provided for this format: it should never be relied upon to load data, it should only be used for comparing outputs from this function. | ||
| * | ||
| * This only includes the "persisted" subset of schema information, which means the portion which gets included in documents. | ||
| * It thus uses "persisted" keys, see {@link FieldProps.key}. | ||
| * | ||
| * If two schema have identical "persisted" schema, then they are considered {@link SchemaCompatibilityStatus.isEquivalent|equivalent}. | ||
| * | ||
| * See also {@link comparePersistedSchema}. | ||
| * | ||
| * @example | ||
| * An application could use this API to generate a `schema.json` file when it first releases, | ||
| * then test that the schema is sill compatible with documents from that version with a test like : | ||
| * ```typescript | ||
| * assert.deepEqual(extractPersistedSchema(MySchema), require("./schema.json")); | ||
| * ``` | ||
| * | ||
| * @privateRemarks | ||
| * This currently uses the schema summary format, but that could be changed to something more human readable (particularly if the encoded format becomes less human readable). | ||
| * This intentionally does not leak the format types in the API. | ||
| * | ||
| * Public API surface uses "persisted" terminology while internally we use "stored". | ||
| * @alpha | ||
| */ | ||
| export function extractPersistedSchema(schema: ImplicitFieldSchema): JsonCompatible { | ||
| const stored = toStoredSchema(schema); | ||
| return encodeTreeSchema(stored); | ||
| } | ||
|
|
||
| /** | ||
| * Compares two schema extracted using {@link extractPersistedSchema}. | ||
| * Reports the same compatibility that {@link TreeView.compatibility} would report if | ||
| * opening a document that used the `persisted` schema and provided `view` to {@link ViewableTree.viewWith}. | ||
| * | ||
| * @param persisted - Schema persisted for a document. Typically persisted alongside the data and assumed to describe that data. | ||
| * @param view - Schema which would be used to view persisted content. Use {@link extractPersistedSchema} to convert the view schema into this format. | ||
| * @param options - {@link ICodecOptions} used when parsing the provided schema. | ||
| * @param canInitialize - Passed through to the return value unchanged and otherwise unused. | ||
| * @returns The {@link SchemaCompatibilityStatus} a {@link TreeView} would report for this combination of schema. | ||
| * | ||
| * @remarks | ||
| * This uses the persisted formats for schema, meaning it only includes data which impacts compatibility. | ||
| * It also uses the persisted format so that this API can be used in tests to compare against saved schema from previous versions of the application. | ||
| * | ||
| * @example | ||
| * An application could use {@link extractPersistedSchema} to generate a `schema.json` file for various versions of the app, | ||
| * then test that documents using those schema can be upgraded to work with the current schema using a test like: | ||
| * ```typescript | ||
| * assert( | ||
| * comparePersistedSchema( | ||
| * require("./schema.json"), | ||
| * extractPersistedSchema(MySchema), | ||
| * { jsonValidator: typeboxValidator }, | ||
| * false, | ||
| * ).canUpgrade, | ||
| * ); | ||
| * ``` | ||
| * @alpha | ||
| */ | ||
| export function comparePersistedSchema( | ||
| persisted: JsonCompatible, | ||
| view: JsonCompatible, | ||
| options: ICodecOptions, | ||
| canInitialize: boolean, | ||
| ): SchemaCompatibilityStatus { | ||
| const schemaCodec = makeSchemaCodec(options); | ||
| const stored = schemaCodec.decode(persisted as Format); | ||
| const viewParsed = schemaCodec.decode(view as Format); | ||
| const viewSchema = new ViewSchema(defaultSchemaPolicy, {}, viewParsed); | ||
| return comparePersistedSchemaInternal(stored, viewSchema, canInitialize); | ||
| } | ||
|
|
||
| /** | ||
| * Compute compatibility for viewing a document with `stored` schema using `viewSchema`. | ||
| * `canInitialize` is passed through to the return value unchanged and otherwise unused. | ||
| */ | ||
| export function comparePersistedSchemaInternal( | ||
| stored: TreeStoredSchema, | ||
| viewSchema: ViewSchema, | ||
| canInitialize: boolean, | ||
| ): SchemaCompatibilityStatus { | ||
| const result = viewSchema.checkCompatibility(stored); | ||
|
|
||
| // TODO: AB#8121: Weaken this check to support viewing under additional circumstances. | ||
| // In the near term, this should support viewing documents with additional optional fields in their schema on object types. | ||
| // Longer-term (as demand arises), we could also add APIs to constructing view schema to allow for more flexibility | ||
| // (e.g. out-of-schema content handlers could allow support for viewing docs which have extra allowed types in a particular field) | ||
| const canView = | ||
| result.write === Compatibility.Compatible && result.read === Compatibility.Compatible; | ||
| const canUpgrade = result.read === Compatibility.Compatible; | ||
| const isEquivalent = canView && canUpgrade; | ||
| const compatibility: SchemaCompatibilityStatus = { | ||
| canView, | ||
| canUpgrade, | ||
| isEquivalent, | ||
| canInitialize, | ||
| }; | ||
|
|
||
| return compatibility; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
48 changes: 48 additions & 0 deletions
48
packages/dds/tree/src/test/simple-tree/api/storedSchema.spec.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| /*! | ||
| * Copyright (c) Microsoft Corporation and contributors. All rights reserved. | ||
| * Licensed under the MIT License. | ||
| */ | ||
|
|
||
| import { strict as assert } from "node:assert"; | ||
|
|
||
| import { | ||
| comparePersistedSchema, | ||
| extractPersistedSchema, | ||
| // eslint-disable-next-line import/no-internal-modules | ||
| } from "../../../simple-tree/api/storedSchema.js"; | ||
| import { testSimpleTrees } from "../../testTrees.js"; | ||
| import { takeJsonSnapshot, useSnapshotDirectory } from "../../snapshots/index.js"; | ||
| import { typeboxValidator } from "../../../external-utilities/index.js"; | ||
|
|
||
| describe("simple-tree storedSchema", () => { | ||
| describe("test-schema", () => { | ||
| useSnapshotDirectory("simple-tree-storedSchema"); | ||
| for (const test of testSimpleTrees) { | ||
| it(test.name, () => { | ||
| const persisted = extractPersistedSchema(test.schema); | ||
| takeJsonSnapshot(persisted); | ||
| }); | ||
|
|
||
| // comparePersistedSchema is a trivial wrapper around functionality that is tested elsewhere, | ||
| // but might as will give it a simple smoke test for the various test schema. | ||
| it(`comparePersistedSchema to self ${test.name}`, () => { | ||
| const persistedA = extractPersistedSchema(test.schema); | ||
| const persistedB = extractPersistedSchema(test.schema); | ||
| const status = comparePersistedSchema( | ||
| persistedA, | ||
| persistedB, | ||
| { | ||
| jsonValidator: typeboxValidator, | ||
| }, | ||
| false, | ||
| ); | ||
| assert.deepEqual(status, { | ||
| isEquivalent: true, | ||
| canView: true, | ||
| canUpgrade: true, | ||
| canInitialize: false, | ||
| }); | ||
| }); | ||
| } | ||
| }); | ||
| }); |
8 changes: 8 additions & 0 deletions
8
packages/dds/tree/src/test/snapshots/simple-tree-storedSchema/empty.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| { | ||
| "version": 1, | ||
| "nodes": {}, | ||
| "root": { | ||
| "kind": "Optional", | ||
| "types": [] | ||
| } | ||
| } |
14 changes: 14 additions & 0 deletions
14
packages/dds/tree/src/test/snapshots/simple-tree-storedSchema/false boolean.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "version": 1, | ||
| "nodes": { | ||
| "com.fluidframework.leaf.boolean": { | ||
| "leaf": 2 | ||
| } | ||
| }, | ||
| "root": { | ||
| "kind": "Value", | ||
| "types": [ | ||
| "com.fluidframework.leaf.boolean" | ||
| ] | ||
| } | ||
| } |
Oops, something went wrong.