feat(gltf): Add GLBArrowLoader #3160
There are no files selected for viewing
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| import {Matrix4} from '@math.gl/core'; | ||
| import {Accessor, Node, PlatformIO, Primitive, WebIO} from '@gltf-transform/core'; | ||
| import {KHRONOS_EXTENSIONS} from '@gltf-transform/extensions'; | ||
| import {unweld, uninstance, dequantize} from '@gltf-transform/functions'; | ||
| import * as arrow from 'apache-arrow'; | ||
| import type {LoaderWithParser, LoaderOptions} from '@loaders.gl/loader-utils'; | ||
| import {ArrowTable, DataType, Field, Schema, SchemaMetadata} from '@loaders.gl/schema'; | ||
| import {deserializeArrowField, deserializeArrowType} from '@loaders.gl/schema-utils'; | ||
| import {GLBLoader} from './glb-loader'; | ||
|
|
||
| /** GLB Arrow loader options */ | ||
| export type GLBArrowLoaderOptions = LoaderOptions & { | ||
| io?: PlatformIO; | ||
| }; | ||
|
|
||
| export type ArrowTableTransformList = [ArrowTable, Matrix4][]; | ||
|
There was a problem hiding this comment. TBD what the output should ideally be here? |
||
|
|
||
| /** | ||
| * GLB Loader - | ||
| * GLB is the binary container format for GLTF | ||
| */ | ||
| export const GLBArrowLoader = { | ||
| ...GLBLoader, | ||
| dataType: null as unknown as ArrowTableTransformList, | ||
| batchType: null as never, | ||
| worker: false, | ||
| parse, | ||
| parseSync: undefined | ||
| } as const satisfies LoaderWithParser<ArrowTableTransformList, never, GLBArrowLoaderOptions>; | ||
|
|
||
| async function parse( | ||
| arrayBuffer: ArrayBuffer, | ||
| options?: GLBArrowLoaderOptions | ||
| ): Promise<ArrowTableTransformList> { | ||
| const io = options?.io || new WebIO().registerExtensions(KHRONOS_EXTENSIONS); | ||
| const document = await io.readBinary(new Uint8Array(arrayBuffer)); | ||
|
|
||
| // Unclear how represent indexed, instanced, or normalized meshes as | ||
| // ArrowTable. Convert to simpler representations for now. | ||
| await document.transform(unweld(), uninstance(), dequantize()); | ||
|
Comment on lines
+38
to
+40
There was a problem hiding this comment. Pre-processing to ensure the vertex buffers can be represented as Arrow tables. Possibly quantized meshes (int8 or int16 vertex attributes) could be represented, but we'd need to put the 'normalized: boolean' option somewhere. Similarly there could be ways to represent instanced draws with a second Arrow table for the instance transforms, if we want to go that direction. There was a problem hiding this comment. My thoughts about indexed tables was to add an index column as a const indexes = new Uint32Array([0, 1, 2, 3, 4, 5]);
const nextIndex = indexes.length;
const indexOffsets = new Uint32Array(5).fill(nextIndex);There was a problem hiding this comment. We can add metadata on every column (field) has a |
||
|
|
||
| const scene = document.getRoot().getDefaultScene() || document.getRoot().listScenes()[0]; | ||
| const meshList: ArrowTableTransformList = []; | ||
|
|
||
| // Traverse the default scene, creating a list of mesh primitives and their | ||
| // corresponding scene transforms. | ||
| scene.traverse((node: Node) => { | ||
| if (node.getMesh()) { | ||
| const matrix = new Matrix4(node.getWorldMatrix()); | ||
| for (const prim of node.getMesh()!.listPrimitives()) { | ||
| meshList.push([convertPrimitiveToArrowTable(prim), matrix]); | ||
| } | ||
| } | ||
| }); | ||
|
|
||
| return meshList; | ||
| } | ||
|
|
||
| /** | ||
| * Encodes a glTF Transform Primitive as an ArrowTable. Currently ignores | ||
| * materials, morph targets, and extras. | ||
| */ | ||
| function convertPrimitiveToArrowTable(prim: Primitive): ArrowTable { | ||
| const fields: Field[] = []; | ||
| const arrowFields: arrow.Field[] = []; | ||
| const arrowAttributes: arrow.Data[] = []; | ||
|
|
||
| let vertexCount = -1; | ||
|
|
||
| for (const name of prim.listSemantics()) { | ||
| const attribute = prim.getAttribute(name)!; | ||
| const type = componentTypeToDataType(attribute.getComponentType()); | ||
|
|
||
| const field: Field = {name, type}; | ||
| const arrowField = deserializeArrowField(field); | ||
| const arrowAttribute = accessorToArrowListData(attribute); | ||
|
|
||
| fields.push(field); | ||
| arrowFields.push(arrowField); | ||
| arrowAttributes.push(arrowAttribute); | ||
|
|
||
| if (vertexCount <= 0) { | ||
| vertexCount = attribute.getCount(); | ||
| } | ||
| } | ||
|
|
||
| const metadata: SchemaMetadata = {}; | ||
| const schema: Schema = {fields, metadata}; | ||
|
|
||
| const arrowSchema = new arrow.Schema(arrowFields); | ||
| const arrowStruct = new arrow.Struct(arrowFields); | ||
| const arrowData = new arrow.Data(arrowStruct, 0, vertexCount, 0, undefined, arrowAttributes); | ||
| const arrowRecordBatch = new arrow.RecordBatch(arrowSchema, arrowData); | ||
| const arrowTable = new arrow.Table([arrowRecordBatch]); | ||
|
Comment on lines
+90
to
+94
There was a problem hiding this comment. |
||
|
|
||
| return {shape: 'arrow-table', schema, data: arrowTable}; | ||
| } | ||
|
|
||
| /** Encodes a glTF component type as an equivalent DataType string. */ | ||
| function componentTypeToDataType(componentType: number): DataType { | ||
| switch (componentType) { | ||
| case Accessor.ComponentType.FLOAT: | ||
| return 'float32'; | ||
| case Accessor.ComponentType.UNSIGNED_BYTE: | ||
| return 'uint8'; | ||
| case Accessor.ComponentType.UNSIGNED_SHORT: | ||
| return 'uint16'; | ||
| case Accessor.ComponentType.UNSIGNED_INT: | ||
| return 'uint32'; | ||
| case Accessor.ComponentType.BYTE: | ||
| return 'int8'; | ||
| case Accessor.ComponentType.SHORT: | ||
| return 'int16'; | ||
| case Accessor.ComponentType.INT: | ||
| return 'int32'; | ||
| default: | ||
| throw new Error(`Unexpected component type, ${componentType}`); | ||
| } | ||
| } | ||
|
|
||
| /** Encodes a glTF Transform Accessor as an arrow.Data list. */ | ||
| function accessorToArrowListData(accessor: Accessor): arrow.Data<arrow.FixedSizeList> { | ||
| const size = accessor.getElementSize(); | ||
| const count = accessor.getCount(); | ||
| const type = componentTypeToDataType(accessor.getComponentType()); | ||
| const arrowType = deserializeArrowType(type); | ||
| const arrowList = new arrow.FixedSizeList(size, new arrow.Field('value', arrowType)); | ||
| const arrowNestedType = arrowList.children[0].type; // TODO: Eh? | ||
| const buffers = {[arrow.BufferType.DATA]: accessor.getArray()}; | ||
| const arrowNestedData = new arrow.Data(arrowNestedType, 0, size * count, 0, buffers); | ||
| const arrowData = new arrow.Data<arrow.FixedSizeList>(arrowList, 0, count, 0, undefined, [ | ||
| arrowNestedData | ||
| ]); | ||
| return arrowData; | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the application is running in a non-browser environment (Node, Deno) or requires other dependencies (Draco, Meshopt) then a custom I/O class should be provided. Example:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Loaders.gl has a bunch of abstractions like ReadableFile etc that have implementations that work under Node, Browser, HTTPs etc. This seems to duplicate some of the IO class responsibilities.
I wonder if we could create a
LoadersIOcustom IO class for gltf-transform on top of the loaders.gl abstractions that glued the two libraries together, so that your gltf-transform would work with the abstractions we typically use in loaders.That way this gltf-transform based loader wouldn't become a one-off loader that doesn't fully work like other loaders do.