[RFC] Catalog & API 3D Model Support

[AetherUnbound]

3D Models in the Catalog & API

Corollary to WordPress/openverse-frontend#889

This outline is also predicated on the idea that we've identified sources of 3D models, but will seek to flesh out the details regarding metadata fields, database migrations, and API development.

Some potential risks

• As it stands currently, each new media type means an additional data refresh. Data refreshes cannot be run simultaneously at present, so we may reach a saturation point where all of our data cannot be refreshed weekly.
• We do not have any standard database migration approach for the catalog, so all changes suggested here will likely be ad hoc unless Implementation Plan: Investigate the use of alembic for openledger migrations #1836 is addressed prior to this implementation.
• The API, while it does have a migration tool, does not have a standardized migration application process. We will need to run the migration manually when the time comes.

Some other considerations

• Similar to the frontend, it would be ideal to have this under a feature flag in the API. We could perform deployments and testing on our staging setup with the flag enabled, but still make other deployments to production with the flag disabled. We should be able to run the API for images & audio successfully with the feature flag disabled AND without having run the migrations.
• As we work through this on both the catalog and the API, we should begin compiling information for a document describing the process for adding a new media type. This will be useful for future media types we are sure to add!

Prior work

We may be able to use previous work regarding the addition of audio to the catalog/API as a guide. These updates required a fair amount of generalization, which we may not need to do a second time around.

#19

Technical implementation

1. Add 3D models to the catalog

• Define necessary fields & schema for model_3d table & associated data
• Add DDL to local Postgres image: https://github.com/WordPress/openverse-catalog/tree/main/docker/local_postgres
• Create a popularity calculation view for model_3d
• Add the database columns to db_columns.py: https://github.com/WordPress/openverse-catalog/blob/main/openverse_catalog/dags/common/storage/db_columns.py
• Add the TSV columns to tsv_columns.py: https://github.com/WordPress/openverse-catalog/blob/main/openverse_catalog/dags/common/storage/tsv_columns.py
• Create a Model3DStore subclass of MediaStore:
• Update the new provider API script template with the model_3d provider
• Create one/several provider API scripts & associated DAGs

2. Add 3D models to the API

Note: Borrowed heavily from #19

• Django models for model_3d
• Add DRF serializers for model_3d
• Add DRF endpoints for model_3d
• Make the thumbnail server work with model thumbnails

3. Add 3D models to the data refresh

• Add Elasticsearch model_3d models
• Ensure the data refresh works for 3D models
• Add 3D model sample data into the sample data CSV for testing

Open Questions

• What data is important to store in the catalog? Some of this is encapsulated in the proposed schema, but there may be other important attributes beyond those listed.
• Will it be important to differentiate the "type" of 3D model that a particular record refers to? (E.g. 3D printer model, game asset, artwork, etc.)
• Many model sites come with asset groups, should we have a table similar to AudioSets for these cases?

Proposed schema

Below is the proposed schema for the model_3d table in the catalog database. It is subject to change based on discussions that happen here and elsewhere.

Many of the fields towards the end are taken from examples on SketchFab (e.g. https://sketchfab.com/3d-models/charmanders-midnight-snack-57866c01da20414ba81885206acbcc85)

Field	Type	Description
identifier	UUID	Identifier for the record
created_on	timestamp	Time the record was created
updated_on	timestamp	Time the record was updated
ingestion_type	varchar	What generated the record
provider	varchar	Provider API that the model comes from
source	varchar	Source within the provider (e.g. NASA on Flickr), typically the same as provider
foreign_id	varchar	Identifier that the provider uses for this model
foreign_landing_url	varchar	URL on the provider that displays the "single-result" page for this model
url	varchar	URL to the actual asset itself (e.g. the image URL)
thumbnail	varchar	Thumbnail URL
filesize	integer	Size of the file
license	varchar	License for the model
license_version	varchar	Version of the model's license
creator	varchar	Creator ID of the model from the provider
creator_url	varchar	URL to the creator's page on the provider's site
title	varchar	Title of the model
meta_data	jsonb	Uncategorized metadata for the model
tags	jsonb	Tags that describe the model, for use in searching
watermarked	bool	Whether the model is watermarked - is this needed for 3D models?
last_synced_with_source	timestamp	Time the record was last updated from the provider
removed_from_source	bool	Whether the model has been removed from the upstream source
category	varchar	Category for the model (is this where we'll make the distinction between 3D printers, game assets, etc.?)
filetype	varchar	File type of the primary file for the model
alt_files	jsonb	Additional file types for the model
vertices	integer	Number of vertices within the model
textures	integer	Number of textures the model uses
materials	integer	Number of materials the model uses
uv_mapping	bool	Whether the model contains UV Mapping
vertex_colors	bool	Whether the model contains vertex colors
pbr	bool	Whether the model supports physically based rendering
rigged_geometries	bool	Whether the model supports rigging for animation
animations	integer	Number of animations the model has
morph_geometries	integer	Number of morph targets for animation

[AetherUnbound]

Closing in favor of #171