Adding support for user-defined boxes

[bradh]

I'm interested in adding support for metadata outside what is currently defined in libavif. Its mainly uuid item properties, but some of it might (eventually) be registered with MP4RA as different boxes.

It looks like the data could be an array of something similar to avifROData, but I'd need a label (at least a 4CC, and a UUID for the uuid case).

I'd anticipate the ipco parser would add to the list (instead of skipping if we don't have a match).

Does this kind of implementation look reasonable? I realise that the detailed implementation matters, and am just looking for general guidance at this stage.

[y-guyon]

You are only talking about the decoding API in libavif, right? Or is this feature also needed at encoding?

[bradh]

Eventually it might be useful to be able to add arbitrary properties, but for now I'd just like to expose the properties at parse / decode time.

[y-guyon]

Would these item properties be only tied to the primary image item?

[bradh]

They could be tied to any image item.

While I'm not expecting support at this time, they could be attached to individual frames in an image sequence too (e.g. using saiz and saio). Image items is enough for now.

[y-guyon]

Do you have examples of this kind of property or box that is not currently exposed by the libavif API?

[bradh]

My motivation is to allow geospatial referencing in HEIF (including AVIF), much like the way GeoTIFF extends TIFF. This is being worked as an OGC experiment (https://www.ogc.org/initiatives/ogc-testbed-20/ for the marketing version).

I'm planning on three new properties - see mp4ra/mp4ra.github.io#241

The application would be to extend something like OSGeo/gdal#10621 to do orthophotos.

A related use of such a property is the way uuid boxes can be used for "private" user extensions. I'm seeing that in draft National Geospatial-Intelligence Agency (NGA) documents that use it to do opaque labels (actually UUIDs) for image items.

I'm not expecting that those would be interpreted by libavif, just exposed as raw bytes.

[y-guyon]

Are there other libraries than libavif that could be easily used to extract the metadata you mentioned?

[bradh]

For libheif, I'm hoping to use the Box_other API . Testing that will be later today.

I'm not sure there is a good way to pull the image data using libavif and separately parse out the metadata for each image item.

[y-guyon]

Here is a draft PR for discussion: #2420

What do others think?

[maryla-uc]

Seems reasonable to me!