[RFC] move descriptor (and layout?) to distribution-spec

[vbatts]

As the references-wg has evolved, and as we've discussed before, we've identified definitions that are contained in this image-spec repo that are not strictly bound to the container images. Namely:

• https://github.com/opencontainers/image-spec/blob/main/descriptor.md
• https://github.com/opencontainers/image-spec/blob/main/image-layout.md
• and to some extent https://github.com/opencontainers/image-spec/blob/main/annotations.md (at least the definitions)

I think we can move these over, and coordinate on releases.

Any golang definitions can move and just import as an alias.

[sudo-bmitch]

While we're thinking of moving things, I'd recommend renaming "Image Layout" to "Filesystem Layout", since it can contain more than images. If you'd like someone else to submit the PR's, let me know, I'd be happy to submit them.

[rchincha]

With dist-spec (with HTTP-only APIs), leaves the door open for filesystem folks or storage vendors to just implement that interface and claim conformance while retaining any benefits that their own layouts may provide. So in this sense, is this approach overreach?

BTW, for precendence the zot registry project [1] has taken this approach and in our experience has worked out so far.

References:
[1] https://github.com/project-zot/zot

[waynr]

I agree with @rchincha, I don't see how the image layout relates to distribution-spec. Let's say it was moved to distribution-spec, how would that actually impact implementers of distribution-spec?

It seems potentially more relevant to the runtime spec since, to my knowledge, it's the runtime that actually deals with storing images locally on runtime system disks where they may need to be shared between multiple runtimes. The image layout description even calls out runtimes:

Given an image layout and a ref, a tool can create an OCI Runtime Specification bundle by:

• Following the ref to find a manifest, possibly via an image index
• Applying the filesystem layers in the specified order
• Converting the image configuration into an OCI Runtime Specification config.json

As far as I can see, the closest it comes to mentioning anything related to distribution-spec is

This layout MAY be used in a variety of different transport mechanisms: archive formats (e.g. tar, zip), shared filesystem environments (e.g. nfs), or networked file fetching (e.g. http, ftp, rsync).

Where there is mention of "networked file fetching", but it seems clear to me that the http example given doesn't have anything to do with distribution.

As the references-wg has evolved, and as we've discussed before, we've identified definitions that are contained in this image-spec repo that are not strictly bound to the container images

On the topic of descriptors, I think I understand what you mean when you point out that they are not strictly bound to container images. Currently, distribution-spec does link to image-spec to establish the definition of "descriptor". But if it were moved into distribution-spec, wouldn't that just mean that image-spec would then have to link to distribution-spec? Is there a reason that descriptor is more fundamental to distribution-spec than image-spec?

In my view, it seems more fundamental to image-spec since the "image" concept itself is more fundamental overall -- ie, distribution-spec only exists to serve the needs of image-spec and runtime-spec.

[waynr]

Oh and one more thing

Any golang definitions can move and just import as an alias.

The implied assumption that registries are only implemented in golang is questionable. I know of at least one runtime written in another language and I am working on a distribution implementation in rust myself, which depends on a crate that maintains mappings for distribution/image/runtime specs. Granted, it shouldn't be too arduous to update that library, but I suspect there are other languages also.

So given the impact for the ecosystem of implementers, it seems to me like moving definitions around should have a pretty strong motivation.

[sudo-bmitch]

@waynr

I agree with @rchincha, I don't see how the image layout relates to distribution-spec. Let's say it was moved to distribution-spec, how would that actually impact implementers of distribution-spec?

From the view of a registry server, there's no value add. But from the view of how images, artifacts, and anything else we store in OCI formats, this is the filesystem method for distributing those things. It's commonly used in air gapped environments, but I'm also pushing to use it more within CI pipelines where it can be more efficient to share OCI objects between steps.

So from the perspective of tools like skopeo and regclient, distributing images by filesystem or registry servers are just different transports and it's useful to have a standard defining each of the transports. For us, having that in the runtime spec wouldn't make much sense, particularly since it's not just images we could be packaging in this format (we're starting to look at sboms, signatures, helm charts, and more).