Imple referrers API of distribution 1.1.0

[wy65701436]

Task to tracking the imple of https://github.com/opencontainers/distribution-spec/blob/v1.1.0-rc1/spec.md#listing-referrers.

[sudo-bmitch]

The current handling responds to the referrers API with a 401 and no www-authenticate header. Is there a possibility of getting that to be a 404 in 2.7.x?

[wy65701436]

Thank you for bringing this up. According to this proposal, we won't be supporting tag schema in version 2.8 because the specification hasn't reached the GA stage yet.

If we return a 404 error in version 2.7.x, the client may try to push the artifact into Harbor with tag schema, which won't be migrated to version 2.8 and beyond.

[sudo-bmitch]

There are a few concerns I have with this. It doesn't prevent the push of digest tags without using the referrers API, and instead blindly copies tags from one repo to another:

$ regctl tag ls demo.goharbor.io/regclient/regctl
scratch
sha256-25acbc9974607efbbfcb94fb7f429cd92edc01664f439efd7f422d64318a7987
sha256-38a49c01979903cb0d88e186f0c8e7b2644865b48063eb74c45bdb856e5aa739
sha256-81b44ad77a83506e00079bfb7df04240df39d8da45891018b2e5e00d5d69aff3
sha256-82206c6245b6b82ef567e606c534331d840aa82e10cc6ed479ff681e24e8569b
sha256-82c1fa5ffa9c65d121c4e1386f30ac51d360f546814ab193adaf9ecf8c6fb0f2
sha256-a63ccc9cd6e5a10b863bc09fc1e327b19f823a7be1407bc782c0ad6aa6de2cf3
sha256-f928e47eea14a96a689c83ca4deb5fa38f7324f1dc4c03fe70bbff2f89e94834

Since the 401 is unexpected output, client behavior is undefined, and I don't think it's safe to assume they won't push the fallback tag. regclient was triggering a backoff for the unexpected output, but the failure of the API still falls back to a tag push.

My biggest concern is that this may break pulls from the registry, where a client checking if an image has additional data to download, mirror, etc, doesn't know how to handle the unexpected response. This was the case for my regctl image mod functionality, where I also copy referrers that point to the modified image digest. When encountering the unexpected response, I was assuming a broken registry and triggering backoffs degraded the user experience. regclient/regclient#384

Something I've seen from Docker Hub, which I don't like but is a lot cleaner, is to reject the manifest put when it contains a subject field. It would be good for OCI to hear from registry operators that don't want to support migrating user data to the new API, so that we can look at how this can be done in a way that servers and clients can cooperate.

[sajayantony]

My understanding is that the OCI-Subject enables clients to not depend on the referrers response code. I realize that this spec hasn't been released yet but wanted to understand if that header would solve the problem.
https://github.com/opencontainers/distribution-spec/blob/main/spec.md#pushing-manifests-with-subject

@sudo-bmitch explained this offline.

[wy65701436]

closed since PR was merged.