Investigate Kubeapps catalogs post index.yaml

[absoludity]

Summary

Kubeapps as a catalog of apps

From inception, Kubeapps’ main functionality is to present a catalog of applications (Helm Charts) that can be installed, upgraded and deleted on a Kubernetes cluster. Presenting a UI for a catalog of applications in Kubeapps was trivial because a traditional Helm repository provides an index.yaml file containing all data required for a catalog.

Initial OCI support in Helm and Kubeapps

Helm has also supported the installation of Charts from an OCI registry in an experimental form for some time and Kubeapps followed suit shortly after with the ability to include charts from OCI registries in the catalog. This feature was limited by the fact that there is no standard way to query an OCI registry for the list of charts. So instead, Kubeapps users are required to list the Helm charts explicitly that are to be included in the catalog when adding an OCI registry source. This worked fine for the level of support required in Kubeapps for OCI charts until recently.

OCI as the default, but without an index

At the beginning of 2022 Helm 3.8 was released with the OCI support enabled by default. In December 2022 Harbor 2.7 was released which began Harbor’s deprecation of its non-OCI Helm repository support (chartmuseum), which was then removed completely in the Harbor 2.8 release in April 2023. In the same month VMware’s Bitnami Application Catalog announced that Bitnami Helm Charts are now generally available as OCI packages in DockerHub and in fact, the Bitnami Catalog is defaulting to OCI packages for charts. Importantly, as part of this move, the consumption of Bitnami Helm Charts via the index.yaml is being deprecated.

This presents a problem for the main function of Kubeapps - presenting a catalog of applications - because the OCI distribution specification does not have a well defined way to list charts (repositories) within an OCI Registry. It does have a /v2/_catalog? endpoint, but this is not implemented in a standard way, or even at all for some registries (importantly, Docker’s), nor is it useful for larger public registries with namespaced repositories. Yet Kubeapps requires the ability to display a catalog of applications for an OCI registry (or a namespace thereof).

Background and rationale

Without a standard OCI API for listing namespaced charts/repositories in a registry, Kubeapps will need to either:

• Continue to enforce users to list the repositories as we do now, or
• Limit the catalog functionality to Helm index.yaml repositories, extending only for external online "hubs" for catalogs with OCI charts.
• Provide a service that can use custom, provider-specific APIs to list charts in OCI registries while exposing a generic API (ie. it knows how to service requests for "show me the charts in the registry registry-1.docker.io in the bitnamicharts namespace or similarly charts in a harbor registry", encapsulating the different APIs required).

Description

Current usage of Kubeapps by different companies leads me to think that the last of the above options will be more important: being able to specify an OCI registry and have Kubeapps include the list of charts from that registry (without having to explicitly list which apps/repositories they are interested in).

I've created a proposal for a small stateless gRPC OCI Catalog service which provides a generic gRPC call for listing repositories but includes implementations for specific providers.

Acceptance criteria
A formalized list of conditions that ensure that the feature can be considered as finished.

Additional context
Add any other context or screenshots about the feature request here.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[absoludity]

A few notes regarding caching available charts in an OCI repository:

• As above, the bitnami charts are now in the docker OCI registry at registry-1.docker.io/bitnamicharts. That is, in the docker registry registry-1.docker.io under the bitnamicharts namespace... there is no such thing as the "bitnamicharts" repository from the OCI pov... it's just a namespace in a registry.
• As we knew previously when we added OCI functionality in Kubeapps, the OCI spec has included the /v2/_catalog? endpoint but this is not well supported. In particular, docker's OCI implementation does not support this at all. You can imagine how large the result would be for, say, registry-1.docker.io, if the request cannot be namespaced, nor is paginating through an unfiltered set useful. There was chatter about just providing results for a specific user (and perhaps some services do this, not sure yet) which could be useful for private registries, but not large public registries.
• The 1.0 version of ORAS from March 2023 includes a namespaced repository listing:

oras repo ls --help                            
List the repositories under the registry

Example - List the repositories under the registry:
  oras repo ls localhost:5000

Example - List the repositories under a namespace in the registry:
  oras repo ls localhost:5000/example-namespace

which is using the OCI /v2/_catalog? endpoint, but is really just a local filter of the full result and so, not helpful for our use-case.

• The result of the above is that the only current option (that I can see) for supporting the indexing of charts in OCI registries (other than asking users to list the specific repos, as we do now) is to use service-specific APIs to obtain the relevant list of repositories (repositories containing chart artifacts, in this case).
• Given that the Bitnami repo is now using the docker OCI registry, if we do move in the direction of providing a catalog service, we should support the docker OCI registry first, while ensuring that we can support any of the OCI services (perhaps defaulting to something like the namespaced _catalog if we don't have an implementation for the specific service).
• If we did go down the route of creating a service for OCI chart catalogs, there would be no reason to limit it to helm charts, but rather, fill a general need of supporting catalog information for OCI artifacts generically, across the available services. This could be a neat tool, filling a current need in the OCI community, useful outside of Kubeapps, or even Helm OCI storage for that matter. (need to check whether there is anything filling this need - can't see anything from a brief search).

[absoludity]

In the other direction, we could include the Bitnami catalog by using the artifact hub API:

https://artifacthub.io/api/v1/packages/search?repo=bitnami&limit=20&offset=0

or similar to obtain a catalog for Kubeapps to display, but this only works for publicly available chart indexes (unless a similar private hub was available internally). For example, I think we should solve this in a way such that a company who has their own version of a public catalog (such as Bitnami but with custom base images or whatever) in their own private OCI repo, can continue to use Kubeapps, which is where this direction falls short.

[absoludity]

I've created a proposal for a small stateless gRPC OCI Catalog service which provides a generic gRPC call for listing repositories but includes implementations for specific providers.

[absoludity]

With the implementation task #6263 created, I'm closing this investigation, though happy to re-open or discuss any feedback of course.