WIP added pack manifest cli

[WYGIN]

CNB Pack Manifest Commands: Streamlining Multi-Architecture Image Management

This proposal introduces a new experimental set of commands to the CNB tooling ecosystem: pack manifest. These commands empower buildpack authors to efficiently manage and distribute multi-architecture images, aligning with functionalities offered by existing tools like podman manifest and docker manifest.

Key Features:

• Create manifest lists: Construct local manifest lists for annotating and pushing to container registries.
• Annotate manifests: Add essential information like architecture, OS, or variant to existing manifest lists.
• Add/remove images: Manage individual image entries within manifest lists.
• Push manifests: Publish constructed manifest lists to container registries.
• Inspect manifests: View detailed information about locally stored manifest lists.

Benefits:

Simplified workflows: Centralized management of multi-architecture image variants for efficient distribution.
Enhanced interoperability: Potential adherence to established standards for seamless integration with existing tools and runtimes.
Improved developer experience: Streamlined workflows and user-friendly commands for managing manifest lists.
Target Users:

Buildpack authors seeking to distribute their buildpacks across diverse architectures.

Example Workflow:

Create a manifest list:

pack manifest create cnbs/sample-package:hello-multiarch-universe \
    cnbs/sample-package:hello-universe \
    cnbs/sample-package:hello-universe-windows

Annotate architecture:

pack manifest annotate --arch amd64 cnbs/sample-package:hello-multiarch-universe cnbs/sample-package:hello-universe
pack manifest annotate --arch amd64 cnbs/sample-package:hello-multiarch-universe cnbs/sample-package:hello-universe-windows

Push to registry:

pack manifest push cnbs/sample-package:hello-multiarch-universe

Current Status:

These commands are initially experimental and gated behind the pack config experimental flag. They will transition to supported status upon deemed readiness by maintainers.

This innovative addition to the CNB ecosystem empowers buildpack authors to streamline multi-architecture image management, fostering efficient distribution and enhanced developer experiences.

Development setup:

This guide outlines the steps to set up your development environment and test the pack manifest commands under development in the image-index branch of the github.com/WYGIN/buildpacks-pack repository against the github.com/WYGIN/buildpacks-imgutil repository.

Prerequisites:

• Docker installed and running on your system.
• Git version control system installed.

Steps:

1. Clone the repositories:

git clone https://github.com/WYGIN/buildpacks-pack.git

2. Navigate to the buildpacks-pack repository:

cd buildpacks-pack

3. Checkout the image-index branch:

git checkout image-index

4. Build the pack binary:

make build

5. Install the pack binary:

sudo make install

6. Enable pack's experimental features:

pack config experimental true

7. Run pack manifest:

pack manifest create cnbs/sample-package:hello-multiarch-universe \
     cnbs/sample-package:hello-universe \

     cnbs/sample-package:hello-universe-windows

# annotate first image
pack manifest annotate --arch amd64 cnbs/sample-package:hello-multiarch-universe cnbs/sample-package:hello-universe
# annotate second image
pack manifest annotate --arch amd64 cnbs/sample-package:hello-multiarch-universe cnbs/sample-package:hello-universe-windows

pack manifest add cnbs/sample-package:hello-multiarch-universe \
					cnbs/sample-package:hello-universe-riscv-linux

pack manifest delete cnbs/sample-package:hello-multiarch-universe

pack manifest rm cnbs/sample-package:hello-multiarch-universe \
					cnbs/sample-package:hello-universe-windows

pack manifest push cnbs/sample-package:hello-multiarch-universe

pack manifest inspect cnbs/sample-builder:multiarch

Switching between modes:

There are two modes, where one only manipulates the ImageIndex, where as the other will manipulates the ImageIndex along with the underlying image.
To switch between these modes use:

pack config index-full-mode [<true> | <false>]

[jjbustamante]

I think the Index is not created correctly, take a look at the media-type saved on disk, it should be application/vnd.oci.image.index.v1+json or the docker equivalent.

Just to complement my last comment, the manifest annotate method worked correctly, but the push failed with the same error

[WYGIN]

I think the Index is not created correctly, take a look at the media-type saved on disk, it should be application/vnd.oci.image.index.v1+json or the docker equivalent.

Just to complement my last comment, the manifest annotate method worked correctly, but the push failed with the same error

I found the following thing on GitHub even though it's not a reason for the current bug I want to take it to your notice
The above bug still exists for manifest-full-mode and one can push only index manifest by ManifestHandler for now and IndexHandler uses OCI layout format and path#ImageIndex's mediaType is defaults to OCIImageIndex even if the Index is Docker Manifest List, so when remote#Write pushing index it is setting header with OCIImageIndex even when pushing Docker manifest list which is why the error is popping up

Any suggestions on how to deal with it?

1. I was thinking either to create a new oci layout for docker index which still saves index in oci layout format but when reading with layout#ImageIndex will return Docker Manifest List format
2. Or creating a pull request to GGCR to set header based on IndexManifest's MediaType
3. or pushing every image in the index concurrently and then push image index finally with the remote#Put to only push IndexManifest(i feel its a better option within the list)

[jjbustamante]

I found the following thing on GitHub even though it's not a reason for the current bug I want to take it to your notice The above bug still exists for manifest-full-mode and one can push only index manifest by ManifestHandler for now and IndexHandler uses OCI layout format and path#ImageIndex's mediaType is defaults to OCIImageIndex even if the Index is Docker Manifest List, so when remote#Write pushing index it is setting header with OCIImageIndex even when pushing Docker manifest list which is why the error is popping up

Any suggestions on how to deal with it?

1. I was thinking either to create a new oci layout for docker index which still saves index in oci layout format but when reading with layout#ImageIndex will return Docker Manifest List format
2. Or creating a pull request to GGCR to set header based on IndexManifest's MediaType
3. or pushing every image in the index concurrently and then push image index finally with the remote#Put to only push IndexManifest(i feel its a better option within the list)

I saw the error you are mentioning, but that is not happening with Husni's code. I managed to push the Image Index with Husni's pack version and the index is using docker media types

[husni-faiz]

Please address the comment