Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new manifest command to support multi-architecture builds #1967

Closed
wants to merge 63 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
17ec0e6
WIP: added manifest_add command
WYGIN Nov 9, 2023
45f39bc
WIP added manifest_annotate command
WYGIN Nov 9, 2023
5b75d95
WIP added manifest_create command
WYGIN Nov 9, 2023
9799f01
WIP added manifest_exists command
WYGIN Nov 9, 2023
1968518
WIP added manifest_inspect command
WYGIN Nov 9, 2023
d8f6039
WIP added manifest_push command
WYGIN Nov 9, 2023
07b9150
WIP added manifest_remove command
WYGIN Nov 9, 2023
a6355e1
WIP added manifest_rm command
WYGIN Nov 9, 2023
f24b167
WIP added add_manifest client
WYGIN Nov 9, 2023
15ed584
WIP added annotate_manifest client
WYGIN Nov 9, 2023
8cf016e
WIP added IndexFactory to Client
WYGIN Nov 9, 2023
0ed954f
WIP added create_manifest client
WYGIN Nov 9, 2023
dfd4bc4
WIP added delete_manifest client
WYGIN Nov 9, 2023
94ba3e3
WIP added exists_manifest client
WYGIN Nov 9, 2023
4ec02e7
WIP added inspect_manifest client
WYGIN Nov 9, 2023
f2161a9
WIP added push_manifest client
WYGIN Nov 9, 2023
2080208
WIP added remove_manifest client
WYGIN Nov 9, 2023
5aa4b9f
WIP added errors
WYGIN Nov 9, 2023
d10a617
build(deps): bump github.com/buildpacks/lifecycle from 0.17.1 to 0.17.2
dependabot[bot] Oct 18, 2023
32e5d1d
Updating default lifecycle version to 0.17.2
jjbustamante Oct 26, 2023
d5d98bc
Group minor/patch version Go Dependabot updates into one PR
edmorley Oct 27, 2023
bf88725
Add buildpacksio/pack:<version>-base images to delivery
natalieparellano Oct 27, 2023
31654b8
Add floating :base tag
natalieparellano Oct 30, 2023
f70fdfe
Ensure the run image os/arch always matches:
natalieparellano Oct 12, 2023
0133424
When downloading buildpacks or extensions for `pack build` or `pack b…
natalieparellano Oct 12, 2023
0848fec
build(deps): bump the go-dependencies group with 6 updates
dependabot[bot] Oct 31, 2023
71821d8
Fix misleading log message when publishing a buildpack package
natalieparellano Sep 22, 2023
4624639
WIP added code to generate env files in buildConfigEnv dir
WYGIN Oct 4, 2023
9854231
WIP added required abstract test cases
WYGIN Oct 4, 2023
161d06a
WIP added tests that still nedd to be fixed
WYGIN Oct 5, 2023
7183dcf
WIP change from action to suffix and delim in builder.toml
WYGIN Oct 5, 2023
1480037
WIP added logic to add buildConfigEnvs to layer of image
WYGIN Oct 8, 2023
b8409a9
build-config-env layer oly added when it is not empty
WYGIN Oct 12, 2023
fc47fd8
added requested changes and fix bugs
WYGIN Oct 25, 2023
4feb437
fix: code format
WYGIN Oct 25, 2023
e41d99e
improved code coverage
WYGIN Oct 26, 2023
e3cc4d9
build(deps): bump buildpacks/github-actions from 5.4.0 to 5.5.0
dependabot[bot] Nov 1, 2023
8e69f1b
Add ServerVersion to docker client interface which is needed by imgutil
jericop Nov 1, 2023
a8335fb
Use imgutil fork to verify platform changes
jericop Nov 2, 2023
f9ec20e
Bump imgutil dependency for default platform change
jericop Nov 2, 2023
b714626
WIP: added manifest_add command
WYGIN Nov 9, 2023
8b109bf
WIP added manifest_annotate command
WYGIN Nov 9, 2023
a76513d
WIP added manifest_create command
WYGIN Nov 9, 2023
128f178
WIP added manifest_exists command
WYGIN Nov 9, 2023
d238c64
WIP added manifest_inspect command
WYGIN Nov 9, 2023
9006ad4
WIP added manifest_push command
WYGIN Nov 9, 2023
11d416d
WIP added manifest_remove command
WYGIN Nov 9, 2023
749b6fe
WIP added manifest_rm command
WYGIN Nov 9, 2023
8b25d63
WIP added add_manifest client
WYGIN Nov 9, 2023
6b8f798
WIP added annotate_manifest client
WYGIN Nov 9, 2023
4aa9362
WIP added IndexFactory to Client
WYGIN Nov 9, 2023
89763fd
WIP added create_manifest client
WYGIN Nov 9, 2023
a5b1faa
WIP added delete_manifest client
WYGIN Nov 9, 2023
6090d11
WIP added exists_manifest client
WYGIN Nov 9, 2023
b8f3186
WIP added inspect_manifest client
WYGIN Nov 9, 2023
b9dd8a4
WIP added push_manifest client
WYGIN Nov 9, 2023
54c3c6c
WIP added remove_manifest client
WYGIN Nov 9, 2023
553f927
WIP added errors
WYGIN Nov 9, 2023
c7180d8
Merge branch 'manifest' of https://github.com/WYGIN/buildpacks-pack i…
WYGIN Nov 9, 2023
4bbb483
WIP generate mock code with 'make generate'
WYGIN Nov 9, 2023
8f81b6c
WIP added tests for add_manifest
WYGIN Nov 9, 2023
f12c849
WIP improved add_manifest client tests
WYGIN Nov 10, 2023
4326812
WIP refactor manifest_add client
WYGIN Nov 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ func NewPackCommand(logger ConfigurableLogger) (*cobra.Command, error) {
rootCmd.AddCommand(commands.SetDefaultRegistry(logger, cfg, cfgPath))
rootCmd.AddCommand(commands.RemoveRegistry(logger, cfg, cfgPath))
rootCmd.AddCommand(commands.YankBuildpack(logger, cfg, packClient))
rootCmd.AddCommand(commands.NewManifestCommand(logger, packClient))
}

packHome, err := config.PackHome()
Expand Down
8 changes: 8 additions & 0 deletions internal/commands/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ type PackClient interface {
InspectExtension(client.InspectExtensionOptions) (*client.ExtensionInfo, error)
PullBuildpack(context.Context, client.PullBuildpackOptions) error
DownloadSBOM(name string, options client.DownloadSBOMOptions) error
CreateManifest(ctx context.Context, name string, images []string, opts client.CreateManifestOptions) (imageID string, err error)
AnnotateManifest(ctx context.Context, name string, image string, opts client.ManifestAnnotateOptions) error
ExistsManifest(ctx context.Context, image string) error
AddManifest(ctx context.Context, index string, images string, opts client.ManifestAddOptions) (imageID string, err error)
DeleteManifest(ctx context.Context, name []string) error
RemoveManifest(ctx context.Context, name string, images []string) error
PushManifest(ctx context.Context, index string, opts client.PushManifestOptions) (imageID string, err error)
InspectManifest(ctx context.Context, name string, opts client.InspectManifestOptions) error
}

func AddHelpFlag(cmd *cobra.Command, commandName string) {
Expand Down
26 changes: 26 additions & 0 deletions internal/commands/manifest.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package commands

import (
"github.com/spf13/cobra"

"github.com/buildpacks/pack/pkg/logging"
)

func NewManifestCommand(logger logging.Logger, client PackClient) *cobra.Command {
cmd := &cobra.Command{
Use: "manifest",
Short: "Interact with image index or manifest list",
RunE: nil,
}

cmd.AddCommand(ManifestCreate(logger, client))
cmd.AddCommand(ManifestAdd(logger, client))
cmd.AddCommand(ManifestAnnotate(logger, client))
cmd.AddCommand(ManifestDelete(logger, client))
cmd.AddCommand(ManifestInspect(logger, client))
cmd.AddCommand(ManifestPush(logger, client))
cmd.AddCommand(ManifestRemove(logger, client))

AddHelpFlag(cmd, "manifest")
return cmd
}
98 changes: 98 additions & 0 deletions internal/commands/manifest_add.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package commands

import (
"fmt"
"strings"

"github.com/spf13/cobra"

"github.com/buildpacks/pack/pkg/client"
"github.com/buildpacks/pack/pkg/logging"
)

// ManifestAddFlags define flags provided to the ManifestAdd
type ManifestAddFlags struct {
OS, OSVersion, OSArch, OSVariant string
OSFeatures, Annotations, Features string
all bool
}

// ManifestAdd modifies a manifest list (Image index) and add a new image to the list of manifests.
func ManifestAdd(logger logging.Logger, pack PackClient) *cobra.Command {
var flags ManifestAddFlags

cmd := &cobra.Command{
Use: "pack manifest add [OPTIONS] <manifest-list> <manifest> [flags]",
Args: cobra.MatchAll(cobra.ExactArgs(2), cobra.OnlyValidArgs),
Short: "manifest add modifies a manifest list (Image index) and add a new image to the list of manifests.",
Example: `pack manifest add cnbs/sample-package:hello-multiarch-universe \
cnbs/sample-package:hello-universe-riscv-linux`,
Long: `manifest add modifies a manifest list (Image index) and add a new image to the list of manifests.

When a manifest list exits locally, user can add a new image to the manifest list using this command`,
RunE: logError(logger, func(cmd *cobra.Command, args []string) error {
imageIndex := args[0]
manifests := args[1]
if err := validateManifestAddFlags(flags); err != nil {
return err
}

osFeatures:= strings.Split(flags.OSFeatures, ";")
features:= strings.Split(flags.Features, ";")
annotations, err := stringToKeyValueMap(flags.Annotations)
if err != nil {
return err
}

imageID, err := pack.AddManifest(cmd.Context(), imageIndex, manifests, client.ManifestAddOptions{
OS: flags.OS,
OSVersion: flags.OSVersion,
OSArch: flags.OSArch,
OSVariant: flags.OSVariant,
OSFeatures: osFeatures,
Features: features,
Annotations: annotations,
All: flags.all,
})

if err != nil {
return err
}

logger.Infof(imageID)
return nil
}),
}

cmd.Flags().BoolVar(&flags.all, "all", false, "add all of the contents to the local list (applies only if <manifest> is an index)")
cmd.Flags().StringVar(&flags.os, "os", "", "Set the operating system")
cmd.Flags().StringVar(&flags.arch, "arch", "", "Set the architecture")
cmd.Flags().StringVar(&flags.variant, "variant", "", "Set the architecture variant")

AddHelpFlag(cmd, "add")
return cmd
}

func validateManifestAddFlags(flags ManifestAddFlags) error {
return nil
}

func stringToKeyValueMap(s string) (map[string]string, error) {
keyValues := strings.Split(s, ";")

m := map[string]string{}

for _, keyValue := range keyValues {
parts := strings.Split(keyValue, "=")
if len(parts) != 2 {
return nil, fmt.Errorf("invalid key-value pair: %s", keyValue)
}

key := parts[0]
value := parts[1]

m[key] = value
}

return m, nil
}
104 changes: 104 additions & 0 deletions internal/commands/manifest_add_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
package commands_test

import (
"bytes"
"testing"

"github.com/golang/mock/gomock"
"github.com/heroku/color"
"github.com/sclevine/spec"
"github.com/sclevine/spec/report"
"github.com/spf13/cobra"

"github.com/buildpacks/pack/internal/commands"
"github.com/buildpacks/pack/internal/commands/testmocks"
"github.com/buildpacks/pack/pkg/logging"
)

func TestManifestAddCommand(t *testing.T) {
color.Disable(true)
defer color.Disable(false)

spec.Run(t, "Commands", testManifestAddCommand, spec.Random(), spec.Report(report.Terminal{}))
}

func testManifestAddCommand(t *testing.T, when spec.G, it spec.S) {
var (
command *cobra.Command
logger *logging.LogWithWriters
outBuf bytes.Buffer
mockController *gomock.Controller
mockClient *testmocks.MockPackClient
)

it.Before(func() {
logger = logging.NewLogWithWriters(&outBuf, &outBuf)
mockController = gomock.NewController(t)
mockClient = testmocks.NewMockPackClient(mockController)

command = commands.ManifestAdd(logger, mockClient)
})

when("#ManifestAdd", func() {
when("no flags specified", func() {

})
when("when --all flags passed", func() {

})
when("when --os flags passed", func() {

})
when("when --arch flags passed", func() {

})
when("when --variant flags passed", func() {

})
when("when --os-version flags passed", func() {

})
when("when --features flags passed", func() {

})
when("when --os-features flags passed", func() {

})
when("when --annotations flags passed", func() {

})
when("when multiple flags passed", func() {

})
when("when no args passed", func() {

})
when("when manifest list reference is incorrect", func() {

})
when("when manifest reference is incorrect", func() {

})
when("when manifest passed in-place of manifest list on first arg", func() {

})
when("when manifest list is passed on second arg", func() {

})
when("when manifest is passed on second arg with --all option", func() {

})
when("when manifest list in locally available", func() {

})
when("when manifest is not locally available", func() {

})
when("when manifest is locally available passed", func() {

})
when("when multiple manifests passed", func() {

})
})
}
42 changes: 42 additions & 0 deletions internal/commands/manifest_annotate.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package commands

import (
"github.com/spf13/cobra"

"github.com/buildpacks/pack/pkg/logging"
)

// ManifestAnnotateFlags define flags provided to the ManifestAnnotate
type ManifestAnnotateFlags struct {
os, arch, variant, osVersion string
features, osFeatures, annotations []string
}

// ManifestAnnotate modifies a manifest list (Image index) and update the platform information for an image included in the manifest list.
func ManifestAnnotate(logger logging.Logger, pack PackClient) *cobra.Command {
var flags ManifestAnnotateFlags

cmd := &cobra.Command{
Use: "pack manifest annotate [OPTIONS] <manifest-list> <manifest> [<manifest>...] [flags]",
Args: cobra.MatchAll(cobra.ExactArgs(2), cobra.OnlyValidArgs),
Short: "manifest annotate modifies a manifest list (Image index) and update the platform information for an image included in the manifest list.",
Example: `pack manifest annotate cnbs/sample-package:hello-universe-multiarch \
cnbs/sample-package:hello-universe --arch amd64`,
Long: `manifest annotate modifies a manifest list (Image index) and update the platform information for an image included in the manifest list.
Sometimes a manifest list could reference an image that doesn't specify the architecture, The "annotate" command allows users to update those values before pushing the manifest list a registry`,
RunE: logError(logger, func(cmd *cobra.Command, args []string) error {
return nil
}),
}

cmd.Flags().StringVar(&flags.os, "os", "", "Set the architecture")
cmd.Flags().StringVar(&flags.arch, "arch", "", "Set the architecture")
cmd.Flags().StringVar(&flags.variant, "variant", "", "Set the architecture")
cmd.Flags().StringVar(&flags.osVersion, "os-version", "", "override the os `version` of the specified image")
cmd.Flags().StringSliceVar(&flags.features, "features", nil, "override the `features` of the specified image")
cmd.Flags().StringSliceVar(&flags.osFeatures, "os-features", nil, "override the os `features` of the specified image")
cmd.Flags().StringSliceVar(&flags.annotations, "annotations", nil, "set an `annotation` for the specified image")

AddHelpFlag(cmd, "annotate")
return cmd
}
101 changes: 101 additions & 0 deletions internal/commands/manifest_annotate_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
package commands_test

import (
"bytes"
"testing"

"github.com/golang/mock/gomock"
"github.com/heroku/color"
"github.com/sclevine/spec"
"github.com/sclevine/spec/report"
"github.com/spf13/cobra"

"github.com/buildpacks/pack/internal/commands"
"github.com/buildpacks/pack/internal/commands/testmocks"
"github.com/buildpacks/pack/pkg/logging"
)

func TestManifestAnnotateCommand(t *testing.T) {
color.Disable(true)
defer color.Disable(false)

spec.Run(t, "Commands", testManifestAnnotateCommand, spec.Random(), spec.Report(report.Terminal{}))
}

func testManifestAnnotateCommand(t *testing.T, when spec.G, it spec.S) {
var (
command *cobra.Command
logger *logging.LogWithWriters
outBuf bytes.Buffer
mockController *gomock.Controller
mockClient *testmocks.MockPackClient
)

it.Before(func() {
logger = logging.NewLogWithWriters(&outBuf, &outBuf)
mockController = gomock.NewController(t)
mockClient = testmocks.NewMockPackClient(mockController)

command = commands.ManifestAnnotate(logger, mockClient)
})

when("#ManifestAnnotate", func() {
when("no flags specified", func() {

})
when("when --os flags passed", func() {

})
when("when --arch flags passed", func() {

})
when("when --variant flags passed", func() {

})
when("when --os-version flags passed", func() {

})
when("when --features flags passed", func() {

})
when("when --os-features flags passed", func() {

})
when("when --annotations flags passed", func() {

})
when("when multiple flags passed", func() {

})
when("when no args passed", func() {

})
when("when manifest list reference is incorrect", func() {

})
when("when manifest reference is incorrect", func() {

})
when("when manifest passed in-place of manifest list on first arg", func() {

})
when("when manifest list is passed on second arg", func() {

})
when("when manifest is passed on second arg with --all option", func() {

})
when("when manifest list in locally available", func() {

})
when("when manifest is not locally available", func() {

})
when("when manifest is locally available passed", func() {

})
when("when multiple manifests passed", func() {

})
})
}
Loading
Loading