Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(al2023): cache and retag pause container #2000

Merged
merged 11 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
24 changes: 24 additions & 0 deletions doc/usage/al2023.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
| `launch_block_device_mappings_volume_size` | |
| `nodeadm_build_image` | Image to use as a build environment for nodeadm |
| `nvidia_driver_major_version` | To be used only when ```enable_accelerator = nvidia```. Driver version to install, depends on what is available in NVIDIA repository. |
| `pause_container_image` | Image ref for the pause container image |
| `remote_folder` | Directory path for shell provisioner scripts on the builder instance |
| `runc_version` | |
| `security_group_id` | |
Expand Down Expand Up @@ -67,3 +68,26 @@ One can pass the NVIDIA driver major version using the following:
make k8s=1.29 os_distro=al2023 enable_accelerator=nvidia enable_efa=true nvidia_driver_major_version=560
```
To see which driver versions are available, one can check the NVIDIA AL2023 [repository](https://developer.download.nvidia.com/compute/cuda/repos/amzn2023/).

## Pause Container Image

Since [PR #2000](https://github.com/awslabs/amazon-eks-ami/pull/2000), the pause
container image is no longer pulled in at runtime on AL2023. `nodeadm` uses a
static pause container reference to populate the
[`sandbox_image`](https://github.com/containerd/containerd/blob/963c2160485631c3d4c1068aa28cea34c884a347/docs/cri/config.md?plain=1#L314-L315)
field of the containerd config.toml, which is baked into the AMI during the
build based on the packer template variable `pause_container_image`.

The default for `pause_container_image` points to an EKS-owned ECR image, but
you can also use another pause image entirely such as `registry.k8s.io/pause`.

If you need coordinates for a different ECR image in another region, see
https://docs.aws.amazon.com/eks/latest/userguide/add-ons-images.html for a list
of region-specific registries that hold identical repositories.

Additionally, if you would like to use a FIPS endpoint, this will require just a
minor change to the registry URI:
```diff
-<ACCOUNT_ID>.dkr.ecr.<REGION>.<DOMAIN>"
+<ACCOUNT_ID>.dkr.ecr-fips.<REGION>.<DOMAIN>"
```
7 changes: 1 addition & 6 deletions nodeadm/cmd/nodeadm/init/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ import (
"k8s.io/utils/strings/slices"

"github.com/awslabs/amazon-eks-ami/nodeadm/internal/api"
"github.com/awslabs/amazon-eks-ami/nodeadm/internal/aws/ecr"
"github.com/awslabs/amazon-eks-ami/nodeadm/internal/aws/imds"
"github.com/awslabs/amazon-eks-ami/nodeadm/internal/cli"
"github.com/awslabs/amazon-eks-ami/nodeadm/internal/configprovider"
Expand Down Expand Up @@ -169,12 +168,8 @@ func enrichConfig(log *zap.Logger, cfg *api.NodeConfig) error {
cfg.Status.Instance = *instanceDetails
log.Info("Instance details populated", zap.Reflect("details", instanceDetails))
log.Info("Fetching default options...")
eksRegistry, err := ecr.GetEKSRegistry(instanceDetails.Region)
if err != nil {
return err
}
cfg.Status.Defaults = api.DefaultOptions{
SandboxImage: eksRegistry.GetSandboxImage(),
SandboxImage: "localhost/kubernetes/pause",
}
log.Info("Default options populated", zap.Reflect("defaults", cfg.Status.Defaults))
return nil
Expand Down
7 changes: 0 additions & 7 deletions nodeadm/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,24 @@ require (
github.com/aws/aws-sdk-go-v2/config v1.28.3
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.19
github.com/aws/aws-sdk-go-v2/service/ec2 v1.187.1
github.com/aws/aws-sdk-go-v2/service/ecr v1.36.5
github.com/aws/smithy-go v1.22.0
github.com/containerd/containerd v1.7.23
github.com/coreos/go-systemd/v22 v22.5.0
github.com/integrii/flaggy v1.5.2
github.com/pelletier/go-toml/v2 v2.2.3
github.com/stretchr/testify v1.9.0
go.uber.org/zap v1.27.0
golang.org/x/mod v0.22.0
k8s.io/apimachinery v0.31.2
k8s.io/cri-api v0.31.2
k8s.io/kubelet v0.31.2
sigs.k8s.io/controller-runtime v0.19.1
)

require (
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/x448/float16 v0.8.4 // indirect
golang.org/x/tools v0.22.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240701130421-f6361c86f094 // indirect
google.golang.org/grpc v1.65.0 // indirect
k8s.io/component-base v0.31.2 // indirect
)

Expand All @@ -56,7 +50,6 @@ require (
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
Expand Down
16 changes: 0 additions & 16 deletions nodeadm/go.sum
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s=
dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/aws/aws-sdk-go-v2 v1.32.4 h1:S13INUiTxgrPueTmrm5DZ+MiAo99zYzHEFh1UNkOxNE=
github.com/aws/aws-sdk-go-v2 v1.32.4/go.mod h1:2SK5n0a2karNTv5tbP1SjsX0uhttou00v/HpXKM1ZUo=
github.com/aws/aws-sdk-go-v2/config v1.28.3 h1:kL5uAptPcPKaJ4q0sDUjUIdueO18Q7JDzl64GpVwdOM=
Expand All @@ -18,8 +16,6 @@ github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 h1:VaRN3TlFdd6KxX1x3ILT5ynH6HvK
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
github.com/aws/aws-sdk-go-v2/service/ec2 v1.187.1 h1:g6N2LDa3UuNR8CZvTYuXUKzfCD6S1iqRIsDFkbtwu0Y=
github.com/aws/aws-sdk-go-v2/service/ec2 v1.187.1/go.mod h1:0A17IIeys01WfjDKehspGP+Cyo/YH/eNADIbEbRS9yM=
github.com/aws/aws-sdk-go-v2/service/ecr v1.36.5 h1:FMF/uaTcIdhvOwZXJfzpwanx2m4Dd6IcN4vDnAn7NAA=
github.com/aws/aws-sdk-go-v2/service/ecr v1.36.5/go.mod h1:xhf509Ba+rG5whtO7w46O0raVzu1Og3Aba80LSvHbbQ=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 h1:TToQNkvGguu209puTojY/ozlqy2d/SFNcoLIqTFi42g=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0/go.mod h1:0jp+ltwkf+SwG2fm/PKo8t4y8pJSgOCO4D8Lz3k0aHQ=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.4 h1:tHxQi/XHPK0ctd/wdOw0t7Xrc2OxcRCnVzv8lwWPu0c=
Expand All @@ -38,8 +34,6 @@ github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM
github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/containerd/containerd v1.7.23 h1:H2CClyUkmpKAGlhQp95g2WXHfLYc7whAuvZGBNYOOwQ=
github.com/containerd/containerd v1.7.23/go.mod h1:7QUzfURqZWCZV7RLNEn1XjUCQLEf0bkaK4GjUaZehxw=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
Expand Down Expand Up @@ -71,10 +65,6 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/integrii/flaggy v1.5.2 h1:bWV20MQEngo4hWhno3i5Z9ISPxLPKj9NOGNwTWb/8IQ=
github.com/integrii/flaggy v1.5.2/go.mod h1:dO13u7SYuhk910nayCJ+s1DeAAGC1THCMj1uSFmwtQ8=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
Expand Down Expand Up @@ -165,10 +155,6 @@ golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8T
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240701130421-f6361c86f094 h1:BwIjyKYGsK9dMCBOorzRri8MQwmi7mT9rGHsCEinZkA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240701130421-f6361c86f094/go.mod h1:Ue6ibwXGpU+dqIcODieyLOcgj7z8+IcskoNIgZxtrFY=
google.golang.org/grpc v1.65.0 h1:bs/cUb4lp1G5iImFFd3u5ixQzweKizoZJAwBNLR42lc=
google.golang.org/grpc v1.65.0/go.mod h1:WgYC2ypjlB0EiQi6wdKixMqukr6lBc0Vo+oOgjrM5ZQ=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Expand All @@ -187,8 +173,6 @@ k8s.io/apimachinery v0.31.2 h1:i4vUt2hPK56W6mlT7Ry+AO8eEsyxMD1U44NR22CLTYw=
k8s.io/apimachinery v0.31.2/go.mod h1:rsPdaZJfTfLsNJSQzNHQvYoTmxhoOEofxtOsF3rtsMo=
k8s.io/component-base v0.31.2 h1:Z1J1LIaC0AV+nzcPRFqfK09af6bZ4D1nAOpWsy9owlA=
k8s.io/component-base v0.31.2/go.mod h1:9PeyyFN/drHjtJZMCTkSpQJS3U9OXORnHQqMLDz0sUQ=
k8s.io/cri-api v0.31.2 h1:O/weUnSHvM59nTio0unxIUFyRHMRKkYn96YDILSQKmo=
k8s.io/cri-api v0.31.2/go.mod h1:Po3TMAYH/+KrZabi7QiwQI4a692oZcUOUThd/rqwxrI=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kubelet v0.31.2 h1:6Hytyw4LqWqhgzoi7sPfpDGClu2UfxmPmaiXPC4FRgI=
Expand Down
141 changes: 0 additions & 141 deletions nodeadm/internal/aws/ecr/ecr.go

This file was deleted.

2 changes: 1 addition & 1 deletion nodeadm/internal/containerd/daemon.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ func (cd *containerd) EnsureRunning() error {
}

func (cd *containerd) PostLaunch(c *api.NodeConfig) error {
return cacheSandboxImage(c)
return nil
}

func (cd *containerd) Name() string {
Expand Down
57 changes: 0 additions & 57 deletions nodeadm/internal/containerd/sandbox.go

This file was deleted.

Loading
Loading