Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use crictl to pull sandbox image #1605

Merged
merged 1 commit into from
Feb 1, 2024
Merged

Use crictl to pull sandbox image #1605

merged 1 commit into from
Feb 1, 2024

Conversation

cartermckinnon
Copy link
Member

@cartermckinnon cartermckinnon commented Jan 31, 2024

Issue #, if available:

Fixes #1597 on containerd 1.7.3+.

Description of changes:

The containerd CRI server will only report an image as "pinned" if it is pulled with the CRI server. The pinned label will not be applied if it is pulled using the bare containerd server with ctr.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Testing Done

On containerd-1.7.11 with sandbox_image = "registry.k8s.io/pause:3.9":

> sudo ctr --namespace k8s.io image pull registry.k8s.io/pause:3.9
> sudo crictl inspecti registry.k8s.io/pause:3.9 | grep pinned
    "pinned": false

> sudo crictl rmi --all
Deleted: registry.k8s.io/pause:3.9

> sudo crictl pull registry.k8s.io/pause:3.9
Image is up to date for sha256:e6f1816883972d4be47bd48879a08919b96afcd344132622e4d444987919323c
> sudo crictl inspecti registry.k8s.io/pause:3.9 | grep pinned
    "pinned": true

@@ -8,4 +8,9 @@ if [[ "$(sudo ctr --namespace k8s.io image ls | grep $sandbox_image)" != "" ]];
exit 0
fi

/etc/eks/containerd/pull-image.sh "${sandbox_image}"
ecr_password=$(aws ecr get-login-password)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there is a problem with getting the password, will the timer (in the other PR) fail and then fail to re-fire again in the next time period?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timer will trigger the unit whether it failed or succeeded on previous runs, per my testing.

I copied over the retry logic we have in the other pull-image.sh script anyway.

@dims
Copy link
Member

dims commented Jan 31, 2024

cc @henry118

@@ -8,4 +8,26 @@ if [[ "$(sudo ctr --namespace k8s.io image ls | grep $sandbox_image)" != "" ]];
exit 0
fi

/etc/eks/containerd/pull-image.sh "${sandbox_image}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to modify the pull-image.sh script itself to use crictl instead of ctr?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use ctr content fetch in that script because it skips image unpacking (an important optimization for the image caching feature's disk usage).

echo >&2 "Unable to retrieve the ECR password."
exit 1
fi
sudo retry crictl pull "${sandbox_image}" --creds AWS:${ecr_password}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked my node and crictl wasn't there. I think you need to install it as well.

Copy link
Member Author

@cartermckinnon cartermckinnon Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, missed in my commit 😓 . We mulled over adding cri-tools in the past because we aren't the maintainers of that package in AL. We don't have much of a choice now; the only other option would be manually labelling the image as pinned after we pull it with ctr, which seems more fragile (the label key or something could change).

@awslabs awslabs deleted a comment from github-actions bot Jan 31, 2024
@awslabs awslabs deleted a comment from github-actions bot Jan 31, 2024
@cartermckinnon
Copy link
Member Author

/ci

Copy link
Contributor

@cartermckinnon roger that! I've dispatched a workflow. 👍

Copy link
Member

@henry118 henry118 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cartermckinnon
Copy link
Member Author

/ci

Copy link
Contributor

github-actions bot commented Feb 1, 2024

@cartermckinnon roger that! I've dispatched a workflow. 👍

Copy link
Contributor

@tzneal tzneal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wwvela wwvela merged commit 7fa037a into master Feb 1, 2024
2 checks passed
@wwvela wwvela deleted the crictl-pull-sandbox branch February 1, 2024 19:32
Copy link
Contributor

github-actions bot commented Feb 1, 2024

@cartermckinnon the workflow that you requested has completed. 🎉

Kubernetes versionBuildLaunchTest
1.23success ✅success ✅success ✅
1.24success ✅success ✅success ✅
1.25failure ❌skipped ⏭️skipped ⏭️
1.26success ✅success ✅success ✅
1.27success ✅success ✅success ✅
1.28success ✅success ✅success ✅
1.29success ✅success ✅success ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sandbox container image being GC'd in 1.29
5 participants