Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disable digest resolution for manifest v2 schema 1 manifests #4249

Merged
merged 2 commits into from
Jul 23, 2024

Conversation

Yiyuanzzz
Copy link
Contributor

@Yiyuanzzz Yiyuanzzz commented Jul 19, 2024

Summary

ECS Agent will disable the digest resolution for Schema 1 manifest as a workaround for a bug in ECR that ECR will reports an incorrect digest during digest resolution phase for Schema V1 manifests which then causes image pull using the digest to fail.

Implementation details

skip digest resolution for manifest v2 schema 1 manifests

Testing

Reproduce the issue by setting Up Pull-Through Cache Rule

aws ecr create-pull-through-cache-rule \
     --ecr-repository-prefix quay \
     --upstream-registry-url quay.io \
     --region us-west-2
docker pull  130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4

Manually ran a task with schema 1 image 130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4, we can see error " image verification failed for digest "

level=debug time=2024-07-22T05:06:56Z msg="Setting image digest on container" taskARN="arn:aws:ecs:us-west-2:130669029530:task/tssbug/ca80402ec1ff44e398c7a88bef4bef7f" containerName="container" imageDigest="sha256:d1f9e51bf7f3f7940faf11553d08f8815d2f0f999f6eafd6a6b8f765adba5895"
level=error time=2024-07-22T05:07:07Z msg="DockerGoClient: failed to pull image 130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4@sha256:d1f9e51bf7f3f7940faf11553d08f8815d2f0f999f6eafd6a6b8f765adba5895: [CannotPullContainerError] image verification failed for digest sha256:d1f9e51bf7f3f7940faf11553d08f8815d2f0f999f6eafd6a6b8f765adba5895" module=docker_client.go

Manually ran a task with schema 1 image 130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4 with test agent(changes of this pr), we can see digest resolution is skipped and pulling image complete.

level=info time=2024-07-22T05:14:24Z msg="skipping digest resolution for manifest v2 schema 1," taskARN="arn:aws:ecs:us-west-2:130669029530:task/default/eb9cf2168d9c406388966b6e2584dae5" containerName="container" ImageMediatype="application/vnd.docker.distribution.manifest.v1+prettyjws" image="130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4"
l

level=debug time=2024-07-22T05:14:25Z msg="DockerGoClient: pulling image 130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4, status Status: Downloaded newer image for 130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4" module=docker_client.go
level=debug time=2024-07-22T05:14:25Z msg="DockerGoClient: pulling image complete: 130669029530.dkr.ecr.us-west-2.amazonaws.com/quay/coreos/etcd:v2.0.4" module=docker_client.go

New tests cover the changes:

Description for the changelog

Bugfix: Disable Digest Resolution for Manifest V2 Schema 1

Does this PR include breaking model changes? If so, Have you added transformation functions?

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@Yiyuanzzz Yiyuanzzz requested a review from a team as a code owner July 19, 2024 18:39
@Yiyuanzzz Yiyuanzzz changed the title [wip]diable digest resolution for manifest v2 schema 1 manifests diable digest resolution for manifest v2 schema 1 manifests Jul 19, 2024
@Yiyuanzzz Yiyuanzzz changed the title diable digest resolution for manifest v2 schema 1 manifests disable digest resolution for manifest v2 schema 1 manifests Jul 19, 2024
field.ImageMediatype: imageManifestMediatype,
field.Image: container.Image,
})
if strings.Contains(imageManifestMediatype, "application/vnd.docker.distribution.manifest.v1") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's create constants for the two possible media type values for Schema 1 and use the constants here.

Kinda like this
https://github.com/distribution/distribution/blob/release/2.8/manifest/schema1/manifest.go#L14-L18

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! updated

agent/engine/docker_task_engine_test.go Outdated Show resolved Hide resolved
ecs-agent/logger/field/constants.go Outdated Show resolved Hide resolved
field.ImageMediaType: imageManifestMediaType,
field.Image: container.Image,
})
if strings.Contains(imageManifestMediaType, mediaTypeManifestV1) || strings.Contains(imageManifestMediaType, mediaTypeSignedManifestV1) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to do strings.Contains instead of just ==?

@Yiyuanzzz Yiyuanzzz merged commit d985688 into aws:dev Jul 23, 2024
40 checks passed
@amogh09 amogh09 mentioned this pull request Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants