Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e job: load image in kind #214

Merged
merged 1 commit into from
Mar 15, 2024
Merged

Conversation

LionelJouin
Copy link
Collaborator

@LionelJouin LionelJouin commented Mar 12, 2024

What this PR does / why we need it:

The Multus e2e environment is no longer using the local registry. The image is now loaded. The tag has been set to e2e so there is no conflict if the ImagePullPolicy is not set

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer (optional):

@LionelJouin
Copy link
Collaborator Author

The issue is related to the registry:

2024-03-13T08:08:55.9382392Z   Normal   Scheduled       2s    default-scheduler  Successfully assigned kube-system/dynamic-networks-controller-ds-wg2hc to kind-worker2
2024-03-13T08:08:55.9384146Z   Normal   AddedInterface  1s    multus             Add eth0 [10.244.2.2/24] from kindnet
2024-03-13T08:08:55.9386121Z   Normal   Pulling         1s    kubelet            Pulling image "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest"
2024-03-13T08:08:55.9392785Z   Warning  Failed          1s    kubelet            Failed to pull image "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest": failed to pull and unpack image "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest": failed to resolve reference "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest": failed to do request: Head "http://localhost:5000/v2/k8snetworkplumbingwg/multus-dynamic-networks-controller/manifests/latest": dial tcp [::1]:5000: connect: connection refused
2024-03-13T08:08:55.9397578Z   Warning  Failed          1s    kubelet            Error: ErrImagePull
2024-03-13T08:08:55.9399164Z   Normal   BackOff         0s    kubelet            Back-off pulling image "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest"
2024-03-13T08:08:55.9400283Z   Warning  Failed          0s    kubelet            Error: ImagePullBackOf

https://github.com/k8snetworkplumbingwg/multus-dynamic-networks-controller/actions/runs/8261278179/job/22598271008?pr=214

@LionelJouin
Copy link
Collaborator Author

The local registry is no longer used in Multus, the image are now loaded: k8snetworkplumbingwg/multus-cni#1230

@LionelJouin LionelJouin force-pushed the ci branch 3 times, most recently from 422be31 to 37bdd17 Compare March 13, 2024 10:12
@maiqueb
Copy link
Collaborator

maiqueb commented Mar 13, 2024

The local registry is no longer used in Multus, the image are now loaded: k8snetworkplumbingwg/multus-cni#1230

I see you're hitting this error:

  Warning  Failed          2s               kubelet            Failed to pull image "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest": failed to pull and unpack image "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest": failed to resolve reference "localhost:5000/k8snetworkplumbingwg/multus-dynamic-networks-controller:latest": failed to do request: Head "http://localhost:5000/v2/k8snetworkplumbingwg/multus-dynamic-networks-controller/manifests/latest": dial tcp [::1]:5000: connect: connection refused

IIUC, the local registry is no longer used - right ? - thus, you might need to adjust the name of the images (removing the local registry part - i.e. localhost:5000 - for this to work.

Just a wild guess.

@LionelJouin LionelJouin force-pushed the ci branch 4 times, most recently from bcadfb7 to 641d367 Compare March 14, 2024 18:47
@LionelJouin
Copy link
Collaborator Author

Right, the local registry is no longer used.
The problem I faced was the image tag, the imagePullPolicy is not set and the tag is latest, so the imagePullPolicy was set to Always
https://kubernetes.io/docs/concepts/containers/images/#required-image-pull

The Multus e2e environment is no longer using the local registry. The
image is now loaded. The tag has been set to e2e so there is no conflict
if the ImagePullPolicy is not set
@LionelJouin LionelJouin changed the title Increase timeout in e2e job e2e job: load image in kind Mar 14, 2024
@LionelJouin LionelJouin marked this pull request as ready for review March 14, 2024 19:39
@maiqueb
Copy link
Collaborator

maiqueb commented Mar 15, 2024

Right, the local registry is no longer used. The problem I faced was the image tag, the imagePullPolicy is not set and the tag is latest, so the imagePullPolicy was set to Always https://kubernetes.io/docs/concepts/containers/images/#required-image-pull

Wait, I don't follow.

I assume that create cluster script is used to also experiment locally; thus, compiling the code and re-deploying just this component is something I had in mind. Hence, IMO, always pulling makes sense.

Maybe there's something I'm missing here ... Could you elaborate on why it fails because of the pull policy ? That's not immediately clear to me.

@LionelJouin
Copy link
Collaborator Author

From what I understood, When the image is loaded via kind load docker-image and you use Always as imagePullPolicy, Kubernetes will try to pull the image again and will not use the local one. To use the local one loaded in Kind, you need to use IfNotPresent (which is not the default value if imagePullPolicy is not set and if the image tag latest).

Here is a blog describing the issue: https://iximiuz.com/en/posts/kubernetes-kind-load-docker-image/

Copy link
Collaborator

@maiqueb maiqueb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get one in now to unlock CI.

We'll figure out the local workflow (re-deploy new versions of the daemonset later) if it turns out to be a problem (I expect all will be OK if we just load another updated version into the cluster).

@maiqueb maiqueb merged commit 808c485 into k8snetworkplumbingwg:main Mar 15, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants