Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure downloading preload images when size is zero #15256

Merged
merged 2 commits into from
Nov 4, 2022
Merged

Ensure downloading preload images when size is zero #15256

merged 2 commits into from
Nov 4, 2022

Conversation

spowelljr
Copy link
Member

@spowelljr spowelljr commented Oct 31, 2022 •
edited
Loading

Fixes #15237

Problem:
We can get in the situation were the cached preload file exists, but the file is empty (size 0). Only the existence of the file is checked, not the size, which results in the preload not being downloaded when it should.

Before (preload is not downloaded):

$ minikube start --download-only
😄  minikube v1.27.1 on Darwin 12.6.1 (arm64)
✨  Automatically selected the docker driver. Other choices: ssh, qemu2 (experimental)
📌  Using Docker Desktop driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.25.3 preload ...
    > preloaded-images-k8s-v18-v1...:  320.81 MiB / 320.81 MiB  100.00% 5.97 Mi
    > kubectl.sha256:  64 B / 64 B [-------------------------] 100.00% ? p/s 0s
    > kubectl:  47.38 MiB / 47.38 MiB [-------------] 100.00% 9.49 MiB p/s 5.2s
    > gcr.io/k8s-minikube/kicbase...:  347.52 MiB / 347.52 MiB  100.00% 5.31 Mi
✅  Download complete!
$ rm ~/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.3-docker-overlay2-arm64.tar.lz4
$ touch ~/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.3-docker-overlay2-arm64.tar.lz4
$ minikube start
😄  minikube v1.27.1 on Darwin 12.6.1 (arm64)
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
    > gcr.io/k8s-minikube/kicbase...:  0 B [_____________________] ?% ? p/s 41s
🔥  Creating docker container (CPUs=2, Memory=1988MB) ...
🐳  Preparing Kubernetes v1.25.3 on Docker 20.10.20 ...
❌  Unable to load cached images: loading cached images: stat /Users/powellsteven/.minikube/cache/images/arm64/registry.k8s.io/pause_3.8: no such file or directory
    > kubelet.sha256:  64 B / 64 B [-------------------------] 100.00% ? p/s 0s
    > kubectl.sha256:  64 B / 64 B [-------------------------] 100.00% ? p/s 0s
    > kubeadm.sha256:  64 B / 64 B [-------------------------] 100.00% ? p/s 0s
    > kubeadm:  40.56 MiB / 40.56 MiB [--------------] 100.00% 3.52 MiB p/s 12s
    > kubectl:  41.56 MiB / 41.56 MiB [--------------] 100.00% 3.42 MiB p/s 12s
    > kubelet:  104.74 MiB / 104.74 MiB [------------] 100.00% 6.13 MiB p/s 17s
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

After (preload is downloaded):

$ minikube start --download-only
😄  minikube v1.27.1 on Darwin 12.6.1 (arm64)
✨  Automatically selected the docker driver. Other choices: ssh, qemu2 (experimental)
📌  Using Docker Desktop driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.25.3 preload ...
    > preloaded-images-k8s-v18-v1...:  320.81 MiB / 320.81 MiB  100.00% 5.97 Mi
    > kubectl.sha256:  64 B / 64 B [-------------------------] 100.00% ? p/s 0s
    > kubectl:  47.38 MiB / 47.38 MiB [-------------] 100.00% 9.49 MiB p/s 5.2s
    > gcr.io/k8s-minikube/kicbase...:  347.52 MiB / 347.52 MiB  100.00% 5.31 Mi
✅  Download complete!
$ rm ~/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.3-docker-overlay2-arm64.tar.lz4
$ touch ~/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.3-docker-overlay2-arm64.tar.lz4
$ minikube start
😄  minikube v1.27.1 on Darwin 12.6.1 (arm64)
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.25.3 preload ...
    > preloaded-images-k8s-v18-v1...:  320.81 MiB / 320.81 MiB  100.00% 9.92 Mi
    > gcr.io/k8s-minikube/kicbase...:  0 B [_____________________] ?% ? p/s 42s
🔥  Creating docker container (CPUs=2, Memory=1988MB) ...
🐳  Preparing Kubernetes v1.25.3 on Docker 20.10.20 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 31, 2022
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 31, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@medyagh
Copy link
Member

medyagh commented Nov 1, 2022

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Nov 1, 2022
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15256) |
+----------------+----------+---------------------+
| minikube start | 55.4s    | 56.1s               |
| enable ingress | 28.9s    | 28.3s               |
+----------------+----------+---------------------+

Times for minikube start: 54.9s 53.4s 56.6s 54.8s 57.0s
Times for minikube (PR 15256) start: 57.8s 56.2s 54.0s 55.6s 56.7s

Times for minikube ingress: 28.8s 29.3s 28.8s 28.8s 28.8s
Times for minikube (PR 15256) ingress: 28.8s 27.8s 28.3s 27.8s 28.8s

docker driver with docker runtime
error collecting results for docker driver: timing run 0 with minikube: timing cmd: [out/minikube addons enable ingress]: waiting for minikube: exit status 10
docker driver with containerd runtime
error downloading artifacts: artifact download start: exit status 90

@medyagh medyagh changed the title Check size of preload Ensure downloading preload images when size is zero Nov 1, 2022
@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Hyper-V_Windows TestPause/serial/PauseAgain (gopogh) 2.04 (chart)
Hyper-V_Windows TestNetworkPlugins/group/enable-default-cni/Start (gopogh) 3.85 (chart)
Hyper-V_Windows TestNetworkPlugins/group/flannel/Start (gopogh) 3.85 (chart)
Hyper-V_Windows TestNetworkPlugins/group/calico/Start (gopogh) 7.69 (chart)
Hyper-V_Windows TestNetworkPlugins/group/bridge/Start (gopogh) 10.00 (chart)
Hyper-V_Windows TestNetworkPlugins/group/kubenet/Start (gopogh) 16.92 (chart)
Docker_macOS TestStartStop/group/newest-cni/serial/Pause (gopogh) 21.59 (chart)
Hyperkit_macOS TestPause/serial/SecondStartNoReconfiguration (gopogh) 27.78 (chart)
Docker_macOS TestMultiNode/serial/RestartMultiNode (gopogh) 28.26 (chart)
Docker_macOS TestMultiNode/serial/RestartKeepsNodes (gopogh) 32.61 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 34.23 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 34.23 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 34.82 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 34.82 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/UserAppExistsAfterStop (gopogh) 37.84 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/FirstStart (gopogh) 38.39 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/DeployApp (gopogh) 38.94 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/SecondStart (gopogh) 38.94 (chart)
KVM_Linux TestPause/serial/SecondStartNoReconfiguration (gopogh) 43.59 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/UserAppExistsAfterStop (gopogh) 46.85 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/FirstStart (gopogh) 48.25 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/SecondStart (gopogh) 48.25 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/DeployApp (gopogh) 48.70 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 66.67 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/bridge/DNS (gopogh) 72.73 (chart)
Docker_Linux TestNetworkPlugins/group/calico/Start (gopogh) 79.02 (chart)
Docker_Linux TestNetworkPlugins/group/bridge/DNS (gopogh) 81.82 (chart)
Docker_Linux TestNetworkPlugins/group/false/DNS (gopogh) 82.52 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/calico/Start (gopogh) 82.88 (chart)
Docker_Linux TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 83.22 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@spowelljr spowelljr merged commit aa99d88 into kubernetes:master Nov 4, 2022
@spowelljr spowelljr deleted the checkForPreloadSize branch November 4, 2022 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@medyagh medyagh medyagh approved these changes

@afbjorklund afbjorklund Awaiting requested review from afbjorklund

@prezha prezha Awaiting requested review from prezha

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unable to load cached images
4 participants
@spowelljr @k8s-ci-robot @medyagh @minikube-pr-bot