China Firewall: Adding Aliyun mirror for preload images and K8s release binaries #12578

denverdino · 2021-09-26T07:47:51Z

This PR is for the proposal in #12535

Alibaba Cloud Container Service team maintain the mirror for the release images for Kubernetes, and also other dependencies for Minikube, e.g. preload images, iso and addon images.

So, it will simplify the Chinese developers to use Minikube without network download issues. E.g.


$ out/minikube start --image-mirror-country='cn'
😄  Darwin 11.6 上的 minikube v1.23.2
✨  自动选择 docker 驱动。其他选项：hyperkit, ssh
✅  正在使用镜像存储库 registry.cn-hangzhou.aliyuncs.com/google_containers
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=3886MB) ...
    > kubeadm.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubelet.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubeadm: 43.71 MiB / 43.71 MiB [---------------] 100.00% 1.42 MiB p/s 31s
    > kubectl: 44.73 MiB / 44.73 MiB [---------------] 100.00% 1.03 MiB p/s 43s
    > kubelet: 146.25 MiB / 146.25 MiB [------------] 100.00% 2.27 MiB p/s 1m5s

    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

$ out/minikube dashboard
🔌  正在开启 dashboard ...
    ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.7
    ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.3.1
🤔  正在验证 dashboard 运行情况 ...
🚀  Launching proxy ...
🤔  正在验证 proxy 运行状况 ...
🎉  Opening http://127.0.0.1:61030/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...

$ out/minikube addons enable ingress
💡  After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"
    ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.0
    ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.0.0-beta.3
    ▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.0
🔎  Verifying ingress addon...
🌟  启动 'ingress' 插件

If new addon is added, just let use know and add the mirror config in "deploy/addons/aliyun_mirror.json"

Pls let us know your feedback for that. Thanks a lot

…netes binaries Signed-off-by: Li Yi <[email protected]>

k8s-ci-robot · 2021-09-26T07:47:58Z

Hi @denverdino. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

minikube-bot · 2021-09-26T07:48:03Z

Can one of the admins verify this patch?

…netes binaries Signed-off-by: Li Yi <[email protected]>

deploy/addons/aliyun_mirror.json

Signed-off-by: Li Yi <[email protected]>

medyagh · 2021-10-18T19:12:22Z

.github/workflows/sync-minikube.yml

+
+    - name: Synchronize images to Alibaba Cloud Container Registry Service
+      env: 
+        ACR_USER: ${{ secrets.ACR_USER }}


maybe use a more unique name

like ALIBABA_ACR_USER

and same for other secrets

and then if u share me those secrets I will add them to to the repo

medyagh · 2021-10-25T22:07:29Z

/ok-to-test

k8s-ci-robot · 2021-10-25T22:07:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: denverdino, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [medyagh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

minikube-pr-bot · 2021-10-25T23:41:22Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12578) |
+----------------+----------+---------------------+
| minikube start | 47.7s    | 47.0s               |
| enable ingress | 31.2s    | 32.0s               |
+----------------+----------+---------------------+

Times for minikube start: 47.5s 48.8s 47.0s 47.7s 47.8s
Times for minikube (PR 12578) start: 47.5s 46.8s 46.3s 48.8s 45.5s

Times for minikube (PR 12578) ingress: 32.2s 31.7s 31.8s 32.2s 32.3s
Times for minikube ingress: 31.3s 31.2s 31.2s 31.8s 30.4s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12578) |
+----------------+----------+---------------------+
| minikube start | 21.9s    | 21.8s               |
| enable ingress | 30.6s    | 27.9s               |
+----------------+----------+---------------------+

Times for minikube ingress: 30.9s 34.4s 34.5s 26.5s 26.9s
Times for minikube (PR 12578) ingress: 25.9s 26.5s 26.9s 33.9s 26.4s

Times for minikube (PR 12578) start: 21.4s 21.9s 21.2s 21.7s 22.6s
Times for minikube start: 22.9s 21.1s 22.5s 21.3s 21.9s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12578) |
+----------------+----------+---------------------+
| minikube start | 37.2s    | 39.2s               |
| enable ingress | 32.0s    | 29.0s               |
+----------------+----------+---------------------+

Times for minikube (PR 12578) start: 29.1s 29.8s 44.8s 48.4s 43.8s
Times for minikube start: 27.5s 44.0s 44.3s 26.0s 44.0s

Times for minikube (PR 12578) ingress: 29.4s 27.4s 27.4s 28.4s 32.4s
Times for minikube ingress: 32.5s 32.9s 32.9s 33.4s 27.9s

minikube-pr-bot · 2021-10-26T03:32:22Z

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
KVM_Linux	TestStartStop/group/old-k8s-version/serial/Pause (gopogh)	0.00 (chart)
Docker_Linux	TestFunctional/serial/ComponentHealth (gopogh)	6.29 (chart)
Docker_Linux	TestStartStop/group/old-k8s-version/serial/Pause (gopogh)	7.69 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/calico/Start (gopogh)	11.88 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/cilium/Start (gopogh)	12.87 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/custom-weave/Start (gopogh)	12.87 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/false/Start (gopogh)	14.85 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/flannel/Start (gopogh)	15.84 (chart)
Docker_Linux_containerd	TestPause/serial/Pause (gopogh)	16.08 (chart)
Docker_Linux_containerd	TestPause/serial/VerifyStatus (gopogh)	16.08 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/bridge/Start (gopogh)	18.81 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/kubenet/Start (gopogh)	19.80 (chart)
KVM_Linux	TestAddons/serial/GCPAuth (gopogh)	36.64 (chart)
Docker_Linux_containerd	TestAddons/serial/GCPAuth (gopogh)	41.96 (chart)
Docker_Linux	TestAddons/serial/GCPAuth (gopogh)	41.96 (chart)
KVM_Linux_containerd	TestAddons/serial/GCPAuth (gopogh)	45.45 (chart)
Docker_Linux_containerd	TestScheduledStopUnix (gopogh)	100.00 (chart)
Hyper-V_Windows	TestDownloadOnly/v1.14.0/preload-exists (gopogh)	100.00 (chart)
Hyper-V_Windows	TestDownloadOnly/v1.22.2/preload-exists (gopogh)	100.00 (chart)
Hyper-V_Windows	TestDownloadOnly/v1.22.3-rc.0/preload-exists (gopogh)	100.00 (chart)
Hyper-V_Windows	TestMultiNode/serial/PingHostFrom2Pods (gopogh)	100.00 (chart)
Hyper-V_Windows	TestMultiNode/serial/RestartKeepsNodes (gopogh)	100.00 (chart)
Hyper-V_Windows	TestRunningBinaryUpgrade (gopogh)	100.00 (chart)
Hyper-V_Windows	TestStoppedBinaryUpgrade/Upgrade (gopogh)	100.00 (chart)

To see the flake rates of all tests by environment, click here.

Initial support for adding Aliyun mirror for preload images and Kuber…

eb66c67

…netes binaries Signed-off-by: Li Yi <[email protected]>

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 26, 2021

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 26, 2021

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 26, 2021

k8s-ci-robot requested review from afbjorklund and medyagh September 26, 2021 07:48

denverdino added 2 commits September 26, 2021 16:04

Initial support for adding Aliyun mirror for preload images and Kuber…

9ae94cf

…netes binaries Signed-off-by: Li Yi <[email protected]>

Fix for boilerplate

f1d5674

medyagh requested changes Sep 27, 2021

View reviewed changes

deploy/addons/aliyun_mirror.json Show resolved Hide resolved

spowelljr added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 5, 2021

denverdino mentioned this pull request Oct 6, 2021

Host preload image on a site accessible from China: Get Alibaba account for minikube #7695

Open

Add github action for synchronization

d38b244

Signed-off-by: Li Yi <[email protected]>

medyagh reviewed Oct 18, 2021

View reviewed changes

Refine the name of secrets

cea5aa2

medyagh approved these changes Oct 25, 2021

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 25, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 25, 2021

medyagh merged commit 6f72fc9 into kubernetes:master Oct 26, 2021

medyagh changed the title ~~Initial support for adding Aliyun mirror for preload images and K8s release binaries~~ China Firewall: Adding official Aliyun mirror for preload images and K8s release binaries Oct 26, 2021

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 26, 2021

medyagh changed the title ~~China Firewall: Adding official Aliyun mirror for preload images and K8s release binaries~~ China Firewall: Adding Aliyun mirror for preload images and K8s release binaries Oct 26, 2021

denverdino mentioned this pull request Nov 2, 2021

Include registry mirror if specified in addon template yamls #10012

Closed

denverdino mentioned this pull request Nov 10, 2021

Proposal for using mirror image for Chinese developer with Alibaba Cloud Registry Service #12535

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

China Firewall: Adding Aliyun mirror for preload images and K8s release binaries #12578

China Firewall: Adding Aliyun mirror for preload images and K8s release binaries #12578

denverdino commented Sep 26, 2021

k8s-ci-robot commented Sep 26, 2021

minikube-bot commented Sep 26, 2021

medyagh Oct 18, 2021

medyagh commented Oct 25, 2021

k8s-ci-robot commented Oct 25, 2021

minikube-pr-bot commented Oct 25, 2021

minikube-pr-bot commented Oct 26, 2021

China Firewall: Adding Aliyun mirror for preload images and K8s release binaries #12578

China Firewall: Adding Aliyun mirror for preload images and K8s release binaries #12578

Conversation

denverdino commented Sep 26, 2021

k8s-ci-robot commented Sep 26, 2021

minikube-bot commented Sep 26, 2021

medyagh Oct 18, 2021

Choose a reason for hiding this comment

medyagh commented Oct 25, 2021

k8s-ci-robot commented Oct 25, 2021

minikube-pr-bot commented Oct 25, 2021

minikube-pr-bot commented Oct 26, 2021