Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

config/jobs: use community hosted images for more jobs #23587

Merged
merged 9 commits into from
Sep 14, 2021
Merged

config/jobs: use community hosted images for more jobs #23587

merged 9 commits into from
Sep 14, 2021

Conversation

spiffxp
Copy link
Member

@spiffxp spiffxp commented Sep 13, 2021

Related:

The first commits are image-by-image search-replace with some manual inspection to add any leftovers

The second-to-last commit removes the cluster-api image as it's been subsumed by kubekins-e2e since 2019

The last commit updates experiment/print-job-image-summary.sh, output as of this PR is now

# images used by prowjobs on prow.k8s.io
- total /                      2395 total, 56 unique
  - gcr.io /                   2257 total, 43 unique
    - kubernetes.io gcp org
      - k8s-staging-test-infra 2102 total, 7 unique
        - kubekins-e2e         1850 total, 1 unique
        - image-builder        167 total, 1 unique
        - krte                 68 total, 1 unique
        - bootstrap            8 total, 1 unique
        - gcloud-in-go         6 total, 1 unique
        - bigquery             2 total, 1 unique
        - triage               1 total, 1 unique
      - k8s-staging-the_rest   52 total, 8 unique
      - k8s.gcr.io             14 total, 5 unique
    - google.com gcp org
      - k8s-prow               17 total, 7 unique
      - k8s-testimages         50 total, 6 unique
        - gcloud-bazel         22 total, 1 unique
        - bazelbuild           5 total, 1 unique
        - bazel                5 total, 1 unique
        - benchmarkjunit       4 total, 1 unique
        - gubernator           1 total, 1 unique
        - e2e-kubemci          1 total, 1 unique
    - other (unsure which org) 22 total, 10 unique
      - gcloud-bazel           23 total, 2 unique
      - bazel                  21 total, 3 unique
      - bazel                  21 total, 3 unique
      - cip                    5 total, 1 unique
      - shellcheck             3 total, 1 unique
      - mdlint                 3 total, 1 unique
      - octodns                2 total, 1 unique
      - ci                     2 total, 1 unique
      - prow-test              1 total, 1 unique
      - minikube-e2e           1 total, 1 unique
  - not gcr.io /               138 total, 13 unique
    - dockerhub /              134 total, 9 unique
    - quay.io /                4 total, 4 unique

@spiffxp
config/jobs: use k8s-staging-test-infra/bigquery image
7e3d580 
affects:

- metrics-kettle
- metrics-bigquery
@spiffxp
config/jobs: use k8s-staging-test-infra/bootstrap
5a1eb99 
affected jobs:
- ci-cadvisor-canarypush
- pull-kubernetes-cross
- ci-kubernetes-build-1-19-deprecated
- ci-kubernetes-build-1-20-deprecated
- ci-kubernetes-build-deprecated-1-21
- ci-kubernetes-build-deprecated-1-22
- ci-build-and-push-k8s-at-golang-tip
- ci-build-and-push-k8s-at-golang-tip-canary
- pull-perf-tests-util-images

affected images:
- kubekins-e2e (when rebuilt)
@spiffxp
config/jobs: use k8s-staging-test-infra/gcloud-in-go
7e81759
@spiffxp
config/jobs: remove outdated gcloud-in-go reference in README
3b2a155 
pull-kubernetes-verify now uses regular golang, not gcloud-in-go
@spiffxp
config/jobs: use k8s-staging-test-infra/image-builder
7b2b888
@spiffxp
config/jobs: use k8s-staging-test-infra/krte
e6657cd 
gcr.io/k8s-staging-test-infra/krte:v20210204-b06ec78-1.17 was manually
pushed to the staging repo from k8s-testimages, everything else was
search-replace to use the new/equivalent tag
@spiffxp
config/jobs: migrate a k8s-testimages/kubekins-e2e job
81d1ec4
@spiffxp
images: remove cluster-api
724b878 
Jobs that used it were gradually replaced with jobs that used
kubekins-e2e instead, around 2019.  See:

- d48d7f5
- 5b845a6
- 04a23a4
@spiffxp
experiment: update print-job-image-summary.sh
5fad4e0
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. area/config Issues or PRs related to code in /config area/conformance Issues or PRs related to kubernetes conformance tests area/images area/jobs area/release-eng Issues or PRs related to the Release Engineering subproject sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Sep 13, 2021
@spiffxp spiffxp mentioned this pull request Sep 13, 2021
Open
77 tasks
@spiffxp
Copy link
Member Author

spiffxp commented Sep 13, 2021

/cc @dims @ameukam

@dims
Copy link
Member

dims commented Sep 13, 2021

/approve

giving @ameukam a chance to /lgtm

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ameukam
Copy link
Member

ameukam commented Sep 14, 2021

mostly find & replace
/lgtm
/hold
Remove hold when ready.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 14, 2021
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 14, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Sep 14, 2021

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 14, 2021
@k8s-ci-robot k8s-ci-robot merged commit 9d13210 into kubernetes:master Sep 14, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.23 milestone Sep 14, 2021
@k8s-ci-robot
Copy link
Contributor

@spiffxp: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

  • key cadvisor.yaml using file config/jobs/cadvisor/cadvisor.yaml
  • key k8s-staging-apiserver-network-proxy.yaml using file config/jobs/image-pushing/k8s-staging-apiserver-network-proxy.yaml
  • key k8s-staging-apisnoop.yaml using file config/jobs/image-pushing/k8s-staging-apisnoop.yaml
  • key k8s-staging-boskos.yaml using file config/jobs/image-pushing/k8s-staging-boskos.yaml
  • key k8s-staging-cloud-provider-aws.yaml using file config/jobs/image-pushing/k8s-staging-cloud-provider-aws.yaml
  • key k8s-staging-cloud-provider-gcp.yaml using file config/jobs/image-pushing/k8s-staging-cloud-provider-gcp.yaml
  • key k8s-staging-cluster-addons.yaml using file config/jobs/image-pushing/k8s-staging-cluster-addons.yaml
  • key k8s-staging-cluster-api.yaml using file config/jobs/image-pushing/k8s-staging-cluster-api.yaml
  • key k8s-staging-cri-tools.yaml using file config/jobs/image-pushing/k8s-staging-cri-tools.yaml
  • key k8s-staging-csi-secrets-store.yaml using file config/jobs/image-pushing/k8s-staging-csi-secrets-store.yaml
  • key k8s-staging-descheduler.yaml using file config/jobs/image-pushing/k8s-staging-descheduler.yaml
  • key k8s-staging-dns.yaml using file config/jobs/image-pushing/k8s-staging-dns.yaml
  • key k8s-staging-e2e-test-images.yaml using file config/jobs/image-pushing/k8s-staging-e2e-test-images.yaml
  • key k8s-staging-etcdadm.yaml using file config/jobs/image-pushing/k8s-staging-etcdadm.yaml
  • key k8s-staging-external-dns.yaml using file config/jobs/image-pushing/k8s-staging-external-dns.yaml
  • key k8s-staging-gateway-api.yaml using file config/jobs/image-pushing/k8s-staging-gateway-api.yaml
  • key k8s-staging-infra-tools.yaml using file config/jobs/image-pushing/k8s-staging-infra-tools.yaml
  • key k8s-staging-ingress-controller-conformance.yaml using file config/jobs/image-pushing/k8s-staging-ingress-controller-conformance.yaml
  • key k8s-staging-ingress-nginx.yaml using file config/jobs/image-pushing/k8s-staging-ingress-nginx.yaml
  • key k8s-staging-k8s-gsm-tools.yaml using file config/jobs/image-pushing/k8s-staging-k8s-gsm-tools.yaml
  • key k8s-staging-kind.yaml using file config/jobs/image-pushing/k8s-staging-kind.yaml
  • key k8s-staging-kops.yaml using file config/jobs/image-pushing/k8s-staging-kops.yaml
  • key k8s-staging-kube-state-metrics.yaml using file config/jobs/image-pushing/k8s-staging-kube-state-metrics.yaml
  • key k8s-staging-kubetest2.yaml using file config/jobs/image-pushing/k8s-staging-kubetest2.yaml
  • key k8s-staging-kustomize.yaml using file config/jobs/image-pushing/k8s-staging-kustomize.yaml
  • key k8s-staging-metrics-server.yaml using file config/jobs/image-pushing/k8s-staging-metrics-server.yaml
  • key k8s-staging-multitenancy.yaml using file config/jobs/image-pushing/k8s-staging-multitenancy.yaml
  • key k8s-staging-nfd-operator.yaml using file config/jobs/image-pushing/k8s-staging-nfd-operator.yaml
  • key k8s-staging-nfd.yaml using file config/jobs/image-pushing/k8s-staging-nfd.yaml
  • key k8s-staging-node-problem-detector.yaml using file config/jobs/image-pushing/k8s-staging-node-problem-detector.yaml
  • key k8s-staging-prometheus-adapter.yaml using file config/jobs/image-pushing/k8s-staging-prometheus-adapter.yaml
  • key k8s-staging-provider-azure.yaml using file config/jobs/image-pushing/k8s-staging-provider-azure.yaml
  • key k8s-staging-scheduler-plugins.yaml using file config/jobs/image-pushing/k8s-staging-scheduler-plugins.yaml
  • key k8s-staging-seccomp-operator.yaml using file config/jobs/image-pushing/k8s-staging-seccomp-operator.yaml
  • key k8s-staging-sig-docs.yaml using file config/jobs/image-pushing/k8s-staging-sig-docs.yaml
  • key k8s-staging-sig-storage.yaml using file config/jobs/image-pushing/k8s-staging-sig-storage.yaml
  • key k8s-staging-slack-infra.yaml using file config/jobs/image-pushing/k8s-staging-slack-infra.yaml
  • key k8s-staging-storage-migrator.yaml using file config/jobs/image-pushing/k8s-staging-storage-migrator.yaml
  • key k8s-staging-test-infra.yaml using file config/jobs/image-pushing/k8s-staging-test-infra.yaml
  • key k8s-staging-artifact-promoter.yaml using file config/jobs/image-pushing/releng/k8s-staging-artifact-promoter.yaml
  • key k8s-staging-build-image.yaml using file config/jobs/image-pushing/releng/k8s-staging-build-image.yaml
  • key k8s-staging-kubernetes.yaml using file config/jobs/image-pushing/releng/k8s-staging-kubernetes.yaml
  • key k8s-staging-releng.yaml using file config/jobs/image-pushing/releng/k8s-staging-releng.yaml
  • key gcp-filestore-csi-driver-config.yaml using file config/jobs/kubernetes-sigs/gcp-filestore-csi-driver/gcp-filestore-csi-driver-config.yaml
  • key kind-presubmits.yaml using file config/jobs/kubernetes-sigs/kind/kind-presubmits.yaml
  • key kind-release-blocking.yaml using file config/jobs/kubernetes-sigs/kind/kind-release-blocking.yaml
  • key kind.yaml using file config/jobs/kubernetes-sigs/kind/kind.yaml
  • key kube-batch-config.yaml using file config/jobs/kubernetes-sigs/kube-batch/kube-batch-config.yaml
  • key metrics-server-presubmits.yaml using file config/jobs/kubernetes-sigs/metrics-server/metrics-server-presubmits.yaml
  • key poseidon-config.yaml using file config/jobs/kubernetes-sigs/poseidon/poseidon-config.yaml
  • key hnc-e2e.yaml using file config/jobs/kubernetes-sigs/wg-multi-tenancy/hnc-e2e.yaml
  • key mtb-presubmit.yaml using file config/jobs/kubernetes-sigs/wg-multi-tenancy/mtb-presubmit.yaml
  • key release-config.yaml using file config/jobs/kubernetes/release/release-config.yaml
  • key conformance-audit.yaml using file config/jobs/kubernetes/sig-arch/conformance-audit.yaml
  • key sig-instrumentation-kind-periodics.yaml using file config/jobs/kubernetes/sig-instrumentation/sig-instrumentation-kind-periodics.yaml
  • key ingress-nginx-periodics-legacy.yaml using file config/jobs/kubernetes/sig-network/ingress-nginx/ingress-nginx-periodics-legacy.yaml
  • key ingress-nginx-periodics-main.yaml using file config/jobs/kubernetes/sig-network/ingress-nginx/ingress-nginx-periodics-main.yaml
  • key ingress-nginx-presubmit.yaml using file config/jobs/kubernetes/sig-network/ingress-nginx/ingress-nginx-presubmit.yaml
  • key sig-network-kind.yaml using file config/jobs/kubernetes/sig-network/sig-network-kind.yaml
  • key container-image-promoter.yaml using file config/jobs/kubernetes/sig-release/cip/container-image-promoter.yaml
  • key kubernetes-builds.yaml using file config/jobs/kubernetes/sig-release/kubernetes-builds.yaml
  • key 1.19.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.19.yaml
  • key 1.20.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.20.yaml
  • key 1.21.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.21.yaml
  • key 1.22.yaml using file config/jobs/kubernetes/sig-release/release-branch-jobs/1.22.yaml
  • key sig-scalability-golang.yaml using file config/jobs/kubernetes/sig-scalability/sig-scalability-golang.yaml
  • key sig-scalability-presubmit-jobs.yaml using file config/jobs/kubernetes/sig-scalability/sig-scalability-presubmit-jobs.yaml
  • key conformance-e2e.yaml using file config/jobs/kubernetes/sig-testing/conformance-e2e.yaml
  • key kubernetes-kind-presubmits.yaml using file config/jobs/kubernetes/sig-testing/kubernetes-kind-presubmits.yaml
  • key test-infra-periodics.yaml using file config/jobs/kubernetes/test-infra/test-infra-periodics.yaml
  • key sig-scalability.yaml using file config/jobs/kubernetes/wg-k8s-infra/canaries/sig-scalability.yaml
  • key wg-k8s-infra-test-infra.yaml using file config/jobs/kubernetes/wg-k8s-infra/trusted/wg-k8s-infra-test-infra.yaml

In response to this:

Related:

The first commits are image-by-image search-replace with some manual inspection to add any leftovers

The second-to-last commit removes the cluster-api image as it's been subsumed by kubekins-e2e since 2019

The last commit updates experiment/print-job-image-summary.sh, output as of this PR is now

# images used by prowjobs on prow.k8s.io
- total /                      2395 total, 56 unique
 - gcr.io /                   2257 total, 43 unique
   - kubernetes.io gcp org
     - k8s-staging-test-infra 2102 total, 7 unique
       - kubekins-e2e         1850 total, 1 unique
       - image-builder        167 total, 1 unique
       - krte                 68 total, 1 unique
       - bootstrap            8 total, 1 unique
       - gcloud-in-go         6 total, 1 unique
       - bigquery             2 total, 1 unique
       - triage               1 total, 1 unique
     - k8s-staging-the_rest   52 total, 8 unique
     - k8s.gcr.io             14 total, 5 unique
   - google.com gcp org
     - k8s-prow               17 total, 7 unique
     - k8s-testimages         50 total, 6 unique
       - gcloud-bazel         22 total, 1 unique
       - bazelbuild           5 total, 1 unique
       - bazel                5 total, 1 unique
       - benchmarkjunit       4 total, 1 unique
       - gubernator           1 total, 1 unique
       - e2e-kubemci          1 total, 1 unique
   - other (unsure which org) 22 total, 10 unique
     - gcloud-bazel           23 total, 2 unique
     - bazel                  21 total, 3 unique
     - bazel                  21 total, 3 unique
     - cip                    5 total, 1 unique
     - shellcheck             3 total, 1 unique
     - mdlint                 3 total, 1 unique
     - octodns                2 total, 1 unique
     - ci                     2 total, 1 unique
     - prow-test              1 total, 1 unique
     - minikube-e2e           1 total, 1 unique
 - not gcr.io /               138 total, 13 unique
   - dockerhub /              134 total, 9 unique
   - quay.io /                4 total, 4 unique

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@spiffxp spiffxp deleted the use-k8s-infra-canary-images branch September 14, 2021 13:55
@sanposhiho sanposhiho mentioned this pull request Sep 14, 2021
Merged
This was referenced Sep 15, 2021
Merged
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BenTheElder BenTheElder Awaiting requested review from BenTheElder

@cadmuxe cadmuxe Awaiting requested review from cadmuxe

@ameukam ameukam Awaiting requested review from ameukam

@dims dims Awaiting requested review from dims

Assignees

@ameukam ameukam

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/config Issues or PRs related to code in /config area/conformance Issues or PRs related to kubernetes conformance tests area/images area/jobs area/release-eng Issues or PRs related to the Release Engineering subproject cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
v1.23
Development

Successfully merging this pull request may close these issues.
4 participants
@spiffxp @dims @k8s-ci-robot @ameukam