Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 KCP: use new registry for Kubernetes >= v1.22 #7471

Merged
merged 1 commit into from
Nov 4, 2022
Merged

🌱 KCP: use new registry for Kubernetes >= v1.22 #7471

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions controlplane/kubeadm/internal/workload_cluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ var (

// minKubernetesVersionImageRegistryMigration is first kubernetes version where
// the default image registry is registry.k8s.io instead of k8s.gcr.io.
minKubernetesVersionImageRegistryMigration = semver.MustParse("1.25.0")
minKubernetesVersionImageRegistryMigration = semver.MustParse("1.22.0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the images available for all 1.22 patches (ie. not starting at a specific patch)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

for i in {0..15}
do
  docker pull registry.k8s.io/kube-apiserver:v1.22.${i}
  docker pull registry.k8s.io/kube-controller-manager:v1.22.${i}
  docker pull registry.k8s.io/kube-scheduler:v1.22.${i}
  docker pull registry.k8s.io/kube-proxy:v1.22.${i}
done

Worked for me.

Tried a few etcd images. Seem to be there as well.

@neolit123 Do you have some information about which related images should be available in the new registry? I guess there was some kind of sync effort?

Copy link
Member

@fabriziopandini fabriziopandini Nov 3, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be @ameukam can help here
the same question applies for older patch release of newer K8s versions

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait for an answer. I don't have a good way to verify that all images exist

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the images present in k8s.gcr.io will be available in registry.k8s.io. the promotion process covert both endpoints.

@sbueringer you can use crane or imgpkg to verify this.

imgpkg tag list -i registry.k8s.io/kube-proxy | grep '1.22.'                                                                    12:55:05 AM
sha256-b137918392d06ed92c521d93c105b12224604e8e425c1e30ec194d665e1db9a8.sig
v1.22.0
v1.22.0-alpha.0
v1.22.0-alpha.1
v1.22.0-alpha.2
v1.22.0-alpha.3
v1.22.0-beta.0
v1.22.0-beta.1
v1.22.0-beta.2
v1.22.0-rc.0
v1.22.1
v1.22.1-rc.0
v1.22.10
v1.22.10-rc.0
v1.22.11
v1.22.11-rc.0
v1.22.12
v1.22.12-rc.0
v1.22.13
v1.22.13-rc.0
v1.22.14
v1.22.14-rc.0
v1.22.15
v1.22.15-rc.0
v1.22.16-rc.0
v1.22.2
v1.22.2-rc.0
v1.22.3
v1.22.3-rc.0
v1.22.4
v1.22.4-rc.0
v1.22.5
v1.22.5-rc.0
v1.22.6
v1.22.6-rc.0
v1.22.7
v1.22.7-rc.0
v1.22.8
v1.22.8-rc.0
v1.22.9
v1.22.9-rc.0

Copy link
Member Author

@sbueringer sbueringer Nov 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think our concern was mostly about old images as we assumed the image promotion process was changed a lot later then the 1.22, ... releases and we didn't know if there was some sort of additional sync.

But I assume this

All the images present in k8s.gcr.io will be available in registry.k8s.io

means there was a full sync from k8s.gcr.io to registry.k8s.io


// nextKubernetesVersionImageRegistryMigration is the next minor version after
// the default image registry changed to registry.k8s.io.
Expand Down Expand Up @@ -624,10 +624,10 @@ func yamlToUnstructured(rawYAML []byte) (*unstructured.Unstructured, error) {

// ImageRepositoryFromClusterConfig returns the image repository to use. It returns:
// * clusterConfig.ImageRepository if set.
// * "registry.k8s.io" if v1.25 <= version < v1.26 to migrate to the new registry
// * "registry.k8s.io" if v1.22 <= version < v1.26 to migrate to the new registry
// * "" otherwise.
// Beginning with kubernetes v1.25, the default registry for kubernetes is registry.k8s.io
// instead of k8s.gcr.io which is why references should get migrated when upgrading to v1.25.
// Beginning with kubernetes v1.22, the default registry for kubernetes is registry.k8s.io
// instead of k8s.gcr.io which is why references should get migrated when upgrading to v1.22.
// The migration follows the behavior of `kubeadm upgrade`.
func ImageRepositoryFromClusterConfig(clusterConfig *bootstrapv1.ClusterConfiguration, kubernetesVersion semver.Version) string {
// If ImageRepository is explicitly specified, return early.
Expand All @@ -636,7 +636,7 @@ func ImageRepositoryFromClusterConfig(clusterConfig *bootstrapv1.ClusterConfigur
return clusterConfig.ImageRepository
}

// If v1.25 <= version < v1.26 return the default Kubernetes image repository to
// If v1.22 <= version < v1.26 return the default Kubernetes image repository to
// migrate to the new location and not cause changes else.
if kubernetesVersion.GTE(minKubernetesVersionImageRegistryMigration) &&
kubernetesVersion.LT(nextKubernetesVersionImageRegistryMigration) {
Expand Down
24 changes: 22 additions & 2 deletions controlplane/kubeadm/internal/workload_cluster_coredns_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1213,7 +1213,7 @@ func TestGetCoreDNSInfo(t *testing.T) {
},
},
{
name: "rename to coredns/coredns when upgrading to coredns=1.8.0 and kubernetesVersion=1.24.0",
name: "rename to coredns/coredns when upgrading to coredns=1.8.0 and kubernetesVersion=1.21.0",
objs: []client.Object{newCoreDNSInfoDeploymentWithimage(image162), cm},
clusterConfig: &bootstrapv1.ClusterConfiguration{
DNS: bootstrapv1.DNS{
Expand All @@ -1222,7 +1222,7 @@ func TestGetCoreDNSInfo(t *testing.T) {
},
},
},
kubernetesVersion: semver.MustParse("1.24.0"),
kubernetesVersion: semver.MustParse("1.21.0"),
expectedInfo: coreDNSInfo{
CurrentMajorMinorPatch: "1.6.2",
FromImageTag: "1.6.2",
Expand Down Expand Up @@ -1252,6 +1252,26 @@ func TestGetCoreDNSInfo(t *testing.T) {
ToImageTag: "1.8.0",
},
},
{
name: "patches ImageRepository to registry.k8s.io if it's set on neither global nor DNS-level and kubernetesVersion >= v1.22 and rename to coredns/coredns",
objs: []client.Object{newCoreDNSInfoDeploymentWithimage(image162), cm},
clusterConfig: &bootstrapv1.ClusterConfiguration{
DNS: bootstrapv1.DNS{
ImageMeta: bootstrapv1.ImageMeta{
ImageTag: "1.8.0",
},
},
},
kubernetesVersion: semver.MustParse("1.22.0"),
expectedInfo: coreDNSInfo{
CurrentMajorMinorPatch: "1.6.2",
FromImageTag: "1.6.2",
TargetMajorMinorPatch: "1.8.0",
FromImage: image162,
ToImage: "registry.k8s.io/coredns/coredns:1.8.0",
ToImageTag: "1.8.0",
},
},
{
name: "patches ImageRepository to registry.k8s.io if it's set on neither global nor DNS-level and kubernetesVersion >= v1.25 and rename to coredns/coredns",
objs: []client.Object{newCoreDNSInfoDeploymentWithimage(image162), cm},
Expand Down
8 changes: 4 additions & 4 deletions test/framework/daemonset_helpers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,12 @@ func WaitForKubeProxyUpgrade(ctx context.Context, input WaitForKubeProxyUpgradeI
parsedVersion, err := semver.ParseTolerant(input.KubernetesVersion)
Expect(err).ToNot(HaveOccurred())

// Beginning with kubernetes v1.25, kubernetes images including kube-proxy get published to registry.k8s.io instead of k8s.gcr.io.
// This ensures that the imageRepository setting gets patched to registry.k8s.io when upgrading from v1.24 or lower,
// but only if there was no imageRespository explicitly set at the KubeadmControlPlanes ClusterConfiguration.
// Beginning with kubernetes v1.22, kubernetes images including kube-proxy get published to registry.k8s.io.
// This ensures that the imageRepository setting gets patched to registry.k8s.io when upgrading from v1.21 or lower,
// but only if there was no imageRepository explicitly set at the KubeadmControlPlanes ClusterConfiguration.
// This follows the behavior of `kubeadm upgrade`.
wantKubeProxyRegistry := "registry.k8s.io"
if parsedVersion.LT(semver.Version{Major: 1, Minor: 25, Patch: 0, Pre: []semver.PRVersion{{VersionStr: "alpha"}}}) {
if parsedVersion.LT(semver.Version{Major: 1, Minor: 22, Patch: 0, Pre: []semver.PRVersion{{VersionStr: "alpha"}}}) {
wantKubeProxyRegistry = "k8s.gcr.io"
}
wantKubeProxyImage := wantKubeProxyRegistry + "/kube-proxy:" + containerutil.SemverToOCIImageTag(input.KubernetesVersion)
Expand Down