Bump to kind 0.14.0

[skitt]

This allows us to test against the current release of Kubernetes,
1.24. Support for 1.17 is no longer provided by default with kind, so
the corresponding entry is removed.

See https://github.com/kubernetes-sigs/kind/releases/tag/v0.13.0 and
https://github.com/kubernetes-sigs/kind/releases/tag/v0.14.0 for
details.

Depends on kubernetes-sigs/kind#2803
Depends on submariner-io/subctl#102
Signed-off-by: Stephen Kitt [email protected]

[submariner-bot]

🤖 Created branch: z_pr825/skitt/kind-0.13.0

[dfarrell07]

It seems like we need to increase the inotify settings in the shared script tests. I can take a look.

[dfarrell07]

Testing a fix here: dfarrell07#41

[dfarrell07]

Seems like the inotify settings were not the actual issue. Might still be good to bump them in these tests to follow our warn_inotify function's advice and not making misleading logs.

[dfarrell07]

Digging through logs, this seems to be the actual failure:

2022-05-18T15:01:25.4137367Z �[36m[15:00:24.753] [shipyard]$ [cluster1] command kubectl --context=cluster1 -n kube-system rollout status deploy/coredns --timeout=1m�[0m
2022-05-18T15:01:25.4137817Z [cluster1] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
2022-05-18T15:01:25.4138189Z [cluster1] error: timed out waiting for the condition
2022-05-18T15:01:25.4138529Z [cluster1] Failed to deploy cluster capabilities, removing the cluster

[skitt]

I’m going to split the kind bump from the k8s bump to see if that changes anything.

[skitt]

The CoreDNS pods aren’t starting because of

  Warning  FailedCreatePodSandBox  73s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "552c0f5ea87fbd12566
5c018ec47e264d352e965ef61bffbb0fc9303b1f953f5": plugin type="portmap" failed (add): failed to parse config: could not parse prevResult: could not parse prevResult: result type sup
ports [0.3.0 0.3.1 0.4.0] but unmarshalled CNIVersion is "0.1.0"

[dfarrell07]

[~/go/src/submariner-io]$ grep -rniI --exclude-dir=vendor CNIVersion .
[~/go/src/submariner-io]$ echo $?
1

😅

[dfarrell07]

https://github.com/containernetworking/cni/blob/main/pkg/types/020/types.go#L58

[dfarrell07]

Seems like the inotify settings were not the actual issue.

Made the logs more clear here: #826

[skitt]

The breakage is caused by the bump to containerd 1.6.4: it expects a CNIVersion field in the information provided by the CNI, Weave doesn’t supply this, and the fall-back value is 0.1.0 which isn’t supported (Weave supports 0.3.0 but doesn’t advertise this correctly). This might end up being fixed in Weave and is likely to be fixed in containerd 1.6.5 (through containernetworking/cni#896).

[skitt]

There is now a kind image for K8s 1.24 with containerd 1.6.6, but not for earlier releases yet.

[vthapar]

submariner-io/subctl#125 should get CI to pass. Someone should add it as a Depends On

[skitt]

Note that the dependency on submariner-io/subctl#102 is temporary; this PR currently changes the default K8s version to 1.24, but that is only because we can test kind 0.14 only with 1.24 for the time being. Before this PR can be merged, the commit changing the default should be removed, which will also remove the dependency.

[github-actions]

This PR/issue depends on:

• Please update containerd to 1.6.6 kubernetes-sigs/kind#2803
• In K8s 1.22/1.24 and later, SAs don’t automatically get a secret with a token subctl#102
  By Dependent Issues (🤖). Happy coding!

[dfarrell07]

All of the Helm jobs are failing with

[cluster1] Installing Submariner...
[16:28:33.529] [shipyard]$ [cluster1] helm --kube-context cluster1 install submariner-operator submariner-latest/submariner-operator --create-namespace --namespace submariner-operator --set ipsec.psk=lWWBU8Qe7sTBCUtmiGj06gdvvsGpmuJkUccILJvfhveWoiOAoJcBy6KSu1NXOcHc --set broker.server=172.18.0.6:6443 --set broker.token= --set broker.namespace=submariner-k8s-broker --set broker.ca= --set broker.globalnet=false --set submariner.serviceDiscovery=false --set submariner.cableDriver=libreswan --set submariner.clusterId=cluster1 --set submariner.clusterCidr=10.1.0.0/16 --set submariner.serviceCidr=100.1.0.0/16 --set submariner.globalCidr= --set serviceAccounts.globalnet.create=false --set serviceAccounts.lighthouseAgent.create=false --set serviceAccounts.lighthouseCoreDns.create=false --set submariner.natEnabled=false --set operator.image.repository=localhost:5000/submariner-operator --set operator.image.tag=local --set operator.image.pullPolicy=IfNotPresent --set submariner.images.repository=localhost:5000 --set submariner.images.tag=local --set brokercrds.create=false
[cluster1] Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: error validating "": error validating data: [ValidationError(Submariner.spec): missing required field "brokerK8sApiServerToken" in io.submariner.v1alpha1.Submariner.spec, ValidationError(Submariner.spec): missing required field "brokerK8sCA" in io.submariner.v1alpha1.Submariner.spec]

The main parts being missing required field "brokerK8sApiServerToken" and missing required field "brokerK8sCA".

[dfarrell07]

submariner-io/subctl#125 should get CI to pass. Someone should add it as a Depends On

The backport of this (submariner-io/submariner-operator#2121) should get the upgrade job passing once 0.12.2 is released.

[dfarrell07]

One interesting thing about the failing Helm jobs is that they constantly fail here but don't fail on any other runs.

Another is that the invocation that fails here passes --set broker.ca= and set broker.token= as null, whereas the passing runs elsewhere supply actual values.

[dfarrell07]

Those missing values come from:

    submariner_broker_ca=$(kubectl -n "${BROKER_NAMESPACE}" get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${BROKER_CLIENT_SA}')].data['ca\.crt']}")
    submariner_broker_token=$(kubectl -n "${BROKER_NAMESPACE}" get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${BROKER_CLIENT_SA}')].data.token}"|base64 --decode)

This seems like it's related to the recent upstream secrets/tokens changes and our adjustments.

[nyechiel]

As per @aswinsuryan, this should also resolve the OVN kind issues we are seeing in CI.

[dfarrell07]

I raised a bug to properly track this Helm failure: submariner-io/submariner-charts#236

[sridhargaddam]

This change seems to be part of the following PR - https://github.com/submariner-io/shipyard/pull/859/files

[mkolesnik]

We already merged #859 and #852

[submariner-bot]

🤖 Closed branches: [z_pr825/skitt/kind-0.13.0]