Update Kubernetes to 1.23

[roberthbailey]

List of items to do for upgrading to 1.23:

• Update terraform submodules
  • GKE
  • Azure
  • EKS
• Update prow cluster (even though we aren't using it yet, we should keep it in sync)
  • Recreate cluster with new scripts: cd build/terraform/prow; terraform apply -var project=agones-images
• Update e2e cluster
  • Recreate cluster with new scripts: cd build/terraform/e2e; terraform apply -var project=agones-images
• Update kubectl in dev tooling
  • Update kubectl in build/build-image/Dockerfile
  • Update kubectl in build/e2e-image/Dockerfile
• Update documentation for creating clusters
  • Config.toml supported_k8s and related (do dev_ before main)
• Update the dev tooling to create clusters
  • Minikube
  • Kind
• Update the k8s image used in the helm pre-delete-hook
• Update client-go
• Update CRD API reference
  • Update links to k8s documentation in site/assets/templates/crd-doc-config.json
  • Regenerate crd api reference docs - make gen-api-docs
  • Regenerate crd client libraries - make gen-crd-client
• Regenerate Kubernetes resource includes (e.g. ObjectMeta, PodTemplateSpec)
  • Start a cluster with make gcloud-test-cluster, uninstall agones using helm uninstall agones -n agones-system, and then run make gen-embedded-openapi and make gen-install
• Update gRPC clients

[roberthbailey]

Following the guidance on https://agones.dev/site/docs/installation/

Agones will update its support to the n-1 version of what is available across the majority of major cloud providers - GKE, EKS and AKS, while also ensuring that all Cloud providers can support that version.

I've included the current version support for each of the major cloud providers.

EKS

From https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html:

The following Kubernetes versions are currently available for new clusters in Amazon EKS:

• 1.22.9
• 1.21.12
• 1.20.15
• 1.19.16

And https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-release-calendar has an ETA for 1.23 in August 2022.

AKS

https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions#aks-kubernetes-release-calendar shows that 1.23 graduated to GA in April 2022.

GKE

$ gcloud container get-server-config --format="yaml(defaultClusterVersion,channels,validMasterVersions)" --region us-central1
Fetching server config for us-central1
channels:
- channel: RAPID
  defaultVersion: 1.23.6-gke.1501
  validVersions:
  - 1.24.0-gke.1801
  - 1.23.6-gke.2200
  - 1.23.6-gke.1700
  - 1.23.6-gke.1501
  - 1.23.6-gke.1500
  - 1.22.9-gke.2000
  - 1.22.9-gke.1500
  - 1.22.9-gke.1300
  - 1.21.12-gke.2200
  - 1.21.12-gke.1700
- channel: REGULAR
  defaultVersion: 1.22.8-gke.202
  validVersions:
  - 1.23.5-gke.1503
  - 1.23.5-gke.1501
  - 1.22.8-gke.202
  - 1.22.8-gke.201
  - 1.21.12-gke.1500
  - 1.21.11-gke.1900
  - 1.20.15-gke.8000
- channel: STABLE
  defaultVersion: 1.21.11-gke.1900
  validVersions:
  - 1.22.8-gke.202
  - 1.22.8-gke.201
  - 1.22.8-gke.200
  - 1.21.11-gke.1900
  - 1.21.11-gke.1100
  - 1.20.15-gke.6000
  - 1.19.16-gke.11800
  - 1.19.16-gke.11000
defaultClusterVersion: 1.22.8-gke.202
validMasterVersions:
- 1.23.6-gke.2200
- 1.23.6-gke.1700
- 1.23.6-gke.1501
- 1.23.6-gke.1500
- 1.23.5-gke.2400
- 1.23.5-gke.1503
- 1.23.5-gke.1501
- 1.22.9-gke.2000
- 1.22.9-gke.1500
- 1.22.9-gke.1300
- 1.22.8-gke.2200
- 1.22.8-gke.202
- 1.22.8-gke.201
- 1.22.8-gke.200
- 1.21.12-gke.2200
- 1.21.12-gke.1700
- 1.21.12-gke.1500
- 1.21.11-gke.1900
- 1.21.11-gke.1100
- 1.20.15-gke.8700
- 1.20.15-gke.8200
- 1.20.15-gke.8000
- 1.20.15-gke.6000
- 1.19.16-gke.14500
- 1.19.16-gke.14000
- 1.19.16-gke.13800
- 1.19.16-gke.11800
- 1.19.16-gke.11000
- 1.19.16-gke.9400

[roberthbailey]

If we upgrade for the Agones 1.25 release (which will be created on August 2nd) then the statement

while also ensuring that all Cloud providers can support that version

is just barely true. We discussed during the community meeting today that we would proceed with the upgrade during this release cycle but I wanted to hear if there were any dissenting voices before getting started.

[roberthbailey]

I haven't had time to do this during the last couple of months but plan to work on it for the next release.

EKS now supports 1.23.7, AKS lists 1.24 in GA in Jul, and GKE has 1.23.7 available in the regular and stable channels.

[roberthbailey]

Going back over the checklist, it looks like I missed one step (updating the kubectl image used in the helm delete hook).

Also, as @markmandel mentioned in #2711 (comment) before closing this out we also need to update grpc clients.

[roberthbailey]

With the 2 PRs to update gRPC merged I believe that this task is now complete.