From 05c0195a06bca08d74adf7c44048d564eee24933 Mon Sep 17 00:00:00 2001 From: Oliver Frolovs Date: Thu, 14 Sep 2023 10:34:51 +0100 Subject: [PATCH] GKE Autopilot module: add network tags (#1675) * gke-cluster-autopilot: add support for network tags * gke-cluster-autopilot: add validation for network tags * gke-cluster-autopilot: expand README and fix some typos * gke-cluster-autopilot: fix Cloud DNS section in README Removed a reference to Standard clusters and updated the section to include a warning because the new versions of Autopilot clusters can only use Cloud DNS and it is pre-configured by default so the example in the README does not apply to them. * gke-cluster-autopilot: cosmetic fixes in README * gke-cluster-autopilot: rollback validation on network tags var * gke-cluster-autopilot: rollback docs string update for network tags var * gke-cluster-autopilot: rollback some updates to README * gke-cluster-autopilot: remove dead code * gke-cluster-autopilot: add a tftest for network tags * gke-cluster-autopilot: fix a tftest --- modules/gke-cluster-autopilot/README.md | 42 +++++++++++++++---- modules/gke-cluster-autopilot/main.tf | 9 ++++ modules/gke-cluster-autopilot/variables.tf | 3 +- .../gke_cluster_autopilot/network_tags.tfvars | 14 +++++++ .../gke_cluster_autopilot/network_tags.yaml | 27 ++++++++++++ .../modules/gke_cluster_autopilot/tftest.yaml | 18 ++++++++ 6 files changed, 104 insertions(+), 9 deletions(-) create mode 100644 tests/modules/gke_cluster_autopilot/network_tags.tfvars create mode 100644 tests/modules/gke_cluster_autopilot/network_tags.yaml create mode 100644 tests/modules/gke_cluster_autopilot/tftest.yaml diff --git a/modules/gke-cluster-autopilot/README.md b/modules/gke-cluster-autopilot/README.md index da6390661e..a110e8f748 100644 --- a/modules/gke-cluster-autopilot/README.md +++ b/modules/gke-cluster-autopilot/README.md @@ -1,10 +1,23 @@ -# GKE cluster Autopilot module +# GKE Autopilot cluster module -This module allows simplified creation and management of GKE Autopilot clusters. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases. +This module offers a way to create and manage Google Kubernetes Engine (GKE) [Autopilot clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview). With its sensible default settings based on best practices and authors' experience as Google Cloud practitioners, the module accommodates for many common use cases out-of-the-box, without having to rely on verbose configuration. -## Example + +- [Examples](#examples) + - [GKE Autopilot cluster](#gke-autopilot-cluster) + - [Cloud DNS](#cloud-dns) + - [Logging configuration](#logging-configuration) + - [Monitoring configuration](#monitoring-configuration) + - [Backup for GKE](#backup-for-gke) +- [Variables](#variables) +- [Outputs](#outputs) + -### GKE Cluster +## Examples + +### GKE Autopilot cluster + +This example shows how to [create a GKE cluster in Autopilot mode](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-an-autopilot-cluster). ```hcl module "cluster-1" { @@ -37,7 +50,10 @@ module "cluster-1" { ### Cloud DNS -This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns) for GKE Standard clusters. +This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns). + +> **Warning** +> [Cloud DNS is the only DNS provider for Autopilot clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/service-discovery#cloud_dns) running version `1.25.9-gke.400` and later, and version `1.26.4-gke.500` and later. It is [pre-configured](https://cloud.google.com/kubernetes-engine/docs/resources/autopilot-standard-feature-comparison#feature-comparison) for those clusters. The following example *only* applies to Autopilot clusters running *earlier* versions. ```hcl module "cluster-1" { @@ -118,7 +134,17 @@ module "cluster-1" { ### Backup for GKE -This example shows how to [enable the Backup for GKE agent and configure a Backup Plan](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke) for GKE Standard clusters. +[Backup for GKE](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke) is a service for backing up and restoring workloads in GKE clusters. It has two components: + +* A [Google Cloud API](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/reference/rest) that serves as the control plane for the service. +* A GKE add-on (the [Backup for GKE agent](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke#agent_overview)) that must be enabled in each cluster for which you wish to perform backup and restore operations. + +> **Note** +> Although Backup for GKE can be enabled as an add-on when configuring your GKE clusters, it is a separate service from GKE. + +Backup for GKE is supported in GKE Autopilot clusters with [some restrictions](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/about-autopilot). + +This example shows how to [enable Backup for GKE on a new Autopilot cluster](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/how-to/install#enable_on_a_new_cluster_optional) and [plan a set of backups](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/how-to/backup-plan). ```hcl module "cluster-1" { @@ -151,7 +177,7 @@ module "cluster-1" { | [location](variables.tf#L110) | Autopilot cluster are always regional. | string | ✓ | | | [name](variables.tf#L170) | Cluster name. | string | ✓ | | | [project_id](variables.tf#L196) | Cluster project id. | string | ✓ | | -| [vpc_config](variables.tf#L224) | VPC-level configuration. | object({…}) | ✓ | | +| [vpc_config](variables.tf#L225) | VPC-level configuration. | object({…}) | ✓ | | | [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | object({…}) | | {} | | [description](variables.tf#L37) | Cluster description. | string | | null | | [enable_addons](variables.tf#L43) | Addons enabled in the cluster (true means enabled). | object({…}) | | {…} | @@ -166,7 +192,7 @@ module "cluster-1" { | [private_cluster_config](variables.tf#L182) | Private cluster configuration. | object({…}) | | null | | [release_channel](variables.tf#L201) | Release channel for GKE upgrades. Clusters created in the Autopilot mode must use a release channel. Choose between \"RAPID\", \"REGULAR\", and \"STABLE\". | string | | "REGULAR" | | [service_account](variables.tf#L212) | The Google Cloud Platform Service Account to be used by the node VMs created by GKE Autopilot. | string | | null | -| [tags](variables.tf#L218) | Network tags applied to nodes. | list(string) | | null | +| [tags](variables.tf#L218) | Network tags applied to nodes. | list(string) | | [] | ## Outputs diff --git a/modules/gke-cluster-autopilot/main.tf b/modules/gke-cluster-autopilot/main.tf index 330c499326..13317f469b 100644 --- a/modules/gke-cluster-autopilot/main.tf +++ b/modules/gke-cluster-autopilot/main.tf @@ -231,6 +231,15 @@ resource "google_container_cluster" "cluster" { } } + dynamic "node_pool_auto_config" { + for_each = length(var.tags) > 0 ? [""] : [] + content { + network_tags { + tags = toset(var.tags) + } + } + } + dynamic "private_cluster_config" { for_each = ( var.private_cluster_config != null ? [""] : [] diff --git a/modules/gke-cluster-autopilot/variables.tf b/modules/gke-cluster-autopilot/variables.tf index 52896bbdc9..3f623347d9 100644 --- a/modules/gke-cluster-autopilot/variables.tf +++ b/modules/gke-cluster-autopilot/variables.tf @@ -218,7 +218,8 @@ variable "service_account" { variable "tags" { description = "Network tags applied to nodes." type = list(string) - default = null + default = [] + nullable = false } variable "vpc_config" { diff --git a/tests/modules/gke_cluster_autopilot/network_tags.tfvars b/tests/modules/gke_cluster_autopilot/network_tags.tfvars new file mode 100644 index 0000000000..4b188f3197 --- /dev/null +++ b/tests/modules/gke_cluster_autopilot/network_tags.tfvars @@ -0,0 +1,14 @@ +project_id = "my-project" +location = "europe-west1" +name = "cluster-1" +vpc_config = { + network = "default" + subnetwork = "default" +} +tags = [ + "deep-dark-wood", + "hello-gruffalo", + "my--precious---nodes", + "cluster-1-nodes", + "nodes-cluster-1", +] diff --git a/tests/modules/gke_cluster_autopilot/network_tags.yaml b/tests/modules/gke_cluster_autopilot/network_tags.yaml new file mode 100644 index 0000000000..5ca4826083 --- /dev/null +++ b/tests/modules/gke_cluster_autopilot/network_tags.yaml @@ -0,0 +1,27 @@ +# Copyright 2023 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +values: + google_container_cluster.cluster: + node_pool_auto_config: + - network_tags: + - tags: + - cluster-1-nodes + - deep-dark-wood + - hello-gruffalo + - my--precious---nodes + - nodes-cluster-1 + +counts: + google_container_cluster: 1 diff --git a/tests/modules/gke_cluster_autopilot/tftest.yaml b/tests/modules/gke_cluster_autopilot/tftest.yaml new file mode 100644 index 0000000000..18fc6235e1 --- /dev/null +++ b/tests/modules/gke_cluster_autopilot/tftest.yaml @@ -0,0 +1,18 @@ +# Copyright 2023 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +module: modules/gke-cluster-autopilot + +tests: + network_tags: