GKE Autopilot module: add network tags (GoogleCloudPlatform#1675)

* gke-cluster-autopilot: add support for network tags * gke-cluster-autopilot: add validation for network tags * gke-cluster-autopilot: expand README and fix some typos * gke-cluster-autopilot: fix Cloud DNS section in README Removed a reference to Standard clusters and updated the section to include a warning because the new versions of Autopilot clusters can only use Cloud DNS and it is pre-configured by default so the example in the README does not apply to them. * gke-cluster-autopilot: cosmetic fixes in README * gke-cluster-autopilot: rollback validation on network tags var * gke-cluster-autopilot: rollback docs string update for network tags var * gke-cluster-autopilot: rollback some updates to README * gke-cluster-autopilot: remove dead code * gke-cluster-autopilot: add a tftest for network tags * gke-cluster-autopilot: fix a tftest
wiktorn · Sep 14, 2023 · 05c0195 · 05c0195
1 parent eab298f
commit 05c0195
Show file tree

Hide file tree

Showing 6 changed files with 104 additions and 9 deletions.
diff --git a/modules/gke-cluster-autopilot/README.md b/modules/gke-cluster-autopilot/README.md
@@ -1,10 +1,23 @@
-# GKE cluster Autopilot  module
+# GKE Autopilot cluster module
 
-This module allows simplified creation and management of GKE Autopilot clusters. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases.
+This module offers a way to create and manage Google Kubernetes Engine (GKE) [Autopilot clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview). With its sensible default settings based on best practices and authors' experience as Google Cloud practitioners, the module accommodates for many common use cases out-of-the-box, without having to rely on verbose configuration.
 
-## Example
+<!-- BEGIN TOC -->
+- [Examples](#examples)
+  - [GKE Autopilot cluster](#gke-autopilot-cluster)
+  - [Cloud DNS](#cloud-dns)
+  - [Logging configuration](#logging-configuration)
+  - [Monitoring configuration](#monitoring-configuration)
+  - [Backup for GKE](#backup-for-gke)
+- [Variables](#variables)
+- [Outputs](#outputs)
+<!-- END TOC -->
 
-### GKE Cluster
+## Examples
+
+### GKE Autopilot cluster
+
+This example shows how to [create a GKE cluster in Autopilot mode](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-an-autopilot-cluster).
 
 ```hcl
 module "cluster-1" {
@@ -37,7 +50,10 @@ module "cluster-1" {
 
 ### Cloud DNS
 
-This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns) for GKE Standard clusters.
+This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns).
+
+> **Warning**
+> [Cloud DNS is the only DNS provider for Autopilot clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/service-discovery#cloud_dns) running version `1.25.9-gke.400` and later, and version `1.26.4-gke.500` and later. It is [pre-configured](https://cloud.google.com/kubernetes-engine/docs/resources/autopilot-standard-feature-comparison#feature-comparison) for those clusters. The following example *only* applies to Autopilot clusters running *earlier* versions.
 
 ```hcl
 module "cluster-1" {
@@ -118,7 +134,17 @@ module "cluster-1" {
 
 ### Backup for GKE
 
-This example shows how to [enable the Backup for GKE agent and configure a Backup Plan](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke) for GKE Standard clusters.
+[Backup for GKE](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke) is a service for backing up and restoring workloads in GKE clusters. It has two components:
+
+* A [Google Cloud API](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/reference/rest) that serves as the control plane for the service.
+* A GKE add-on (the [Backup for GKE agent](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke#agent_overview)) that must be enabled in each cluster for which you wish to perform backup and restore operations.
+
+> **Note**
+> Although Backup for GKE can be enabled as an add-on when configuring your GKE clusters, it is a separate service from GKE.
+
+Backup for GKE is supported in GKE Autopilot clusters with [some restrictions](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/about-autopilot).
+
+This example shows how to [enable Backup for GKE on a new Autopilot cluster](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/how-to/install#enable_on_a_new_cluster_optional) and [plan a set of backups](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/how-to/backup-plan).
 
 ```hcl
 module "cluster-1" {
@@ -151,7 +177,7 @@ module "cluster-1" {
 | [location](variables.tf#L110) | Autopilot cluster are always regional. | <code>string</code> | ✓ |  |
 | [name](variables.tf#L170) | Cluster name. | <code>string</code> | ✓ |  |
 | [project_id](variables.tf#L196) | Cluster project id. | <code>string</code> | ✓ |  |
-| [vpc_config](variables.tf#L224) | VPC-level configuration. | <code title="object&#40;&#123;&#10;  network                &#61; string&#10;  subnetwork             &#61; string&#10;  master_ipv4_cidr_block &#61; optional&#40;string&#41;&#10;  secondary_range_blocks &#61; optional&#40;object&#40;&#123;&#10;    pods     &#61; string&#10;    services &#61; string&#10;  &#125;&#41;&#41;&#10;  secondary_range_names &#61; optional&#40;object&#40;&#123;&#10;    pods     &#61; string&#10;    services &#61; string&#10;  &#125;&#41;, &#123; pods &#61; &#34;pods&#34;, services &#61; &#34;services&#34; &#125;&#41;&#10;  master_authorized_ranges &#61; optional&#40;map&#40;string&#41;&#41;&#10;  stack_type               &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
+| [vpc_config](variables.tf#L225) | VPC-level configuration. | <code title="object&#40;&#123;&#10;  network                &#61; string&#10;  subnetwork             &#61; string&#10;  master_ipv4_cidr_block &#61; optional&#40;string&#41;&#10;  secondary_range_blocks &#61; optional&#40;object&#40;&#123;&#10;    pods     &#61; string&#10;    services &#61; string&#10;  &#125;&#41;&#41;&#10;  secondary_range_names &#61; optional&#40;object&#40;&#123;&#10;    pods     &#61; string&#10;    services &#61; string&#10;  &#125;&#41;, &#123; pods &#61; &#34;pods&#34;, services &#61; &#34;services&#34; &#125;&#41;&#10;  master_authorized_ranges &#61; optional&#40;map&#40;string&#41;&#41;&#10;  stack_type               &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
 | [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | <code title="object&#40;&#123;&#10;  enable_backup_agent &#61; optional&#40;bool, false&#41;&#10;  backup_plans &#61; optional&#40;map&#40;object&#40;&#123;&#10;    encryption_key                    &#61; optional&#40;string&#41;&#10;    include_secrets                   &#61; optional&#40;bool, true&#41;&#10;    include_volume_data               &#61; optional&#40;bool, true&#41;&#10;    namespaces                        &#61; optional&#40;list&#40;string&#41;&#41;&#10;    region                            &#61; string&#10;    schedule                          &#61; string&#10;    retention_policy_days             &#61; optional&#40;string&#41;&#10;    retention_policy_lock             &#61; optional&#40;bool, false&#41;&#10;    retention_policy_delete_lock_days &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
 | [description](variables.tf#L37) | Cluster description. | <code>string</code> |  | <code>null</code> |
 | [enable_addons](variables.tf#L43) | Addons enabled in the cluster (true means enabled). | <code title="object&#40;&#123;&#10;  cloudrun                   &#61; optional&#40;bool, false&#41;&#10;  config_connector           &#61; optional&#40;bool, false&#41;&#10;  dns_cache                  &#61; optional&#40;bool, false&#41;&#10;  horizontal_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10;  http_load_balancing        &#61; optional&#40;bool, false&#41;&#10;  istio &#61; optional&#40;object&#40;&#123;&#10;    enable_tls &#61; bool&#10;  &#125;&#41;&#41;&#10;  kalm           &#61; optional&#40;bool, false&#41;&#10;  network_policy &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  horizontal_pod_autoscaling &#61; true&#10;  http_load_balancing        &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> |
@@ -166,7 +192,7 @@ module "cluster-1" {
 | [private_cluster_config](variables.tf#L182) | Private cluster configuration. | <code title="object&#40;&#123;&#10;  enable_private_endpoint &#61; optional&#40;bool&#41;&#10;  master_global_access    &#61; optional&#40;bool&#41;&#10;  peering_config &#61; optional&#40;object&#40;&#123;&#10;    export_routes &#61; optional&#40;bool&#41;&#10;    import_routes &#61; optional&#40;bool&#41;&#10;    project_id    &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
 | [release_channel](variables.tf#L201) | Release channel for GKE upgrades. Clusters created in the Autopilot mode must use a release channel. Choose between \"RAPID\", \"REGULAR\", and \"STABLE\". | <code>string</code> |  | <code>&#34;REGULAR&#34;</code> |
 | [service_account](variables.tf#L212) | The Google Cloud Platform Service Account to be used by the node VMs created by GKE Autopilot. | <code>string</code> |  | <code>null</code> |
-| [tags](variables.tf#L218) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> |  | <code>null</code> |
+| [tags](variables.tf#L218) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> |  | <code>&#91;&#93;</code> |
 
 ## Outputs
 

diff --git a/modules/gke-cluster-autopilot/main.tf b/modules/gke-cluster-autopilot/main.tf
@@ -231,6 +231,15 @@ resource "google_container_cluster" "cluster" {
     }
   }
 
+  dynamic "node_pool_auto_config" {
+    for_each = length(var.tags) > 0 ? [""] : []
+    content {
+      network_tags {
+        tags = toset(var.tags)
+      }
+    }
+  }
+
   dynamic "private_cluster_config" {
     for_each = (
       var.private_cluster_config != null ? [""] : []

diff --git a/modules/gke-cluster-autopilot/variables.tf b/modules/gke-cluster-autopilot/variables.tf
@@ -218,7 +218,8 @@ variable "service_account" {
 variable "tags" {
   description = "Network tags applied to nodes."
   type        = list(string)
-  default     = null
+  default     = []
+  nullable    = false
 }
 
 variable "vpc_config" {

diff --git a/tests/modules/gke_cluster_autopilot/network_tags.tfvars b/tests/modules/gke_cluster_autopilot/network_tags.tfvars
@@ -0,0 +1,14 @@
+project_id = "my-project"
+location   = "europe-west1"
+name       = "cluster-1"
+vpc_config = {
+  network    = "default"
+  subnetwork = "default"
+}
+tags = [
+  "deep-dark-wood",
+  "hello-gruffalo",
+  "my--precious---nodes",
+  "cluster-1-nodes",
+  "nodes-cluster-1",
+]
diff --git a/tests/modules/gke_cluster_autopilot/network_tags.yaml b/tests/modules/gke_cluster_autopilot/network_tags.yaml
@@ -0,0 +1,27 @@
+# Copyright 2023 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+values:
+  google_container_cluster.cluster:
+    node_pool_auto_config:
+    - network_tags:
+      - tags:
+        - cluster-1-nodes
+        - deep-dark-wood
+        - hello-gruffalo
+        - my--precious---nodes
+        - nodes-cluster-1
+
+counts:
+  google_container_cluster: 1
diff --git a/tests/modules/gke_cluster_autopilot/tftest.yaml b/tests/modules/gke_cluster_autopilot/tftest.yaml
@@ -0,0 +1,18 @@
+# Copyright 2023 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+module: modules/gke-cluster-autopilot
+
+tests:
+  network_tags: