feat: Add dataplane-v2 provisioning support (hashicorp#753)

* add datapath_provider var for beta clusters * incorporate datapath_provider option * update tests for datapath_provider beta cluster * example README var updates * remove network_policy var usage * finish docs updates Co-authored-by: ryan-atkins <>
flozzone · Mar 5, 2021 · d1fbef4 · d1fbef4
1 parent 3354205
commit d1fbef4
Show file tree

Hide file tree

Showing 19 changed files with 57 additions and 0 deletions.
diff --git a/autogen/main/cluster.tf.tmpl b/autogen/main/cluster.tf.tmpl
@@ -189,6 +189,9 @@ resource "google_container_cluster" "primary" {
     }
     {% endif %}
   }
+  {% if beta_cluster %}
+  datapath_provider = var.datapath_provider
+  {% endif %}
 
   {% if beta_cluster %}
   networking_mode = "VPC_NATIVE"

diff --git a/autogen/main/variables.tf.tmpl b/autogen/main/variables.tf.tmpl
@@ -107,6 +107,13 @@ variable "network_policy_provider" {
   description = "The network policy provider."
   default     = "CALICO"
 }
+{% if beta_cluster %}
+variable "datapath_provider" {
+  type        = string
+  description = "The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation."
+  default     = "DATAPATH_PROVIDER_UNSPECIFIED"
+}
+{% endif %}
 
 variable "maintenance_start_time" {
   type        = string

diff --git a/examples/simple_regional_beta/README.md b/examples/simple_regional_beta/README.md
@@ -11,6 +11,7 @@ This example illustrates how to create a simple cluster with beta features.
 | cluster\_name\_suffix | A suffix to append to the default cluster name | `string` | `""` | no |
 | compute\_engine\_service\_account | Service account to associate to the nodes in the cluster | `any` | n/a | yes |
 | database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key\_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key\_name is the name of a CloudKMS key. | `list(object({ state = string, key_name = string }))` | <pre>[<br>  {<br>    "key_name": "",<br>    "state": "DECRYPTED"<br>  }<br>]</pre> | no |
+| datapath\_provider | The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation. | `string` | `"DATAPATH_PROVIDER_UNSPECIFIED"` | no |
 | dns\_cache | (Beta) The status of the NodeLocal DNSCache addon. | `bool` | `false` | no |
 | enable\_binary\_authorization | Enable BinAuthZ Admission controller | `bool` | `false` | no |
 | enable\_pod\_security\_policy | enabled - Enable the PodSecurityPolicy controller for this cluster. If enabled, pods must be valid under a PodSecurityPolicy to be created. | `bool` | `false` | no |

diff --git a/examples/simple_regional_beta/main.tf b/examples/simple_regional_beta/main.tf
@@ -60,4 +60,7 @@ module "gke" {
   # Disable workload identity
   identity_namespace = null
   node_metadata      = "UNSPECIFIED"
+
+  # Enable Dataplane Setup
+  datapath_provider = "ADVANCED_DATAPATH"
 }
diff --git a/examples/simple_regional_beta/variables.tf b/examples/simple_regional_beta/variables.tf
@@ -123,3 +123,9 @@ variable "regional" {
   description = "Whether is a regional cluster (zonal cluster if set false. WARNING: changing this after cluster creation is destructive!)"
   default     = true
 }
+
+variable "datapath_provider" {
+  type        = string
+  description = "The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation."
+  default     = "DATAPATH_PROVIDER_UNSPECIFIED"
+}
diff --git a/modules/beta-private-cluster-update-variant/README.md b/modules/beta-private-cluster-update-variant/README.md
@@ -170,6 +170,7 @@ Then perform the following commands on the root folder:
 | configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | `bool` | `false` | no |
 | create\_service\_account | Defines if service account specified to run nodes should be created. | `bool` | `true` | no |
 | database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key\_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key\_name is the name of a CloudKMS key. | `list(object({ state = string, key_name = string }))` | <pre>[<br>  {<br>    "key_name": "",<br>    "state": "DECRYPTED"<br>  }<br>]</pre> | no |
+| datapath\_provider | The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation. | `string` | `"DATAPATH_PROVIDER_UNSPECIFIED"` | no |
 | default\_max\_pods\_per\_node | The maximum number of pods to schedule per node | `number` | `110` | no |
 | deploy\_using\_private\_endpoint | (Beta) A toggle for Terraform and kubectl to connect to the master's internal IP address during deployment. | `bool` | `false` | no |
 | description | The description of the cluster | `string` | `""` | no |

diff --git a/modules/beta-private-cluster-update-variant/cluster.tf b/modules/beta-private-cluster-update-variant/cluster.tf
@@ -172,6 +172,7 @@ resource "google_container_cluster" "primary" {
       enabled = var.config_connector
     }
   }
+  datapath_provider = var.datapath_provider
 
   networking_mode = "VPC_NATIVE"
   ip_allocation_policy {

diff --git a/modules/beta-private-cluster-update-variant/variables.tf b/modules/beta-private-cluster-update-variant/variables.tf
@@ -107,6 +107,11 @@ variable "network_policy_provider" {
   description = "The network policy provider."
   default     = "CALICO"
 }
+variable "datapath_provider" {
+  type        = string
+  description = "The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation."
+  default     = "DATAPATH_PROVIDER_UNSPECIFIED"
+}
 
 variable "maintenance_start_time" {
   type        = string

diff --git a/modules/beta-private-cluster/README.md b/modules/beta-private-cluster/README.md
@@ -148,6 +148,7 @@ Then perform the following commands on the root folder:
 | configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | `bool` | `false` | no |
 | create\_service\_account | Defines if service account specified to run nodes should be created. | `bool` | `true` | no |
 | database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key\_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key\_name is the name of a CloudKMS key. | `list(object({ state = string, key_name = string }))` | <pre>[<br>  {<br>    "key_name": "",<br>    "state": "DECRYPTED"<br>  }<br>]</pre> | no |
+| datapath\_provider | The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation. | `string` | `"DATAPATH_PROVIDER_UNSPECIFIED"` | no |
 | default\_max\_pods\_per\_node | The maximum number of pods to schedule per node | `number` | `110` | no |
 | deploy\_using\_private\_endpoint | (Beta) A toggle for Terraform and kubectl to connect to the master's internal IP address during deployment. | `bool` | `false` | no |
 | description | The description of the cluster | `string` | `""` | no |

diff --git a/modules/beta-private-cluster/cluster.tf b/modules/beta-private-cluster/cluster.tf
@@ -172,6 +172,7 @@ resource "google_container_cluster" "primary" {
       enabled = var.config_connector
     }
   }
+  datapath_provider = var.datapath_provider
 
   networking_mode = "VPC_NATIVE"
   ip_allocation_policy {

diff --git a/modules/beta-private-cluster/variables.tf b/modules/beta-private-cluster/variables.tf
@@ -107,6 +107,11 @@ variable "network_policy_provider" {
   description = "The network policy provider."
   default     = "CALICO"
 }
+variable "datapath_provider" {
+  type        = string
+  description = "The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation."
+  default     = "DATAPATH_PROVIDER_UNSPECIFIED"
+}
 
 variable "maintenance_start_time" {
   type        = string

diff --git a/modules/beta-public-cluster-update-variant/README.md b/modules/beta-public-cluster-update-variant/README.md
@@ -164,6 +164,7 @@ Then perform the following commands on the root folder:
 | configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | `bool` | `false` | no |
 | create\_service\_account | Defines if service account specified to run nodes should be created. | `bool` | `true` | no |
 | database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key\_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key\_name is the name of a CloudKMS key. | `list(object({ state = string, key_name = string }))` | <pre>[<br>  {<br>    "key_name": "",<br>    "state": "DECRYPTED"<br>  }<br>]</pre> | no |
+| datapath\_provider | The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation. | `string` | `"DATAPATH_PROVIDER_UNSPECIFIED"` | no |
 | default\_max\_pods\_per\_node | The maximum number of pods to schedule per node | `number` | `110` | no |
 | description | The description of the cluster | `string` | `""` | no |
 | disable\_default\_snat | Whether to disable the default SNAT to support the private use of public IP addresses | `bool` | `false` | no |

diff --git a/modules/beta-public-cluster-update-variant/cluster.tf b/modules/beta-public-cluster-update-variant/cluster.tf
@@ -172,6 +172,7 @@ resource "google_container_cluster" "primary" {
       enabled = var.config_connector
     }
   }
+  datapath_provider = var.datapath_provider
 
   networking_mode = "VPC_NATIVE"
   ip_allocation_policy {

diff --git a/modules/beta-public-cluster-update-variant/variables.tf b/modules/beta-public-cluster-update-variant/variables.tf
@@ -107,6 +107,11 @@ variable "network_policy_provider" {
   description = "The network policy provider."
   default     = "CALICO"
 }
+variable "datapath_provider" {
+  type        = string
+  description = "The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation."
+  default     = "DATAPATH_PROVIDER_UNSPECIFIED"
+}
 
 variable "maintenance_start_time" {
   type        = string

diff --git a/modules/beta-public-cluster/README.md b/modules/beta-public-cluster/README.md
@@ -142,6 +142,7 @@ Then perform the following commands on the root folder:
 | configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | `bool` | `false` | no |
 | create\_service\_account | Defines if service account specified to run nodes should be created. | `bool` | `true` | no |
 | database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key\_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key\_name is the name of a CloudKMS key. | `list(object({ state = string, key_name = string }))` | <pre>[<br>  {<br>    "key_name": "",<br>    "state": "DECRYPTED"<br>  }<br>]</pre> | no |
+| datapath\_provider | The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation. | `string` | `"DATAPATH_PROVIDER_UNSPECIFIED"` | no |
 | default\_max\_pods\_per\_node | The maximum number of pods to schedule per node | `number` | `110` | no |
 | description | The description of the cluster | `string` | `""` | no |
 | disable\_default\_snat | Whether to disable the default SNAT to support the private use of public IP addresses | `bool` | `false` | no |

diff --git a/modules/beta-public-cluster/cluster.tf b/modules/beta-public-cluster/cluster.tf
@@ -172,6 +172,7 @@ resource "google_container_cluster" "primary" {
       enabled = var.config_connector
     }
   }
+  datapath_provider = var.datapath_provider
 
   networking_mode = "VPC_NATIVE"
   ip_allocation_policy {

diff --git a/modules/beta-public-cluster/variables.tf b/modules/beta-public-cluster/variables.tf
@@ -107,6 +107,11 @@ variable "network_policy_provider" {
   description = "The network policy provider."
   default     = "CALICO"
 }
+variable "datapath_provider" {
+  type        = string
+  description = "The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation."
+  default     = "DATAPATH_PROVIDER_UNSPECIFIED"
+}
 
 variable "maintenance_start_time" {
   type        = string

diff --git a/test/fixtures/beta_cluster/main.tf b/test/fixtures/beta_cluster/main.tf
@@ -61,6 +61,9 @@ module "this" {
   enable_binary_authorization = true
 
   enable_pod_security_policy = true
+
+  // Dataplane-V2 Feature
+  datapath_provider = "ADVANCED_DATAPATH"
 }
 
 data "google_client_config" "default" {

diff --git a/test/integration/beta_cluster/controls/gcloud.rb b/test/integration/beta_cluster/controls/gcloud.rb
@@ -73,6 +73,12 @@
         )
       end
 
+      it "has the expected datapathProvider config" do
+        expect(data['networkConfig']).to include(
+          "datapathProvider" => "ADVANCED_DATAPATH"
+        )
+      end
+
       it "has the expected binaryAuthorization config" do
         expect(data['binaryAuthorization']).to eq({
           "enabled" => true,