Skip to content

Commit

Permalink
feat(cluster.tf): add support for setting cgroup mode (#2001)
Browse files Browse the repository at this point in the history
Co-authored-by: Chico Venancio <[email protected]>
Co-authored-by: Andrew Peabody <[email protected]>
  • Loading branch information
3 people authored Aug 6, 2024
1 parent 65ccfa0 commit 3fc4db4
Show file tree
Hide file tree
Showing 31 changed files with 183 additions and 8 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion autogen/main/cluster.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -1038,14 +1038,16 @@ resource "google_container_node_pool" "windows_pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : {cgroup = local.node_pools_cgroup_mode[each.value["name"]]}
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}
{% endif %}
Expand Down
10 changes: 10 additions & 0 deletions autogen/main/variables.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,16 @@ variable "node_pools_linux_node_configs_sysctls" {
default-node-pool = {}
}
}
variable "node_pools_cgroup_mode" {
type = map(string)
description = "Map of strings containing cgroup node config by node-pool name"

# Default is being set in variables_defaults.tf
default = {
all = ""
default-node-pool = ""
}
}
{% endif %}

variable "enable_cost_allocation" {
Expand Down
9 changes: 9 additions & 0 deletions autogen/main/variables_defaults.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -128,5 +128,14 @@ locals {
),
var.node_pools_linux_node_configs_sysctls
)
node_pools_cgroup_mode = merge(
{ all = "" },
{ default-node-pool = "" },
zipmap(
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : ""]
),
var.node_pools_cgroup_mode
)
}
{% endif %}
4 changes: 3 additions & 1 deletion cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -733,14 +733,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
1 change: 1 addition & 0 deletions modules/beta-private-cluster-update-variant/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion modules/beta-private-cluster-update-variant/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -911,14 +911,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
10 changes: 10 additions & 0 deletions modules/beta-private-cluster-update-variant/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,16 @@ variable "node_pools_linux_node_configs_sysctls" {
default-node-pool = {}
}
}
variable "node_pools_cgroup_mode" {
type = map(string)
description = "Map of strings containing cgroup node config by node-pool name"

# Default is being set in variables_defaults.tf
default = {
all = ""
default-node-pool = ""
}
}

variable "enable_cost_allocation" {
type = bool
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,13 @@ locals {
),
var.node_pools_linux_node_configs_sysctls
)
node_pools_cgroup_mode = merge(
{ all = "" },
{ default-node-pool = "" },
zipmap(
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : ""]
),
var.node_pools_cgroup_mode
)
}
1 change: 1 addition & 0 deletions modules/beta-private-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion modules/beta-private-cluster/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -831,14 +831,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
10 changes: 10 additions & 0 deletions modules/beta-private-cluster/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,16 @@ variable "node_pools_linux_node_configs_sysctls" {
default-node-pool = {}
}
}
variable "node_pools_cgroup_mode" {
type = map(string)
description = "Map of strings containing cgroup node config by node-pool name"

# Default is being set in variables_defaults.tf
default = {
all = ""
default-node-pool = ""
}
}

variable "enable_cost_allocation" {
type = bool
Expand Down
9 changes: 9 additions & 0 deletions modules/beta-private-cluster/variables_defaults.tf
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,13 @@ locals {
),
var.node_pools_linux_node_configs_sysctls
)
node_pools_cgroup_mode = merge(
{ all = "" },
{ default-node-pool = "" },
zipmap(
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : ""]
),
var.node_pools_cgroup_mode
)
}
1 change: 1 addition & 0 deletions modules/beta-public-cluster-update-variant/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion modules/beta-public-cluster-update-variant/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -890,14 +890,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
10 changes: 10 additions & 0 deletions modules/beta-public-cluster-update-variant/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,16 @@ variable "node_pools_linux_node_configs_sysctls" {
default-node-pool = {}
}
}
variable "node_pools_cgroup_mode" {
type = map(string)
description = "Map of strings containing cgroup node config by node-pool name"

# Default is being set in variables_defaults.tf
default = {
all = ""
default-node-pool = ""
}
}

variable "enable_cost_allocation" {
type = bool
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,13 @@ locals {
),
var.node_pools_linux_node_configs_sysctls
)
node_pools_cgroup_mode = merge(
{ all = "" },
{ default-node-pool = "" },
zipmap(
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : ""]
),
var.node_pools_cgroup_mode
)
}
1 change: 1 addition & 0 deletions modules/beta-public-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion modules/beta-public-cluster/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -810,14 +810,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
10 changes: 10 additions & 0 deletions modules/beta-public-cluster/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,16 @@ variable "node_pools_linux_node_configs_sysctls" {
default-node-pool = {}
}
}
variable "node_pools_cgroup_mode" {
type = map(string)
description = "Map of strings containing cgroup node config by node-pool name"

# Default is being set in variables_defaults.tf
default = {
all = ""
default-node-pool = ""
}
}

variable "enable_cost_allocation" {
type = bool
Expand Down
9 changes: 9 additions & 0 deletions modules/beta-public-cluster/variables_defaults.tf
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,13 @@ locals {
),
var.node_pools_linux_node_configs_sysctls
)
node_pools_cgroup_mode = merge(
{ all = "" },
{ default-node-pool = "" },
zipmap(
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : ""]
),
var.node_pools_cgroup_mode
)
}
1 change: 1 addition & 0 deletions modules/private-cluster-update-variant/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion modules/private-cluster-update-variant/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -834,14 +834,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
10 changes: 10 additions & 0 deletions modules/private-cluster-update-variant/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,16 @@ variable "node_pools_linux_node_configs_sysctls" {
default-node-pool = {}
}
}
variable "node_pools_cgroup_mode" {
type = map(string)
description = "Map of strings containing cgroup node config by node-pool name"

# Default is being set in variables_defaults.tf
default = {
all = ""
default-node-pool = ""
}
}

variable "enable_cost_allocation" {
type = bool
Expand Down
9 changes: 9 additions & 0 deletions modules/private-cluster-update-variant/variables_defaults.tf
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,13 @@ locals {
),
var.node_pools_linux_node_configs_sysctls
)
node_pools_cgroup_mode = merge(
{ all = "" },
{ default-node-pool = "" },
zipmap(
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : ""]
),
var.node_pools_cgroup_mode
)
}
1 change: 1 addition & 0 deletions modules/private-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ Then perform the following commands on the root folder:
| network\_tags | (Optional) - List of network tags applied to auto-provisioned node pools. | `list(string)` | `[]` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
| node\_pools | List of maps containing node pools | `list(map(any))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
| node\_pools\_cgroup\_mode | Map of strings containing cgroup node config by node-pool name | `map(string)` | <pre>{<br> "all": "",<br> "default-node-pool": ""<br>}</pre> | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_linux\_node\_configs\_sysctls | Map of maps containing linux node config sysctls by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
Expand Down
4 changes: 3 additions & 1 deletion modules/private-cluster/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -754,14 +754,16 @@ resource "google_container_node_pool" "pools" {
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
local.node_pools_linux_node_configs_sysctls[each.value["name"]],
local.node_pools_cgroup_mode[each.value["name"]] == "" ? {} : { cgroup = local.node_pools_cgroup_mode[each.value["name"]] }
)) != 0 ? [1] : []

content {
sysctls = merge(
local.node_pools_linux_node_configs_sysctls["all"],
local.node_pools_linux_node_configs_sysctls[each.value["name"]]
)
cgroup_mode = local.node_pools_cgroup_mode[each.value["name"]] == "" ? null : local.node_pools_cgroup_mode[each.value["name"]]
}
}

Expand Down
Loading

0 comments on commit 3fc4db4

Please sign in to comment.