OrgPolicy module (factory) using new org-policy API, #698

README.md

@@ -29,7 +29,7 @@ The current list of modules supports most of the core foundational and networkin
 
 Currently available modules:
 
-- **foundational** - [folder](./modules/folder), [organization](./modules/organization), [project](./modules/project), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [billing budget](./modules/billing-budget), [naming convention](./modules/naming-convention), [projects-data-source](./modules/projects-data-source)
+- **foundational** - [folder](./modules/folder), [organization](./modules/organization), [project](./modules/project), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [billing budget](./modules/billing-budget), [naming convention](./modules/naming-convention), [projects-data-source](./modules/projects-data-source), [organization-policy](./modules/organization-policy)
 - **networking** - [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN static](./modules/net-vpn-static), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [NAT](./modules/net-cloudnat), [address reservation](./modules/net-address), [DNS](./modules/dns), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [Service Directory](./modules/service-directory), [Cloud Endpoints](./modules/endpoints)
 - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [GKE cluster](./modules/gke-cluster), [GKE nodepool](./modules/gke-nodepool), [GKE hub](./modules/gke-hub), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid)
 - **data** - [GCS](./modules/gcs), [BigQuery dataset](./modules/bigquery-dataset), [Pub/Sub](./modules/pubsub), [Datafusion](./modules/datafusion), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag)

examples/factories/net-vpc-firewall-yaml/README.md

@@ -12,7 +12,7 @@ Nested folder structure for yaml configurations is optionally supported, which a
 
 ```hcl
 module "prod-firewall" {
-  source = "./modules/net-vpc-firewall-yaml"
+  source = "./examples/factories/net-vpc-firewall-yaml"
 
   project_id         = "my-prod-project"
   network            = "my-prod-network"
@@ -27,7 +27,7 @@ module "prod-firewall" {
 }
 
 module "dev-firewall" {
-  source = "./modules/net-vpc-firewall-yaml"
+  source = "./examples/factories/net-vpc-firewall-yaml"
 
   project_id         = "my-dev-project"
   network            = "my-dev-network"

modules/README.md

@@ -37,6 +37,7 @@ These modules are used in the examples included in this repository. If you are u
 - [project](./project)
 - [projects-data-source](./projects-data-source)
 - [service account](./iam-service-account)
+- [organization policy](./organization-policy)
 
 ## Networking modules
 

modules/organization-policy/README.md

@@ -0,0 +1,167 @@
+# Google Cloud Organization Policy
+
+This module allows creation and management of [GCP Organization Policies](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) by defining them in a well formatted `yaml` files or with HCL.
+
+Yaml based factory can simplify centralized management of Org Policies for a DevSecOps team by providing a simple way to define/structure policies and exclusions.
+
+> **_NOTE:_**  This module uses experimental feature `module_variable_optional_attrs` which will be included into [terraform release 1.3](https://github.com/hashicorp/terraform/releases/tag/v1.3.0-alpha20220706).
+
+## Example
+
+### Terraform code
+
+```hcl
+# using configuration provided in a set of yaml files
+module "org-policy-factory" {
+  source = "./modules/organization-policy"
+
+  config_directory = "./policies"
+}
+
+# using configuration provided in the module variable
+module "org-policy" {
+  source = "./modules/organization-policy"
+
+  policies = {
+    "folders/1234567890" = {
+      # enforce boolean policy with no conditions
+      "iam.disableServiceAccountKeyUpload" = {
+        rules = [
+          {
+            enforce = true
+          }
+        ]
+      },
+      # Deny All for compute.vmCanIpForward policy
+      "compute.vmCanIpForward" = {
+        inherit_from_parent = false
+        rules = [
+          deny = [] # stands for deny_all
+        ]
+      }
+    },
+    "organizations/1234567890" = {
+      # allow only internal ingress when match condition env=prod
+      "run.allowedIngress" = {
+        rules = [
+          {
+            allow = ["internal"]
+            condition = {
+              description= "allow ingress"
+              expression = "resource.matchTag('123456789/environment', 'prod')"
+              title = "allow-for-prod-org"
+            }
+          }
+        ]
+      }
+    }
+  } 
+}
+# tftest skip
+```
+
+## Org Policy definition format and structure
+
+### Structure of `policies` variable
+
+```hcl
+policies = {
+  "parent_id" = { # parent id in format projects/project-id, folders/1234567890 or organizations/1234567890.
+    "policy_name" = { # policy constraint id, for example compute.vmExternalIpAccess.
+      inherit_from_parent = true|false # (Optional) Only for list constraints. Determines the inheritance behavior for this policy.
+      reset               = true|false # (Optional) Ignores policies set above this resource and restores the constraint_default enforcement behavior.
+      rules               = [ # Up to 10 PolicyRules are allowed.
+        {
+          allow = ["value1", "value2"] # (Optional) Only for list constraints. Stands for `allow_all` if set to empty list `[]` or to `values.allowed_values` if set to a list of values
+          denyl = ["value3", "value4"] # (Optional) Only for list constraints. Stands for `deny_all` if set to empty list `[]` or to `values.denied_values` if set to a list of values
+          enforce   = true|false       # (Optional) Only for boolean constraints. If true, then the Policy is enforced.
+          condition = {                # (Optional) A condition which determines whether this rule is used in the evaluation of the policy.
+            description = "Condition description" # (Optional)
+            expression  = "Condition expression"  # (Optional) For example "resource.matchTag('123456789/environment', 'prod')".
+            location    = "policy-error.log"      # (Optional) String indicating the location of the expression for error reporting.
+            title       = "condition-title"       # (Optional)
+          }
+        }
+      ]
+    }
+  }
+}
+# tftest skip
+```
+
+### Structure of configuration provided in a yaml file/s
+
+Configuration should be placed in a set of yaml files in the config directory. Policy entry structure as follows:
+
+```yaml
+parent_id: # parent id in format projects/project-id, folders/1234567890 or organizations/1234567890.
+  policy_name1: # policy constraint id, for example compute.vmExternalIpAccess.
+    inherit_from_parent: true|false # (Optional) Only for list constraints. Determines the inheritance behavior for this policy.
+    reset: true|false               # (Optional) Ignores policies set above this resource and restores the constraint_default enforcement behavior.
+    rules:
+      - allow: ["value1", "value2"] # (Optional) Only for list constraints. Stands for `allow_all` if set to empty list `[]` or to `values.allowed_values` if set to a list of values
+        deny: ["value3", "value4"] # (Optional) Only for list constraints. Stands for `deny_all` if set to empty list `[]` or to `values.denied_values` if set to a list of values
+        enforce: true|false   # (Optional) Only for boolean constraints. If true, then the Policy is enforced.
+        condition:            # (Optional) A condition which determines whether this rule is used in the evaluation of the policy.
+          description: Condition description   # (Optional)
+          expression: Condition expression     # (Optional) For example resource.matchTag("123456789/environment", "prod")
+          location: policy-error.log           # (Optional) String indicating the location of the expression for error reporting.
+          title: condition-title               # (Optional)
+```
+
+Module allows policies to be distributed into multiple yaml files for a better management and navigation.
+
+```bash
+├── org-policies
+│   ├── baseline.yaml
+│   ├── image-import-projects.yaml
+│   └── exclusions.yaml
+```
+
+Organization policies example yaml configuration
+
+```bash
+cat ./policies/baseline.yaml
+organizations/1234567890:
+  constraints/compute.vmExternalIpAccess:
+    rules:
+      - deny_all: true
+folders/1234567890:
+  compute.vmCanIpForward:
+    inherit_from_parent: false
+    reset: false
+    rules:
+      - allow: [] # Stands for allow_all = true
+projects/my-project-id:
+  run.allowedIngress:
+    inherit_from_parent: true
+    rules:
+      - condition:
+          description: allow internal ingress
+          expression: resource.matchTag("123456789/environment", "prod")
+          location: test.log
+          title: allow-for-prod
+        values: 
+          allowed_values: ['internal']
+  iam.allowServiceAccountCredentialLifetimeExtension:
+    rules:
+      - deny: [] # Stands for deny_all = true
+  compute.disableGlobalLoadBalancing:
+    reset: true
+```
+<!-- BEGIN TFDOC -->
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [config_directory](variables.tf#L17) | Paths to a folder where organization policy configs are stored in yaml format. Files suffix must be `.yaml`. | <code>string</code> |  | <code>null</code> |
+| [policies](variables.tf#L23) | Organization policies keyed by parent in format `projects/project-id`, `folders/1234567890` or `organizations/1234567890`. | <code title="map&#40;map&#40;object&#40;&#123;&#10;  inherit_from_parent &#61; optional&#40;bool&#41; &#35; List policy only.&#10;  reset               &#61; optional&#40;bool&#41;&#10;  rules &#61; optional&#40;&#10;    list&#40;object&#40;&#123;&#10;      allow   &#61; optional&#40;list&#40;string&#41;&#41; &#35; List policy only. Stands for &#96;allow_all&#96; if set to empty list &#96;&#91;&#93;&#96; or to &#96;values.allowed_values&#96; if set to a list of values &#10;      deny    &#61; optional&#40;list&#40;string&#41;&#41; &#35; List policy only. Stands for &#96;deny_all&#96; if set to empty list &#96;&#91;&#93;&#96; or to &#96;values.denied_values&#96; if set to a list of values&#10;      enforce &#61; optional&#40;bool&#41;         &#35; Boolean policy only.    &#10;      condition &#61; optional&#40;&#10;        object&#40;&#123;&#10;          description &#61; optional&#40;string&#41;&#10;          expression  &#61; optional&#40;string&#41;&#10;          location    &#61; optional&#40;string&#41;&#10;          title       &#61; optional&#40;string&#41;&#10;        &#125;&#41;&#10;      &#41;&#10;    &#125;&#41;&#41;&#10;  &#41;&#10;&#125;&#41;&#41;&#41;">map&#40;map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+
+## Outputs
+
+| name | description | sensitive |
+|---|---|:---:|
+| [policies](outputs.tf#L17) | Organization policies. |  |
+
+<!-- END TFDOC -->

modules/organization-policy/main.tf

@@ -0,0 +1,102 @@
+/**
+ * Copyright 2022 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+locals {
+  policy_files = var.config_directory == null ? [] : concat(
+    [
+      for config_file in fileset("${path.root}/${var.config_directory}", "**/*.yaml") :
+      "${path.root}/${var.config_directory}/${config_file}"
+    ]
+  )
+
+  policies_raw = merge(
+    merge(
+      [
+        for config_file in local.policy_files :
+        try(yamldecode(file(config_file)), {})
+      ]...
+  ), var.policies)
+
+  policies_list = flatten([
+    for parent, policies in local.policies_raw : [
+      for policy_name, policy in policies : {
+        parent              = parent,
+        policy_name         = policy_name,
+        inherit_from_parent = try(policy["inherit_from_parent"], null),
+        reset               = try(policy["reset"], null),
+        rules = [
+          for rule in try(policy["rules"], []) : {
+            allow_all = try(length(rule["allow"]), -1) == 0 ? "TRUE" : null
+            deny_all  = try(length(rule["deny"]), -1) == 0 ? "TRUE" : null
+            enforce = try(rule["enforce"], null) == true ? "TRUE" : try(
+            rule["enforce"], null) == false ? "FALSE" : null,
+            condition = try(rule["condition"], null) != null ? {
+              description = try(rule["condition"]["description"], null),
+              expression  = try(rule["condition"]["expression"], null),
+              location    = try(rule["condition"]["location"], null),
+              title       = try(rule["condition"]["title"], null)
+            } : null,
+            values = try(length(rule["allow"]), 0) > 0 || try(length(rule["deny"]), 0) > 0 ? {
+              allowed_values = try(length(rule["allow"]), 0) > 0 ? rule["allow"] : null
+              denied_values  = try(length(rule["deny"]), 0) > 0 ? rule["deny"] : null
+            } : null
+          }
+        ]
+      }
+    ]
+  ])
+
+  policies_map = {
+    for item in local.policies_list :
+    format("%s-%s", item["parent"], item["policy_name"]) => item
+  }
+}
+
+resource "google_org_policy_policy" "primary" {
+  for_each = local.policies_map
+  name     = format("%s/policies/%s", each.value.parent, each.value.policy_name)
+  parent   = each.value.parent
+
+  spec {
+    inherit_from_parent = each.value.inherit_from_parent
+    reset               = each.value.reset
+    dynamic "rules" {
+      for_each = each.value.rules
+      content {
+        allow_all = rules.value.allow_all
+        deny_all  = rules.value.deny_all
+        enforce   = rules.value.enforce
+        dynamic "condition" {
+          for_each = rules.value.condition != null ? [""] : []
+          content {
+            description = rules.value.condition.description
+            expression  = rules.value.condition.expression
+            location    = rules.value.condition.location
+            title       = rules.value.condition.title
+          }
+        }
+        dynamic "values" {
+          for_each = rules.value.values != null ? [""] : []
+          content {
+            allowed_values = rules.value.values.allowed_values
+            denied_values  = rules.value.values.denied_values
+          }
+        }
+      }
+    }
+  }
+}

modules/organization-policy/outputs.tf

@@ -0,0 +1,20 @@
+/**
+ * Copyright 2022 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+output "policies" {
+  description = "Organization policies."
+  value       = google_org_policy_policy.primary
+}

modules/organization-policy/variables.tf

@@ -0,0 +1,45 @@
+/**
+ * Copyright 2022 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "config_directory" {
+  description = "Paths to a folder where organization policy configs are stored in yaml format. Files suffix must be `.yaml`."
+  type        = string
+  default     = null
+}
+
+variable "policies" {
+  description = "Organization policies keyed by parent in format `projects/project-id`, `folders/1234567890` or `organizations/1234567890`."
+  type = map(map(object({
+    inherit_from_parent = optional(bool) # List policy only.
+    reset               = optional(bool)
+    rules = optional(
+      list(object({
+        allow   = optional(list(string)) # List policy only. Stands for `allow_all` if set to empty list `[]` or to `values.allowed_values` if set to a list of values 
+        deny    = optional(list(string)) # List policy only. Stands for `deny_all` if set to empty list `[]` or to `values.denied_values` if set to a list of values
+        enforce = optional(bool)         # Boolean policy only.    
+        condition = optional(
+          object({
+            description = optional(string)
+            expression  = optional(string)
+            location    = optional(string)
+            title       = optional(string)
+          })
+        )
+      }))
+    )
+  })))
+  default = {}
+}