diff --git a/.gitignore b/.gitignore index cbf110dd29..a4e6b621fd 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ +venv/* +*/venv/* **/.terraform **/terraform.tfstate* **/terraform.tfvars diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a71669a2d..609382f2c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,10 @@ All notable changes to this project will be documented in this file. ### BLUEPRINTS +- [[#1838](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1838)] Simplify #1836 fix, Avoid map-related casting errors in project factory ([wiktorn](https://github.com/wiktorn)) +- [[#1836](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1836)] **incompatible change:** Avoid map-related casting errors in project factory ([ludoo](https://github.com/ludoo)) +- [[#1832](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1832)] [Minimal Data Platform] Fix Landing and curated IAM ([lcaggio](https://github.com/lcaggio)) +- [[#1825](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1825)] Handling SQL IP address issue ([aurelienlegrand](https://github.com/aurelienlegrand)) - [[#1821](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1821)] [net-address] enable ipv6 ([LucaPrete](https://github.com/LucaPrete)) - [[#1814](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1814)] **incompatible change:** Allow specifying arbitrary project roles for service accounts in project factory ([ludoo](https://github.com/ludoo)) - [[#1812](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1812)] Stop wrapping yamldecode with try() ([sruffilli](https://github.com/sruffilli)) @@ -21,12 +25,15 @@ All notable changes to this project will be documented in this file. ### DOCUMENTATION +- [[#1832](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1832)] [Minimal Data Platform] Fix Landing and curated IAM ([lcaggio](https://github.com/lcaggio)) +- [[#1831](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1831)] Update wording in FAST and gcve module READMEs ([bluPhy](https://github.com/bluPhy)) - [[#1782](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1782)] Add upper cap to versions, update copyright notices ([sruffilli](https://github.com/sruffilli)) - [[#1773](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1773)] Add service usage consumer role to IaC SAs, refactor delegated grants in FAST ([ludoo](https://github.com/ludoo)) - [[#1743](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1743)] Billing account module ([ludoo](https://github.com/ludoo)) ### FAST +- [[#1836](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1836)] **incompatible change:** Avoid map-related casting errors in project factory ([ludoo](https://github.com/ludoo)) - [[#1818](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1818)] FAST: rename VPC-related files to `net-*` ([sruffilli](https://github.com/sruffilli)) - [[#1812](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1812)] Stop wrapping yamldecode with try() ([sruffilli](https://github.com/sruffilli)) - [[#1810](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1810)] FAST: Add access transparency logs to the default sinks ([sruffilli](https://github.com/sruffilli)) @@ -49,6 +56,14 @@ All notable changes to this project will be documented in this file. ### MODULES +- [[#1841](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1841)] Fix modules to support new Apigee X environment types ([Teodelas](https://github.com/Teodelas)) +- [[#1842](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1842)] Bump provider version to 5.4.0 ([wiktorn](https://github.com/wiktorn)) +- [[#1823](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1823)] Add end-to-end tests for project module ([wiktorn](https://github.com/wiktorn)) +- [[#1837](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1837)] Added envoy as SNI dynamic forward proxy to cloud-config-container ([apichick](https://github.com/apichick)) +- [[#1839](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1839)] Added create_before_destroy = true for self-managed certificates ([apichick](https://github.com/apichick)) +- [[#1833](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1833)] Net VPC Peering: added stack_type field ([cmalpe](https://github.com/cmalpe)) +- [[#1826](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1826)] Add public_access_prevention field to GCS module ([devuonocar](https://github.com/devuonocar)) +- [[#1817](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1817)] KMS module: Import job feature ([cmalpe](https://github.com/cmalpe)) - [[#1822](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1822)] Billing budget factory ([ludoo](https://github.com/ludoo)) - [[#1821](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1821)] [net-address] enable ipv6 ([LucaPrete](https://github.com/LucaPrete)) - [[#1820](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1820)] Added iam_bindings and iam_bindings_additive to apigee module ([apichick](https://github.com/apichick)) diff --git a/blueprints/data-solutions/cloudsql-multiregion/cloudsql.tf b/blueprints/data-solutions/cloudsql-multiregion/cloudsql.tf index e25812df5b..5dab5d6b60 100644 --- a/blueprints/data-solutions/cloudsql-multiregion/cloudsql.tf +++ b/blueprints/data-solutions/cloudsql-multiregion/cloudsql.tf @@ -34,7 +34,9 @@ module "db" { } databases = [var.postgres_database] users = { - postgres = var.postgres_user_password + postgres = { + password = var.postgres_user_password + } } } diff --git a/blueprints/data-solutions/data-platform-minimal/01-landing.tf b/blueprints/data-solutions/data-platform-minimal/01-landing.tf index 2b4d5165fa..3539e001ac 100644 --- a/blueprints/data-solutions/data-platform-minimal/01-landing.tf +++ b/blueprints/data-solutions/data-platform-minimal/01-landing.tf @@ -22,8 +22,9 @@ locals { "roles/storage.objectViewer" = [ module.processing-sa-cmp-0.iam_email ] - "roles/storage.objectAdmin" = [ - module.processing-sa-0.iam_email + "roles/storage.admin" = [ + module.processing-sa-0.iam_email, + local.groups_iam.data-engineers ] } # this only works because the service account module uses a static output diff --git a/blueprints/data-solutions/data-platform-minimal/03-curated.tf b/blueprints/data-solutions/data-platform-minimal/03-curated.tf index 81112c9b88..faf9301129 100644 --- a/blueprints/data-solutions/data-platform-minimal/03-curated.tf +++ b/blueprints/data-solutions/data-platform-minimal/03-curated.tf @@ -58,11 +58,11 @@ locals { ] "roles/storage.objectViewer" = [ module.cur-sa-0.iam_email, - local.groups_iam.data-analysts, - local.groups_iam.data-engineers + local.groups_iam.data-analysts ] - "roles/storage.objectAdmin" = [ - module.processing-sa-0.iam_email + "roles/storage.admin" = [ + module.processing-sa-0.iam_email, + local.groups_iam.data-engineers ] } # this only works because the service account module uses a static output diff --git a/blueprints/data-solutions/data-platform-minimal/IAM.md b/blueprints/data-solutions/data-platform-minimal/IAM.md index d5c1ccbb34..ea796fbf0b 100644 --- a/blueprints/data-solutions/data-platform-minimal/IAM.md +++ b/blueprints/data-solutions/data-platform-minimal/IAM.md @@ -16,15 +16,16 @@ Legend: + additive, • conditional. | members | roles | |---|---| |gcp-data-analysts
group|[roles/bigquery.dataViewer](https://cloud.google.com/iam/docs/understanding-roles#bigquery.dataViewer)
[roles/bigquery.jobUser](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser)
[roles/datacatalog.tagTemplateViewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.tagTemplateViewer)
[roles/datacatalog.viewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.viewer)
[roles/storage.objectViewer](https://cloud.google.com/iam/docs/understanding-roles#storage.objectViewer) | -|gcp-data-engineers
group|[roles/bigquery.dataViewer](https://cloud.google.com/iam/docs/understanding-roles#bigquery.dataViewer)
[roles/bigquery.jobUser](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser)
[roles/datacatalog.tagTemplateViewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.tagTemplateViewer)
[roles/datacatalog.viewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.viewer)
[roles/storage.objectViewer](https://cloud.google.com/iam/docs/understanding-roles#storage.objectViewer) | +|gcp-data-engineers
group|[roles/bigquery.dataViewer](https://cloud.google.com/iam/docs/understanding-roles#bigquery.dataViewer)
[roles/bigquery.jobUser](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser)
[roles/datacatalog.tagTemplateViewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.tagTemplateViewer)
[roles/datacatalog.viewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.viewer)
[roles/storage.admin](https://cloud.google.com/iam/docs/understanding-roles#storage.admin) | |SERVICE_IDENTITY_service-networking
serviceAccount|[roles/servicenetworking.serviceAgent](https://cloud.google.com/iam/docs/understanding-roles#servicenetworking.serviceAgent) +| |cur-sa-0
serviceAccount|[roles/bigquery.dataViewer](https://cloud.google.com/iam/docs/understanding-roles#bigquery.dataViewer)
[roles/bigquery.jobUser](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser)
[roles/datacatalog.tagTemplateViewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.tagTemplateViewer)
[roles/datacatalog.viewer](https://cloud.google.com/iam/docs/understanding-roles#datacatalog.viewer)
[roles/storage.objectViewer](https://cloud.google.com/iam/docs/understanding-roles#storage.objectViewer) | -|prc-0
serviceAccount|[roles/bigquery.dataOwner](https://cloud.google.com/iam/docs/understanding-roles#bigquery.dataOwner)
[roles/bigquery.jobUser](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser)
[roles/storage.objectAdmin](https://cloud.google.com/iam/docs/understanding-roles#storage.objectAdmin) | +|prc-0
serviceAccount|[roles/bigquery.dataOwner](https://cloud.google.com/iam/docs/understanding-roles#bigquery.dataOwner)
[roles/bigquery.jobUser](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser)
[roles/storage.admin](https://cloud.google.com/iam/docs/understanding-roles#storage.admin) | ## Project lnd | members | roles | |---|---| +|gcp-data-engineers
group|[roles/storage.admin](https://cloud.google.com/iam/docs/understanding-roles#storage.admin) | |lnd-sa-0
serviceAccount|[roles/storage.objectCreator](https://cloud.google.com/iam/docs/understanding-roles#storage.objectCreator) | |prc-0
serviceAccount|[roles/storage.objectAdmin](https://cloud.google.com/iam/docs/understanding-roles#storage.objectAdmin) | |prc-cmp-0
serviceAccount|[roles/storage.objectViewer](https://cloud.google.com/iam/docs/understanding-roles#storage.objectViewer) | diff --git a/blueprints/data-solutions/sqlserver-alwayson/secrets.tf b/blueprints/data-solutions/sqlserver-alwayson/secrets.tf index 2a0bba8d6c..f25871deb8 100644 --- a/blueprints/data-solutions/sqlserver-alwayson/secrets.tf +++ b/blueprints/data-solutions/sqlserver-alwayson/secrets.tf @@ -19,7 +19,9 @@ module "secret-manager" { source = "../../../modules/secret-manager" project_id = var.project_id secrets = { - (local.ad_user_password_secret) = null + (local.ad_user_password_secret) = { + locations = null + } } versions = { (local.ad_user_password_secret) = { diff --git a/blueprints/data-solutions/vertex-mlops/ci-cd.tf b/blueprints/data-solutions/vertex-mlops/ci-cd.tf index 3c7c863151..28fb320bc1 100644 --- a/blueprints/data-solutions/vertex-mlops/ci-cd.tf +++ b/blueprints/data-solutions/vertex-mlops/ci-cd.tf @@ -57,10 +57,12 @@ module "secret-manager" { project_id = module.project.project_id source = "../../../modules/secret-manager" secrets = { - github-key = [var.region] - } - encryption_key = { - "${var.region}" = var.service_encryption_keys.secretmanager + github-key = { + locations = [var.region] + keys = { + "${var.region}" = var.service_encryption_keys.secretmanager + } + } } iam = { github-key = { diff --git a/blueprints/factories/project-factory/README.md b/blueprints/factories/project-factory/README.md index d2d07c43d6..2e67158277 100644 --- a/blueprints/factories/project-factory/README.md +++ b/blueprints/factories/project-factory/README.md @@ -55,11 +55,9 @@ module "project-factory" { prefix = "test-pf" } # location where the yaml files are read from - factory_data = { - data_path = "data" - } + factory_data_path = "data" } -# tftest modules=6 resources=17 files=prj-app-1,prj-app-2 +# tftest modules=7 resources=26 files=prj-app-1,prj-app-2,prj-app-3 inventory=example.yaml ``` ```yaml @@ -72,7 +70,8 @@ service_encryption_key_ids: compute: - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce services: -- storage.googleapis.com + - container.googleapis.com + - storage.googleapis.com service_accounts: app-1-be: iam_project_roles: @@ -86,29 +85,35 @@ service_accounts: ```yaml labels: - app: app-1 + app: app-2 team: foo parent: folders/12345678 service_accounts: app-2-be: {} -org_policies: - compute.disableGuestAttributesAccess: - rules: - - enforce: false - iam.disableServiceAccountKeyCreation: - rules: - - enforce: false +services: +- compute.googleapis.com +- run.googleapis.com +- storage.googleapis.com shared_vpc_service_config: host_project: foo-host # tftest-file id=prj-app-2 path=data/prj-app-2.yaml ``` + +```yaml +parent: folders/12345678 +services: +- run.googleapis.com +- storage.googleapis.com + +# tftest-file id=prj-app-3 path=data/prj-app-3.yaml +``` ## Variables | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [factory_data](variables.tf#L88) | Project data from either YAML files or externally parsed data. | object({…}) | ✓ | | +| [factory_data_path](variables.tf#L88) | Path to folder with YAML project description data files. | string | ✓ | | | [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | object({…}) | | {} | | [data_merges](variables.tf#L46) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | object({…}) | | {} | | [data_overrides](variables.tf#L66) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | object({…}) | | {} | @@ -120,3 +125,53 @@ shared_vpc_service_config: | [projects](outputs.tf#L17) | Project module outputs. | | | [service_accounts](outputs.tf#L22) | Service account emails. | | +## Tests + +These tests validate fixes to the project factory. + +```hcl +module "project-factory" { + source = "./fabric/blueprints/factories/project-factory" + data_defaults = { + billing_account = "012345-67890A-ABCDEF" + } + data_merges = { + labels = { + owner = "foo" + } + services = [ + "compute.googleapis.com" + ] + } + data_overrides = { + prefix = "foo" + } + factory_data_path = "data" +} +# tftest modules=4 resources=14 files=test-0,test-1,test-2 +``` + +```yaml +parent: folders/1234567890 +services: + - iam.googleapis.com + - contactcenteraiplatform.googleapis.com + - container.googleapis.com +# tftest-file id=test-0 path=data/test-0.yaml +``` + +```yaml +parent: folders/1234567890 +services: + - iam.googleapis.com + - contactcenteraiplatform.googleapis.com +# tftest-file id=test-1 path=data/test-1.yaml +``` + +```yaml +parent: folders/1234567890 +services: + - iam.googleapis.com + - storage.googleapis.com +# tftest-file id=test-2 path=data/test-2.yaml +``` diff --git a/blueprints/factories/project-factory/factory.tf b/blueprints/factories/project-factory/factory.tf index e0351a0aa1..eabb551ad1 100644 --- a/blueprints/factories/project-factory/factory.tf +++ b/blueprints/factories/project-factory/factory.tf @@ -16,15 +16,13 @@ locals { _data = ( - var.factory_data.data != null - ? var.factory_data.data - : { - for f in fileset("${local._data_path}", "**/*.yaml") : + { + for f in fileset(local._data_path, "**/*.yaml") : trimsuffix(f, ".yaml") => yamldecode(file("${local._data_path}/${f}")) } ) - _data_path = var.factory_data.data_path == null ? null : pathexpand( - var.factory_data.data_path + _data_path = var.factory_data_path == null ? null : pathexpand( + var.factory_data_path ) projects = { for k, v in local._data : k => merge(v, { diff --git a/blueprints/factories/project-factory/variables.tf b/blueprints/factories/project-factory/variables.tf index 55578562f9..a0ff81ebd3 100644 --- a/blueprints/factories/project-factory/variables.tf +++ b/blueprints/factories/project-factory/variables.tf @@ -85,18 +85,8 @@ variable "data_overrides" { default = {} } -variable "factory_data" { - description = "Project data from either YAML files or externally parsed data." - type = object({ - data = optional(map(any)) - data_path = optional(string) - }) - nullable = false - validation { - condition = ( - (var.factory_data.data != null ? 1 : 0) + - (var.factory_data.data_path != null ? 1 : 0) - ) == 1 - error_message = "One of data or data_path needs to be set." - } +variable "factory_data_path" { + description = "Path to folder with YAML project description data files." + type = string + nullable = false } diff --git a/blueprints/third-party-solutions/phpipam/cloudsql.tf b/blueprints/third-party-solutions/phpipam/cloudsql.tf index 24a47b661c..f3b7a78665 100644 --- a/blueprints/third-party-solutions/phpipam/cloudsql.tf +++ b/blueprints/third-party-solutions/phpipam/cloudsql.tf @@ -27,6 +27,8 @@ module "cloudsql" { region = var.region tier = local.cloudsql_conf.tier users = { - "${local.cloudsql_conf.user}" = var.cloudsql_password + "${local.cloudsql_conf.user}" = { + password = var.cloudsql_password + } } } diff --git a/blueprints/third-party-solutions/wordpress/cloudrun/cloudsql.tf b/blueprints/third-party-solutions/wordpress/cloudrun/cloudsql.tf index 4ed2ed1992..2ebe9e1436 100644 --- a/blueprints/third-party-solutions/wordpress/cloudrun/cloudsql.tf +++ b/blueprints/third-party-solutions/wordpress/cloudrun/cloudsql.tf @@ -61,7 +61,9 @@ module "cloudsql" { tier = local.cloudsql_conf.tier databases = [local.cloudsql_conf.db] users = { - "${local.cloudsql_conf.user}" = var.cloudsql_password + "${local.cloudsql_conf.user}" = { + password = var.cloudsql_password + } } deletion_protection = false } diff --git a/default-versions.tf b/default-versions.tf index af3463955f..3adb51d3bd 100644 --- a/default-versions.tf +++ b/default-versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/fast/stages/0-bootstrap/README.md b/fast/stages/0-bootstrap/README.md index 1a9f1bbbd8..9ed7528916 100644 --- a/fast/stages/0-bootstrap/README.md +++ b/fast/stages/0-bootstrap/README.md @@ -72,7 +72,7 @@ A full reference of IAM roles managed by this stage [is available here](./IAM.md It's often desirable to have organization policies deployed before any other resource in the org, so as to ensure compliance with specific requirements (e.g. location restrictions), or control the configuration of specific resources (e.g. default network at project creation or service account grants). -To cover this use case, organization policies have been moved from the resource management to the bootstrap stage in FAST versions after 26.0.0. They are managed via the usual factory approach, and a [sample set of data files](./data/org-policies/) is included with this stage. They are not applied during the initial run when the `bootstrap_user` variable is set, to work around incompatibilies with user credentials. +To cover this use case, organization policies have been moved from the resource management to the bootstrap stage in FAST versions after 26.0.0. They are managed via the usual factory approach, and a [sample set of data files](./data/org-policies/) is included with this stage. They are not applied during the initial run when the `bootstrap_user` variable is set, to work around incompatibilities with user credentials. The only current exception to the factory approach is the `iam.allowedPolicyMemberDomains` constraint, which is managed in code so as to be able to auto-allow the organization's domain. More domains can be added via the `org_policies_config` variable, which also serves as an umbrella for future policies that will need to be managed in code. diff --git a/fast/stages/3-project-factory/dev/README.md b/fast/stages/3-project-factory/dev/README.md index 03bfa86c5e..07b748c50f 100644 --- a/fast/stages/3-project-factory/dev/README.md +++ b/fast/stages/3-project-factory/dev/README.md @@ -79,8 +79,8 @@ terraform apply | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| | [billing_account](variables.tf#L19) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | -| [prefix](variables.tf#L51) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | -| [factory_data](variables.tf#L32) | Project data from either YAML files or externally parsed data. | object({…}) | | {…} | | +| [prefix](variables.tf#L39) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | +| [factory_data_path](variables.tf#L32) | Path to folder containing YAML project data files. | string | | "data/projects" | | ## Outputs diff --git a/fast/stages/3-project-factory/dev/main.tf b/fast/stages/3-project-factory/dev/main.tf index 4f23b49281..48867a9984 100644 --- a/fast/stages/3-project-factory/dev/main.tf +++ b/fast/stages/3-project-factory/dev/main.tf @@ -33,7 +33,7 @@ module "projects" { data_overrides = { prefix = "${var.prefix}-dev" } - factory_data = var.factory_data + factory_data_path = var.factory_data_path } diff --git a/fast/stages/3-project-factory/dev/variables.tf b/fast/stages/3-project-factory/dev/variables.tf index c7165e3ced..5a8ef4f870 100644 --- a/fast/stages/3-project-factory/dev/variables.tf +++ b/fast/stages/3-project-factory/dev/variables.tf @@ -29,23 +29,11 @@ variable "billing_account" { } } -variable "factory_data" { - description = "Project data from either YAML files or externally parsed data." - type = object({ - data = optional(map(any)) - data_path = optional(string) - }) - nullable = false - default = { - data_path = "data/projects" - } - validation { - condition = ( - (var.factory_data.data != null ? 1 : 0) + - (var.factory_data.data_path != null ? 1 : 0) - ) == 1 - error_message = "One of data or data_path needs to be set." - } +variable "factory_data_path" { + description = "Path to folder containing YAML project data files." + type = string + nullable = false + default = "data/projects" } variable "prefix" { diff --git a/modules/README.md b/modules/README.md index 03c5cf65bc..cefc5e5592 100644 --- a/modules/README.md +++ b/modules/README.md @@ -21,7 +21,7 @@ These modules are used in the examples included in this repository. If you are u ```terraform module "project" { - source = "github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/project?ref=v13.0.0" + source = "github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/project?ref=v13.0.0&depth=1" name = "my-project" billing_account = "123456-123456-123456" parent = "organizations/123456" diff --git a/modules/__experimental/alloydb-instance/versions.tf b/modules/__experimental/alloydb-instance/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/__experimental/alloydb-instance/versions.tf +++ b/modules/__experimental/alloydb-instance/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/__experimental/net-neg/versions.tf b/modules/__experimental/net-neg/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/__experimental/net-neg/versions.tf +++ b/modules/__experimental/net-neg/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/__experimental/project-iam-magic/versions.tf b/modules/__experimental/project-iam-magic/versions.tf new file mode 100644 index 0000000000..2ce9578209 --- /dev/null +++ b/modules/__experimental/project-iam-magic/versions.tf @@ -0,0 +1,29 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +terraform { + required_version = ">= 1.4.4" + required_providers { + google = { + source = "hashicorp/google" + version = ">= 4.71.0" # tftest + } + google-beta = { + source = "hashicorp/google-beta" + version = ">= 4.71.0" # tftest + } + } +} + + diff --git a/modules/api-gateway/versions.tf b/modules/api-gateway/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/api-gateway/versions.tf +++ b/modules/api-gateway/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/apigee/README.md b/modules/apigee/README.md index 5a05ecb353..24c6f4166a 100644 --- a/modules/apigee/README.md +++ b/modules/apigee/README.md @@ -354,19 +354,18 @@ module "apigee" { } # tftest modules=1 resources=10 ``` - ## Variables | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [project_id](variables.tf#L125) | Project ID. | string | ✓ | | +| [project_id](variables.tf#L126) | Project ID. | string | ✓ | | | [addons_config](variables.tf#L17) | Addons configuration. | object({…}) | | null | | [endpoint_attachments](variables.tf#L29) | Endpoint attachments. | map(object({…})) | | {} | | [envgroups](variables.tf#L39) | Environment groups (NAME => [HOSTNAMES]). | map(list(string)) | | {} | -| [environments](variables.tf#L46) | Environments. | map(object({…})) | | {} | -| [instances](variables.tf#L72) | Instances ([REGION] => [INSTANCE]). | map(object({…})) | | {} | -| [organization](variables.tf#L97) | Apigee organization. If set to null the organization must already exist. | object({…}) | | null | +| [environments](variables.tf#L46) | Environments. | map(object({…})) | | {} | +| [instances](variables.tf#L73) | Instances ([REGION] => [INSTANCE]). | map(object({…})) | | {} | +| [organization](variables.tf#L98) | Apigee organization. If set to null the organization must already exist. | object({…}) | | null | ## Outputs diff --git a/modules/apigee/main.tf b/modules/apigee/main.tf index be571a8ae2..4db47ee0fd 100644 --- a/modules/apigee/main.tf +++ b/modules/apigee/main.tf @@ -40,11 +40,11 @@ resource "google_apigee_envgroup" "envgroups" { resource "google_apigee_environment" "environments" { for_each = var.environments - name = each.key - display_name = each.value.display_name - description = each.value.description - deployment_type = each.value.deployment_type api_proxy_type = each.value.api_proxy_type + deployment_type = each.value.deployment_type + description = each.value.description + display_name = each.value.display_name + name = each.key dynamic "node_config" { for_each = try(each.value.node_config, null) != null ? [""] : [] content { @@ -53,6 +53,7 @@ resource "google_apigee_environment" "environments" { } } org_id = local.org_id + type = each.value.type lifecycle { ignore_changes = [ node_config["current_aggregate_node_count"] diff --git a/modules/apigee/variables.tf b/modules/apigee/variables.tf index 7ec2cc2d64..027ad05e47 100644 --- a/modules/apigee/variables.tf +++ b/modules/apigee/variables.tf @@ -50,6 +50,7 @@ variable "environments" { description = optional(string, "Terraform-managed") deployment_type = optional(string) api_proxy_type = optional(string) + type = optional(string) node_config = optional(object({ min_node_count = optional(number) max_node_count = optional(number) diff --git a/modules/apigee/versions.tf b/modules/apigee/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/apigee/versions.tf +++ b/modules/apigee/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/artifact-registry/versions.tf b/modules/artifact-registry/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/artifact-registry/versions.tf +++ b/modules/artifact-registry/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/bigquery-dataset/versions.tf b/modules/bigquery-dataset/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/bigquery-dataset/versions.tf +++ b/modules/bigquery-dataset/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/bigtable-instance/versions.tf b/modules/bigtable-instance/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/bigtable-instance/versions.tf +++ b/modules/bigtable-instance/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/billing-account/versions.tf b/modules/billing-account/versions.tf index 0cc9b9727b..cee7f9c764 100644 --- a/modules/billing-account/versions.tf +++ b/modules/billing-account/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/binauthz/versions.tf b/modules/binauthz/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/binauthz/versions.tf +++ b/modules/binauthz/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/__need_fixing/onprem/versions.tf b/modules/cloud-config-container/__need_fixing/onprem/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/__need_fixing/onprem/versions.tf +++ b/modules/cloud-config-container/__need_fixing/onprem/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/coredns/versions.tf b/modules/cloud-config-container/coredns/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/coredns/versions.tf +++ b/modules/cloud-config-container/coredns/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/cos-generic-metadata/versions.tf b/modules/cloud-config-container/cos-generic-metadata/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/cos-generic-metadata/versions.tf +++ b/modules/cloud-config-container/cos-generic-metadata/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/README.md b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/README.md new file mode 100644 index 0000000000..76d1b19d4c --- /dev/null +++ b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/README.md @@ -0,0 +1,57 @@ +# Containerized Envoy as SNI dynamic forward proxy on Container Optimized OS + +This module manages a `cloud-config` configuration that starts a containerized [Envoy SNI Dynamic forward proxy]https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/sni_dynamic_forward_proxy_filter) service on Container Optimized OS running on port 443. + +This module depends on the cos-generic-metadata module being in the parent folder. If you change its location be sure to adjust the source attribute in main.tf. + +Logging and monitoring are enabled via the [Google Cloud Logging agent](https://cloud.google.com/container-optimized-os/docs/how-to/logging) configured for the instance via the `google-logging-enabled` metadata property, and the [Node Problem Detector](https://cloud.google.com/container-optimized-os/docs/how-to/monitoring) service started by default on boot. + +## Examples + +### Default configuration + +This example will create a `cloud-config` that uses the module's defaults, creating a simple hello web server showing host name and request id. + +```hcl +module "cos-envoy-sni-dyn-fwd-proxy" { + source = "./fabric/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy" + envoy_image = "envoyproxy/envoy:v1.28-latest" +} + +module "vm-envoy-sni-dyn-fwd-proxy" { + source = "./fabric/modules/compute-vm" + project_id = "my-project" + zone = "europe-west8-b" + name = "cos-envoy-sni-dyn-fw-proxy" + network_interfaces = [{ + network = "default" + subnetwork = "gce" + }] + metadata = { + user-data = module.cos-envoy-sni-dyn-fwd-proxy.cloud_config + google-logging-enabled = true + } + boot_disk = { + initialize_params = { + image = "projects/cos-cloud/global/images/family/cos-stable" + type = "pd-ssd" + size = 10 + } + } + tags = ["https-server", "ssh"] +} +# tftest modules=1 resources=1 +``` + +## Variables + +| name | description | type | required | default | +|---|---|:---:|:---:|:---:| +| [envoy_image](variables.tf#L17) | Image. | string | ✓ | | + +## Outputs + +| name | description | sensitive | +|---|---|:---:| +| [cloud_config](outputs.tf#L17) | Rendered cloud-config file to be passed as user-data instance metadata. | | + diff --git a/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/files/envoy.yaml b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/files/envoy.yaml new file mode 100644 index 0000000000..55a5e08fb1 --- /dev/null +++ b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/files/envoy.yaml @@ -0,0 +1,56 @@ +# Copyright 2023 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +admin: + address: + socket_address: + protocol: TCP + address: 127.0.0.1 + port_value: 9991 +static_resources: + listeners: + - name: listener + address: + socket_address: + protocol: TCP + address: 0.0.0.0 + port_value: 8443 + listener_filters: + - name: envoy.filters.listener.tls_inspector + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector + filter_chains: + - filters: + - name: envoy.filters.network.sni_dynamic_forward_proxy + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.network.sni_dynamic_forward_proxy.v3.FilterConfig + port_value: 443 + dns_cache_config: + name: dynamic_forward_proxy_cache_config + dns_lookup_family: V4_ONLY + - name: envoy.tcp_proxy + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy + stat_prefix: tcp + cluster: dynamic_forward_proxy_cluster + clusters: + - name: dynamic_forward_proxy_cluster + lb_policy: CLUSTER_PROVIDED + cluster_type: + name: envoy.clusters.dynamic_forward_proxy + typed_config: + "@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig + dns_cache_config: + name: dynamic_forward_proxy_cache_config + dns_lookup_family: V4_ONLY% diff --git a/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/main.tf b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/main.tf new file mode 100644 index 0000000000..f0b7cdfb2c --- /dev/null +++ b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/main.tf @@ -0,0 +1,44 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +module "cos-envoy" { + source = "../cos-generic-metadata" + container_image = var.envoy_image + container_name = "envoy" + container_args = "-c /etc/envoy/envoy.yaml --log-level info --allow-unknown-static-fields" + container_volumes = [ + { host = "/etc/envoy/", container = "/etc/envoy/" } + ] + docker_args = "--network host --pid host" + files = { + "/etc/envoy/envoy.yaml" = { + content = file("${path.module}/files/envoy.yaml") + owner = "root" + permissions = "0644" + } + } + run_commands = [ + "iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8443", + "iptables -A INPUT -p tcp --dport 8443 -j ACCEPT", + "iptables -t mangle -I PREROUTING -p tcp --dport 8443 -j DROP", + "systemctl daemon-reload", + "systemctl start envoy", + ] + users = [{ + username = "envoy", + uid = 1337 + }] +} diff --git a/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/outputs.tf b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/outputs.tf new file mode 100644 index 0000000000..417c73e7d7 --- /dev/null +++ b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/outputs.tf @@ -0,0 +1,20 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +output "cloud_config" { + description = "Rendered cloud-config file to be passed as user-data instance metadata." + value = module.cos-envoy.cloud_config +} diff --git a/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/variables.tf b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/variables.tf new file mode 100644 index 0000000000..3868a17424 --- /dev/null +++ b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/variables.tf @@ -0,0 +1,20 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "envoy_image" { + description = "Image." + type = string +} diff --git a/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/versions.tf b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/versions.tf new file mode 100644 index 0000000000..ceb6930fb3 --- /dev/null +++ b/modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/versions.tf @@ -0,0 +1,28 @@ +# Copyright 2023 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +terraform { + required_version = ">= 1.4.4" + required_providers { + google = { + source = "hashicorp/google" + version = ">= 5.4.0, < 6.0.0" # tftest + } + google-beta = { + source = "hashicorp/google-beta" + version = ">= 5.4.0, < 6.0.0" # tftest + } + } +} + diff --git a/modules/cloud-config-container/envoy-traffic-director/versions.tf b/modules/cloud-config-container/envoy-traffic-director/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/envoy-traffic-director/versions.tf +++ b/modules/cloud-config-container/envoy-traffic-director/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/mysql/versions.tf b/modules/cloud-config-container/mysql/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/mysql/versions.tf +++ b/modules/cloud-config-container/mysql/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/nginx-tls/versions.tf b/modules/cloud-config-container/nginx-tls/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/nginx-tls/versions.tf +++ b/modules/cloud-config-container/nginx-tls/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/nginx/versions.tf b/modules/cloud-config-container/nginx/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/nginx/versions.tf +++ b/modules/cloud-config-container/nginx/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/simple-nva/versions.tf b/modules/cloud-config-container/simple-nva/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/simple-nva/versions.tf +++ b/modules/cloud-config-container/simple-nva/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-config-container/squid/versions.tf b/modules/cloud-config-container/squid/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-config-container/squid/versions.tf +++ b/modules/cloud-config-container/squid/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-function-v1/versions.tf b/modules/cloud-function-v1/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-function-v1/versions.tf +++ b/modules/cloud-function-v1/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-function-v2/versions.tf b/modules/cloud-function-v2/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-function-v2/versions.tf +++ b/modules/cloud-function-v2/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-identity-group/versions.tf b/modules/cloud-identity-group/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-identity-group/versions.tf +++ b/modules/cloud-identity-group/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloud-run/versions.tf b/modules/cloud-run/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloud-run/versions.tf +++ b/modules/cloud-run/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/cloudsql-instance/README.md b/modules/cloudsql-instance/README.md index c0f72cae08..1a9347691d 100644 --- a/modules/cloudsql-instance/README.md +++ b/modules/cloudsql-instance/README.md @@ -86,9 +86,13 @@ module "db" { users = { # generatea password for user1 - user1 = null + user1 = { + password = null + } # assign a password to user2 - user2 = "mypassword" + user2 = { + password = "mypassword" + } } } # tftest modules=1 resources=6 inventory=custom.yaml @@ -212,7 +216,7 @@ module "db" { | [replicas](variables.tf#L179) | Map of NAME=> {REGION, KMS_KEY} for additional read replicas. Set to null to disable replica creation. | map(object({…})) | | {} | | [require_ssl](variables.tf#L188) | Enable SSL connections only. | bool | | null | | [root_password](variables.tf#L194) | Root password of the Cloud SQL instance. Required for MS SQL Server. | string | | null | -| [users](variables.tf#L205) | Map of users to create in the primary instance (and replicated to other replicas) in the format USER=>PASSWORD. For MySQL, anything afterr the first `@` (if persent) will be used as the user's host. Set PASSWORD to null if you want to get an autogenerated password. | map(string) | | null | +| [users](variables.tf#L205) | Map of users to create in the primary instance (and replicated to other replicas). For MySQL, anything afterr the first `@` (if persent) will be used as the user's host. Set PASSWORD to null if you want to get an autogenerated password. The user types available are: 'BUILT_IN', 'CLOUD_IAM_USER' or 'CLOUD_IAM_SERVICE_ACCOUNT'. | map(object({…})) | | null | ## Outputs diff --git a/modules/cloudsql-instance/main.tf b/modules/cloudsql-instance/main.tf index fd3d9abdbd..cc6e62af8f 100644 --- a/modules/cloudsql-instance/main.tf +++ b/modules/cloudsql-instance/main.tf @@ -1,4 +1,4 @@ -/** +/** TO MOD * Copyright 2022 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); @@ -17,6 +17,7 @@ locals { prefix = var.prefix == null ? "" : "${var.prefix}-" is_mysql = can(regex("^MYSQL", var.database_version)) + is_postgres = can(regex("^POSTGRES", var.database_version)) has_replicas = try(length(var.replicas) > 0, false) is_regional = var.availability_type == "REGIONAL" ? true : false @@ -25,20 +26,20 @@ locals { enable_backup = var.backup_configuration.enabled || (local.is_mysql && local.has_replicas) || (local.is_mysql && local.is_regional) users = { - for user, password in coalesce(var.users, {}) : - (user) => ( - local.is_mysql - ? { - name = split("@", user)[0] - host = try(split("@", user)[1], null) - password = try(random_password.passwords[user].result, password) - } - : { - name = user - host = null - password = try(random_password.passwords[user].result, password) - } - ) + for k, v in coalesce(var.users, {}) : + k => + local.is_mysql ? + { + name = try(v.type, "BUILT_IN") == "BUILT_IN" ? split("@", k)[0] : k + host = try(v.type, "BUILT_IN") == "BUILT_IN" ? try(split("@", k)[1], null) : null + password = try(v.type, "BUILT_IN") == "BUILT_IN" ? try(random_password.passwords[k].result, v.password) : null + type = try(v.type, "BUILT_IN") + } : { + name = local.is_postgres ? try(trimsuffix(k, ".gserviceaccount.com"), k) : k + host = null + password = try(v.type, "BUILT_IN") == "BUILT_IN" ? try(random_password.passwords[k].result, v.password) : null + type = try(v.type, "BUILT_IN") + } } } @@ -178,14 +179,15 @@ resource "google_sql_database" "databases" { resource "random_password" "passwords" { for_each = toset([ - for user, password in coalesce(var.users, {}) : - user - if password == null + for k, v in coalesce(var.users, {}) : + k + if v.password == null ]) length = 16 special = true } + resource "google_sql_user" "users" { for_each = local.users project = var.project_id @@ -193,6 +195,7 @@ resource "google_sql_user" "users" { name = each.value.name host = each.value.host password = each.value.password + type = each.value.type } resource "google_sql_ssl_cert" "postgres_client_certificates" { diff --git a/modules/cloudsql-instance/variables.tf b/modules/cloudsql-instance/variables.tf index d13889b0c4..7fda3e9b02 100644 --- a/modules/cloudsql-instance/variables.tf +++ b/modules/cloudsql-instance/variables.tf @@ -203,8 +203,11 @@ variable "tier" { } variable "users" { - description = "Map of users to create in the primary instance (and replicated to other replicas) in the format USER=>PASSWORD. For MySQL, anything afterr the first `@` (if persent) will be used as the user's host. Set PASSWORD to null if you want to get an autogenerated password." - type = map(string) - default = null + description = "Map of users to create in the primary instance (and replicated to other replicas). For MySQL, anything afterr the first `@` (if persent) will be used as the user's host. Set PASSWORD to null if you want to get an autogenerated password. The user types available are: 'BUILT_IN', 'CLOUD_IAM_USER' or 'CLOUD_IAM_SERVICE_ACCOUNT'." + type = map(object({ + password = optional(string) + type = optional(string) + })) + default = null } diff --git a/modules/cloudsql-instance/versions.tf b/modules/cloudsql-instance/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/cloudsql-instance/versions.tf +++ b/modules/cloudsql-instance/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/compute-mig/versions.tf b/modules/compute-mig/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/compute-mig/versions.tf +++ b/modules/compute-mig/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/compute-vm/README.md b/modules/compute-vm/README.md index 052d71b76b..450ccad730 100644 --- a/modules/compute-vm/README.md +++ b/modules/compute-vm/README.md @@ -516,7 +516,7 @@ module "kms-vm-example" { } encryption = { encrypt_boot = true - kms_key_self_link = var.kms_key.self_link + kms_key_self_link = var.kms_key.id } } # tftest modules=1 resources=3 inventory=cmek.yaml diff --git a/modules/compute-vm/versions.tf b/modules/compute-vm/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/compute-vm/versions.tf +++ b/modules/compute-vm/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/container-registry/versions.tf b/modules/container-registry/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/container-registry/versions.tf +++ b/modules/container-registry/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/data-catalog-policy-tag/versions.tf b/modules/data-catalog-policy-tag/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/data-catalog-policy-tag/versions.tf +++ b/modules/data-catalog-policy-tag/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/datafusion/versions.tf b/modules/datafusion/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/datafusion/versions.tf +++ b/modules/datafusion/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/dataplex-datascan/versions.tf b/modules/dataplex-datascan/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/dataplex-datascan/versions.tf +++ b/modules/dataplex-datascan/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/dataplex/versions.tf b/modules/dataplex/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/dataplex/versions.tf +++ b/modules/dataplex/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/dataproc/README.md b/modules/dataproc/README.md index 5cd220cbaa..f00101345c 100644 --- a/modules/dataproc/README.md +++ b/modules/dataproc/README.md @@ -149,7 +149,7 @@ module "processing-dp-cluster" { | [name](variables.tf#L235) | Cluster name. | string | ✓ | | | [project_id](variables.tf#L250) | Project ID. | string | ✓ | | | [region](variables.tf#L255) | Dataproc region. | string | ✓ | | -| [dataproc_config](variables.tf#L17) | Dataproc cluster config. | object({…}) | | {} | +| [dataproc_config](variables.tf#L17) | Dataproc cluster config. | object({…}) | | {} | | [group_iam](variables.tf#L185) | Authoritative IAM binding for organization groups, in {GROUP_EMAIL => [ROLES]} format. Group emails need to be static. Can be used in combination with the `iam` variable. | map(list(string)) | | {} | | [iam](variables.tf#L192) | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | | [iam_bindings](variables.tf#L199) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | map(object({…})) | | {} | diff --git a/modules/dataproc/main.tf b/modules/dataproc/main.tf index 55bef5c70e..f229d7c7ae 100644 --- a/modules/dataproc/main.tf +++ b/modules/dataproc/main.tf @@ -108,8 +108,8 @@ resource "google_dataproc_cluster" "cluster" { dynamic "accelerators" { for_each = var.dataproc_config.cluster_config.worker_config.accelerators == null ? [] : [""] content { - accelerator_type = var.dataproc_config.cluster_config.accelerators.accelerator_type - accelerator_count = var.dataproc_config.cluster_config.accelerators.accelerator_count + accelerator_type = var.dataproc_config.cluster_config.worker_config.accelerators.accelerator_type + accelerator_count = var.dataproc_config.cluster_config.worker_config.accelerators.accelerator_count } } } @@ -185,10 +185,10 @@ resource "google_dataproc_cluster" "cluster" { for_each = var.dataproc_config.cluster_config.dataproc_metric_config == null ? [] : [""] content { dynamic "metrics" { - for_each = var.dataproc_config.cluster_config.dataproc_metric_config.metrics == null ? [] : [""] + for_each = coalesce(var.dataproc_config.cluster_config.dataproc_metric_config.metrics, []) content { - metric_source = var.dataproc_config.cluster_config.dataproc_metric_config.metrics.metric_source - metric_overrides = var.dataproc_config.cluster_config.dataproc_metric_config.metrics.metric_overrides + metric_source = metrics.value.metric_source + metric_overrides = metrics.value.metric_overrides } } } diff --git a/modules/dataproc/variables.tf b/modules/dataproc/variables.tf index 8b77c5b96b..f4170a94f0 100644 --- a/modules/dataproc/variables.tf +++ b/modules/dataproc/variables.tf @@ -128,7 +128,7 @@ variable "dataproc_config" { dataproc_metric_config = optional(object({ metrics = list(object({ metric_source = string - metric_overrides = optional(string) + metric_overrides = optional(list(string)) })) })) metastore_config = optional(object({ diff --git a/modules/dataproc/versions.tf b/modules/dataproc/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/dataproc/versions.tf +++ b/modules/dataproc/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/dns-response-policy/versions.tf b/modules/dns-response-policy/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/dns-response-policy/versions.tf +++ b/modules/dns-response-policy/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/dns/versions.tf b/modules/dns/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/dns/versions.tf +++ b/modules/dns/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/endpoints/versions.tf b/modules/endpoints/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/endpoints/versions.tf +++ b/modules/endpoints/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/folder/README.md b/modules/folder/README.md index 2ba7e9107a..8c75cadd5e 100644 --- a/modules/folder/README.md +++ b/modules/folder/README.md @@ -295,12 +295,12 @@ module "folder" { | [id](variables.tf#L83) | Folder ID in case you use folder_create=false. | string | | null | | [logging_data_access](variables.tf#L89) | Control activation of data access logs. Format is service => { log type => [exempted members]}. The special 'allServices' key denotes configuration for all services. | map(map(list(string))) | | {} | | [logging_exclusions](variables.tf#L104) | Logging exclusions for this folder in the form {NAME -> FILTER}. | map(string) | | {} | -| [logging_sinks](variables.tf#L111) | Logging sinks to create for the folder. | map(object({…})) | | {} | -| [name](variables.tf#L141) | Folder name. | string | | null | -| [org_policies](variables.tf#L147) | Organization policies applied to this folder keyed by policy name. | map(object({…})) | | {} | -| [org_policies_data_path](variables.tf#L174) | Path containing org policies in YAML format. | string | | null | -| [parent](variables.tf#L180) | Parent in folders/folder_id or organizations/org_id format. | string | | null | -| [tag_bindings](variables.tf#L190) | Tag bindings for this folder, in key => tag value id format. | map(string) | | null | +| [logging_sinks](variables.tf#L111) | Logging sinks to create for the folder. | map(object({…})) | | {} | +| [name](variables.tf#L142) | Folder name. | string | | null | +| [org_policies](variables.tf#L148) | Organization policies applied to this folder keyed by policy name. | map(object({…})) | | {} | +| [org_policies_data_path](variables.tf#L175) | Path containing org policies in YAML format. | string | | null | +| [parent](variables.tf#L181) | Parent in folders/folder_id or organizations/org_id format. | string | | null | +| [tag_bindings](variables.tf#L191) | Tag bindings for this folder, in key => tag value id format. | map(string) | | null | ## Outputs diff --git a/modules/folder/logging.tf b/modules/folder/logging.tf index 8000a02145..033113ff2e 100644 --- a/modules/folder/logging.tf +++ b/modules/folder/logging.tf @@ -22,7 +22,7 @@ locals { type => { for name, sink in var.logging_sinks : name => sink - if sink.type == type + if sink.iam == true && sink.type == type } } } diff --git a/modules/folder/variables.tf b/modules/folder/variables.tf index 1c55168187..91e8e11e81 100644 --- a/modules/folder/variables.tf +++ b/modules/folder/variables.tf @@ -117,6 +117,7 @@ variable "logging_sinks" { disabled = optional(bool, false) exclusions = optional(map(string), {}) filter = string + iam = optional(bool, true) include_children = optional(bool, true) type = string })) diff --git a/modules/folder/versions.tf b/modules/folder/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/folder/versions.tf +++ b/modules/folder/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/gcs/README.md b/modules/gcs/README.md index 8a26958d62..993e0435ff 100644 --- a/modules/gcs/README.md +++ b/modules/gcs/README.md @@ -196,12 +196,13 @@ module "bucket" { | [notification_config](variables.tf#L169) | GCS Notification configuration. | object({…}) | | null | | [objects_to_upload](variables.tf#L183) | Objects to be uploaded to bucket. | map(object({…})) | | {} | | [prefix](variables.tf#L209) | Optional prefix used to generate the bucket name. | string | | null | -| [requester_pays](variables.tf#L224) | Enables Requester Pays on a storage bucket. | bool | | null | -| [retention_policy](variables.tf#L230) | Bucket retention policy. | object({…}) | | null | -| [storage_class](variables.tf#L239) | Bucket storage class. | string | | "MULTI_REGIONAL" | -| [uniform_bucket_level_access](variables.tf#L249) | Allow using object ACLs (false) or not (true, this is the recommended behavior) , defaults to true (which is the recommended practice, but not the behavior of storage API). | bool | | true | -| [versioning](variables.tf#L255) | Enable versioning, defaults to false. | bool | | false | -| [website](variables.tf#L261) | Bucket website. | object({…}) | | null | +| [public_access_prevention](variables.tf#L224) | Prevents public access to a bucket. Acceptable values are inherited or enforced. If inherited, the bucket uses public access prevention, only if the bucket is subject to the public access prevention organization policy constraint. | string | | null | +| [requester_pays](variables.tf#L230) | Enables Requester Pays on a storage bucket. | bool | | null | +| [retention_policy](variables.tf#L236) | Bucket retention policy. | object({…}) | | null | +| [storage_class](variables.tf#L245) | Bucket storage class. | string | | "MULTI_REGIONAL" | +| [uniform_bucket_level_access](variables.tf#L255) | Allow using object ACLs (false) or not (true, this is the recommended behavior) , defaults to true (which is the recommended practice, but not the behavior of storage API). | bool | | true | +| [versioning](variables.tf#L261) | Enable versioning, defaults to false. | bool | | false | +| [website](variables.tf#L267) | Bucket website. | object({…}) | | null | ## Outputs diff --git a/modules/gcs/main.tf b/modules/gcs/main.tf index 1341e707cd..c19c7912e0 100644 --- a/modules/gcs/main.tf +++ b/modules/gcs/main.tf @@ -29,6 +29,7 @@ resource "google_storage_bucket" "bucket" { labels = var.labels default_event_based_hold = var.default_event_based_hold requester_pays = var.requester_pays + public_access_prevention = var.public_access_prevention versioning { enabled = var.versioning } diff --git a/modules/gcs/variables.tf b/modules/gcs/variables.tf index 350c74baf5..de8a6abd83 100644 --- a/modules/gcs/variables.tf +++ b/modules/gcs/variables.tf @@ -221,6 +221,12 @@ variable "project_id" { type = string } +variable "public_access_prevention" { + description = "Prevents public access to a bucket. Acceptable values are inherited or enforced. If inherited, the bucket uses public access prevention, only if the bucket is subject to the public access prevention organization policy constraint." + type = string + default = null +} + variable "requester_pays" { description = "Enables Requester Pays on a storage bucket." type = bool diff --git a/modules/gcs/versions.tf b/modules/gcs/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/gcs/versions.tf +++ b/modules/gcs/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/gcve-private-cloud/README.md b/modules/gcve-private-cloud/README.md index 901b307971..0e3a9ef7b3 100644 --- a/modules/gcve-private-cloud/README.md +++ b/modules/gcve-private-cloud/README.md @@ -2,7 +2,7 @@ This module implements the creation and management of a Google Cloud VMWare Engine Private Cloud with its management cluster. If configured, it also creates the VMWare engine network or it can work with an existing one. The creation of the private connection with the user VPC requires the execution of the [Google SDK command](https://cloud.google.com/sdk/gcloud/reference/vmware/private-connections/create#--routing-mode) the module provides as an output. -To understand the limits and to propertly configure the vSphere/vSAN subnets CIDR range please refer to the [GCVE public documetation](https://cloud.google.com/vmware-engine/docs/quickstart-networking-requirements). +To understand the limits and to properly configure the vSphere/vSAN subnets CIDR range please refer to the [GCVE public documentation](https://cloud.google.com/vmware-engine/docs/quickstart-networking-requirements). Be aware that the deployment of this module might require up to 2 hours depending on the selected private cloud target zone. diff --git a/modules/gcve-private-cloud/versions.tf b/modules/gcve-private-cloud/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/gcve-private-cloud/versions.tf +++ b/modules/gcve-private-cloud/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/gke-cluster-autopilot/versions.tf b/modules/gke-cluster-autopilot/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/gke-cluster-autopilot/versions.tf +++ b/modules/gke-cluster-autopilot/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/gke-cluster-standard/README.md b/modules/gke-cluster-standard/README.md index 53b57e8f80..e895b4819c 100644 --- a/modules/gke-cluster-standard/README.md +++ b/modules/gke-cluster-standard/README.md @@ -310,28 +310,28 @@ module "cluster-1" { | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [location](variables.tf#L154) | Cluster zone or region. | string | ✓ | | -| [name](variables.tf#L265) | Cluster name. | string | ✓ | | -| [project_id](variables.tf#L291) | Cluster project id. | string | ✓ | | -| [vpc_config](variables.tf#L314) | VPC-level configuration. | object({…}) | ✓ | | +| [location](variables.tf#L178) | Cluster zone or region. | string | ✓ | | +| [name](variables.tf#L289) | Cluster name. | string | ✓ | | +| [project_id](variables.tf#L315) | Cluster project id. | string | ✓ | | +| [vpc_config](variables.tf#L338) | VPC-level configuration. | object({…}) | ✓ | | | [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | object({…}) | | {} | -| [cluster_autoscaling](variables.tf#L38) | Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler. | object({…}) | | null | -| [deletion_protection](variables.tf#L59) | Whether or not to allow Terraform to destroy the cluster. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the cluster will fail. | bool | | true | -| [description](variables.tf#L66) | Cluster description. | string | | null | -| [enable_addons](variables.tf#L72) | Addons enabled in the cluster (true means enabled). | object({…}) | | {…} | -| [enable_features](variables.tf#L96) | Enable cluster-level features. Certain features allow configuration. | object({…}) | | {…} | -| [issue_client_certificate](variables.tf#L142) | Enable issuing client certificate. | bool | | false | -| [labels](variables.tf#L148) | Cluster resource labels. | map(string) | | null | -| [logging_config](variables.tf#L159) | Logging configuration. | object({…}) | | {} | -| [maintenance_config](variables.tf#L180) | Maintenance window configuration. | object({…}) | | {…} | -| [max_pods_per_node](variables.tf#L203) | Maximum number of pods per node in this cluster. | number | | 110 | -| [min_master_version](variables.tf#L209) | Minimum version of the master, defaults to the version of the most recent official release. | string | | null | -| [monitoring_config](variables.tf#L215) | Monitoring configuration. Google Cloud Managed Service for Prometheus is enabled by default. | object({…}) | | {} | -| [node_locations](variables.tf#L270) | Zones in which the cluster's nodes are located. | list(string) | | [] | -| [private_cluster_config](variables.tf#L277) | Private cluster configuration. | object({…}) | | null | -| [release_channel](variables.tf#L296) | Release channel for GKE upgrades. | string | | null | -| [service_account](variables.tf#L302) | Service account used for the default node pool, only useful if the default GCE service account has been disabled. | string | | null | -| [tags](variables.tf#L308) | Network tags applied to nodes. | list(string) | | null | +| [cluster_autoscaling](variables.tf#L38) | Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler. | object({…}) | | null | +| [deletion_protection](variables.tf#L83) | Whether or not to allow Terraform to destroy the cluster. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the cluster will fail. | bool | | true | +| [description](variables.tf#L90) | Cluster description. | string | | null | +| [enable_addons](variables.tf#L96) | Addons enabled in the cluster (true means enabled). | object({…}) | | {…} | +| [enable_features](variables.tf#L120) | Enable cluster-level features. Certain features allow configuration. | object({…}) | | {…} | +| [issue_client_certificate](variables.tf#L166) | Enable issuing client certificate. | bool | | false | +| [labels](variables.tf#L172) | Cluster resource labels. | map(string) | | null | +| [logging_config](variables.tf#L183) | Logging configuration. | object({…}) | | {} | +| [maintenance_config](variables.tf#L204) | Maintenance window configuration. | object({…}) | | {…} | +| [max_pods_per_node](variables.tf#L227) | Maximum number of pods per node in this cluster. | number | | 110 | +| [min_master_version](variables.tf#L233) | Minimum version of the master, defaults to the version of the most recent official release. | string | | null | +| [monitoring_config](variables.tf#L239) | Monitoring configuration. Google Cloud Managed Service for Prometheus is enabled by default. | object({…}) | | {} | +| [node_locations](variables.tf#L294) | Zones in which the cluster's nodes are located. | list(string) | | [] | +| [private_cluster_config](variables.tf#L301) | Private cluster configuration. | object({…}) | | null | +| [release_channel](variables.tf#L320) | Release channel for GKE upgrades. | string | | null | +| [service_account](variables.tf#L326) | Service account used for the default node pool, only useful if the default GCE service account has been disabled. | string | | null | +| [tags](variables.tf#L332) | Network tags applied to nodes. | list(string) | | null | ## Outputs diff --git a/modules/gke-cluster-standard/main.tf b/modules/gke-cluster-standard/main.tf index f5d8fe7511..42f115e26b 100644 --- a/modules/gke-cluster-standard/main.tf +++ b/modules/gke-cluster-standard/main.tf @@ -123,13 +123,31 @@ resource "google_container_cluster" "cluster" { content { enabled = true + autoscaling_profile = var.cluster_autoscaling.autoscaling_profile + dynamic "auto_provisioning_defaults" { for_each = var.cluster_autoscaling.auto_provisioning_defaults != null ? [""] : [] content { boot_disk_kms_key = var.cluster_autoscaling.auto_provisioning_defaults.boot_disk_kms_key + disk_size = var.cluster_autoscaling.auto_provisioning_defaults.disk_size + disk_type = var.cluster_autoscaling.auto_provisioning_defaults.disk_type image_type = var.cluster_autoscaling.auto_provisioning_defaults.image_type oauth_scopes = var.cluster_autoscaling.auto_provisioning_defaults.oauth_scopes service_account = var.cluster_autoscaling.auto_provisioning_defaults.service_account + dynamic "management" { + for_each = var.cluster_autoscaling.auto_provisioning_defaults.management != null ? [""] : [] + content { + auto_repair = var.cluster_autoscaling.auto_provisioning_defaults.management.auto_repair + auto_upgrade = var.cluster_autoscaling.auto_provisioning_defaults.management.auto_upgrade + } + } + dynamic "shielded_instance_config" { + for_each = var.cluster_autoscaling.auto_provisioning_defaults.shielded_instance_config != null ? [""] : [] + content { + enable_integrity_monitoring = var.cluster_autoscaling.auto_provisioning_defaults.shielded_instance_config.integrity_monitoring + enable_secure_boot = var.cluster_autoscaling.auto_provisioning_defaults.shielded_instance_config.secure_boot + } + } } } dynamic "resource_limits" { @@ -148,7 +166,19 @@ resource "google_container_cluster" "cluster" { maximum = var.cluster_autoscaling.mem_limits.max } } - // TODO: support GPUs too + dynamic "resource_limits" { + for_each = ( + try(var.cluster_autoscaling.gpu_resources, null) == null + ? [] + : var.cluster_autoscaling.gpu_resources + ) + iterator = gpu_resources + content { + resource_type = gpu_resources.value.resource_type + minimum = gpu_resources.value.min + maximum = gpu_resources.value.max + } + } } } diff --git a/modules/gke-cluster-standard/variables.tf b/modules/gke-cluster-standard/variables.tf index 221f6b8a55..eebd595a05 100644 --- a/modules/gke-cluster-standard/variables.tf +++ b/modules/gke-cluster-standard/variables.tf @@ -38,11 +38,22 @@ variable "backup_configs" { variable "cluster_autoscaling" { description = "Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler." type = object({ + autoscaling_profile = optional(string, "BALANCED") auto_provisioning_defaults = optional(object({ boot_disk_kms_key = optional(string) + disk_size = optional(number) + disk_type = optional(string, "pd-standard") image_type = optional(string) oauth_scopes = optional(list(string)) service_account = optional(string) + management = optional(object({ + auto_repair = optional(bool, true) + auto_upgrade = optional(bool, true) + })) + shielded_instance_config = object({ + integrity_monitoring = optional(bool, true) + secure_boot = optional(bool, false) + }) })) cpu_limits = optional(object({ min = number @@ -52,8 +63,21 @@ variable "cluster_autoscaling" { min = number max = number })) + gpu_resources = optional(list(object({ + resource_type = string + min = number + max = number + }))) }) default = null + validation { + condition = (var.cluster_autoscaling == null ? true : contains(["BALANCED", "OPTIMIZE_UTILIZATION"], var.cluster_autoscaling.autoscaling_profile)) + error_message = "Invalid autoscaling_profile." + } + validation { + condition = (var.cluster_autoscaling == null ? true : contains(["pd-standard", "pd-ssd", "pd-balanced"], var.cluster_autoscaling.auto_provisioning_defaults.disk_type)) + error_message = "Invalid disk_type." + } } variable "deletion_protection" { diff --git a/modules/gke-cluster-standard/versions.tf b/modules/gke-cluster-standard/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/gke-cluster-standard/versions.tf +++ b/modules/gke-cluster-standard/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/gke-hub/versions.tf b/modules/gke-hub/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/gke-hub/versions.tf +++ b/modules/gke-hub/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/gke-nodepool/main.tf b/modules/gke-nodepool/main.tf index 40f7fcf31c..20d4fcad16 100644 --- a/modules/gke-nodepool/main.tf +++ b/modules/gke-nodepool/main.tf @@ -168,10 +168,7 @@ resource "google_container_node_pool" "nodepool" { gpu_partition_size = var.node_config.guest_accelerator.gpu_driver == null ? null : var.node_config.guest_accelerator.gpu_driver.partition_size dynamic "gpu_sharing_config" { - for_each = lookup( - lookup(var.node_config.guest_accelerator, "gpu_driver", {}), - "max_shared_clients_per_gpu" - ) != null ? [""] : [] + for_each = try(var.node_config.guest_accelerator.gpu_driver.max_shared_clients_per_gpu, null) != null ? [""] : [] content { gpu_sharing_strategy = var.node_config.guest_accelerator.gpu_driver.max_shared_clients_per_gpu != null ? "TIME_SHARING" : null max_shared_clients_per_gpu = var.node_config.guest_accelerator.gpu_driver.max_shared_clients_per_gpu diff --git a/modules/gke-nodepool/variables.tf b/modules/gke-nodepool/variables.tf index 17cfd88c5a..caa735884c 100644 --- a/modules/gke-nodepool/variables.tf +++ b/modules/gke-nodepool/variables.tf @@ -100,7 +100,7 @@ variable "node_config" { validation { condition = ( alltrue([ - for k, v in var.node_config.guest_accelerator[*].gpu_driver : contains([ + for k, v in try(var.node_config.guest_accelerator[0].gpu_driver, {}) : contains([ "GPU_DRIVER_VERSION_UNSPECIFIED", "INSTALLATION_DISABLED", "DEFAULT", "LATEST" ], v.version) diff --git a/modules/gke-nodepool/versions.tf b/modules/gke-nodepool/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/gke-nodepool/versions.tf +++ b/modules/gke-nodepool/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/iam-service-account/versions.tf b/modules/iam-service-account/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/iam-service-account/versions.tf +++ b/modules/iam-service-account/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/kms/versions.tf b/modules/kms/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/kms/versions.tf +++ b/modules/kms/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/logging-bucket/versions.tf b/modules/logging-bucket/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/logging-bucket/versions.tf +++ b/modules/logging-bucket/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/ncc-spoke-ra/versions.tf b/modules/ncc-spoke-ra/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/ncc-spoke-ra/versions.tf +++ b/modules/ncc-spoke-ra/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-address/versions.tf b/modules/net-address/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-address/versions.tf +++ b/modules/net-address/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-cloudnat/versions.tf b/modules/net-cloudnat/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-cloudnat/versions.tf +++ b/modules/net-cloudnat/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-firewall-policy/versions.tf b/modules/net-firewall-policy/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-firewall-policy/versions.tf +++ b/modules/net-firewall-policy/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-ipsec-over-interconnect/versions.tf b/modules/net-ipsec-over-interconnect/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-ipsec-over-interconnect/versions.tf +++ b/modules/net-ipsec-over-interconnect/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-lb-app-ext/main.tf b/modules/net-lb-app-ext/main.tf index ebe438ec13..94a79d9996 100644 --- a/modules/net-lb-app-ext/main.tf +++ b/modules/net-lb-app-ext/main.tf @@ -53,6 +53,9 @@ resource "google_compute_ssl_certificate" "default" { name = "${var.name}-${each.key}" certificate = trimspace(each.value.certificate) private_key = trimspace(each.value.private_key) + lifecycle { + create_before_destroy = true + } } resource "google_compute_managed_ssl_certificate" "default" { diff --git a/modules/net-lb-app-ext/versions.tf b/modules/net-lb-app-ext/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-lb-app-ext/versions.tf +++ b/modules/net-lb-app-ext/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-lb-app-int/README.md b/modules/net-lb-app-int/README.md index dfc4f5eccb..3616494a7d 100644 --- a/modules/net-lb-app-int/README.md +++ b/modules/net-lb-app-int/README.md @@ -223,7 +223,9 @@ module "ilb-l7" { backends = [{ balancing_mode = "RATE" group = "projects/myprj/zones/europe-west1-a/networkEndpointGroups/my-neg" - max_rate = { per_endpoint = 1 } + max_rate = { + per_endpoint = 1 + } }] } } @@ -259,7 +261,9 @@ module "ilb-l7" { backends = [{ balancing_mode = "RATE" group = "my-neg" - max_rate = { per_endpoint = 1 } + max_rate = { + per_endpoint = 1 + } }] } } @@ -296,7 +300,13 @@ module "ilb-l7" { region = "europe-west1" backend_service_configs = { default = { - backends = [{ group = "my-neg" }] + backends = [{ + balancing_mode = "RATE" + group = "my-neg" + max_rate = { + per_endpoint = 1 + } + }] } } neg_configs = { @@ -518,7 +528,9 @@ module "ilb-l7" { backends = [{ balancing_mode = "RATE" group = "neg-nginx-ew8-c" - max_rate = { per_endpoint = 1 } + max_rate = { + per_endpoint = 1 + } }] } home = { @@ -625,7 +637,6 @@ module "ilb-l7" { - ## Files | name | description | resources | @@ -651,7 +662,7 @@ module "ilb-l7" { | [region](variables.tf#L156) | The region where to allocate the ILB resources. | string | ✓ | | | [vpc_config](variables.tf#L183) | VPC-level configuration. | object({…}) | ✓ | | | [address](variables.tf#L17) | Optional IP address used for the forwarding rule. | string | | null | -| [backend_service_configs](variables-backend-service.tf#L19) | Backend service level configuration. | map(object({…})) | | {} | +| [backend_service_configs](variables-backend-service.tf#L19) | Backend service level configuration. | map(object({…})) | | {} | | [description](variables.tf#L23) | Optional description used for resources. | string | | "Terraform managed." | | [global_access](variables.tf#L30) | Allow client access from all regions. | bool | | null | | [group_configs](variables.tf#L36) | Optional unmanaged groups to create. Can be referenced in backends via key or outputs. | map(object({…})) | | {} | @@ -677,5 +688,4 @@ module "ilb-l7" { | [health_check_ids](outputs.tf#L48) | Autogenerated health check ids. | | | [id](outputs.tf#L55) | Fully qualified forwarding rule id. | | | [neg_ids](outputs.tf#L60) | Autogenerated network endpoint group ids. | | - diff --git a/modules/net-lb-app-int/backend-service.tf b/modules/net-lb-app-int/backend-service.tf index 669a291aff..0dcfa46a99 100644 --- a/modules/net-lb-app-int/backend-service.tf +++ b/modules/net-lb-app-int/backend-service.tf @@ -114,28 +114,6 @@ resource "google_compute_region_backend_service" "default" { } } - dynamic "connection_tracking_policy" { - for_each = ( - each.value.connection_tracking == null - ? [] - : [each.value.connection_tracking] - ) - iterator = cb - content { - connection_persistence_on_unhealthy_backends = ( - cb.value.persist_conn_on_unhealthy != null - ? cb.value.persist_conn_on_unhealthy - : null - ) - idle_timeout_sec = cb.value.idle_timeout_sec - tracking_mode = ( - cb.value.track_per_session != null - ? cb.value.track_per_session - : null - ) - } - } - dynamic "consistent_hash" { for_each = ( each.value.consistent_hash == null ? [] : [each.value.consistent_hash] diff --git a/modules/net-lb-app-int/variables-backend-service.tf b/modules/net-lb-app-int/variables-backend-service.tf index 5cfe9a5aca..728e21dbc3 100644 --- a/modules/net-lb-app-int/variables-backend-service.tf +++ b/modules/net-lb-app-int/variables-backend-service.tf @@ -35,11 +35,6 @@ variable "backend_service_configs" { capacity_scaler = optional(number, 1) description = optional(string, "Terraform managed.") failover = optional(bool, false) - max_connections = optional(object({ - per_endpoint = optional(number) - per_group = optional(number) - per_instance = optional(number) - })) max_rate = optional(object({ per_endpoint = optional(number) per_group = optional(number) @@ -58,11 +53,6 @@ variable "backend_service_configs" { nanos = optional(number) })) })) - connection_tracking = optional(object({ - idle_timeout_sec = optional(number) - persist_conn_on_unhealthy = optional(string) - track_per_session = optional(bool) - })) consistent_hash = optional(object({ http_header_name = optional(string) minimum_ring_size = optional(number) @@ -79,7 +69,6 @@ variable "backend_service_configs" { failover_config = optional(object({ disable_conn_drain = optional(bool) drop_traffic_if_unhealthy = optional(bool) - ratio = optional(number) })) iap_config = optional(object({ oauth2_client_id = string diff --git a/modules/net-lb-app-int/versions.tf b/modules/net-lb-app-int/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-lb-app-int/versions.tf +++ b/modules/net-lb-app-int/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-lb-ext/versions.tf b/modules/net-lb-ext/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-lb-ext/versions.tf +++ b/modules/net-lb-ext/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-lb-int/versions.tf b/modules/net-lb-int/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-lb-int/versions.tf +++ b/modules/net-lb-int/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-lb-proxy-int/versions.tf b/modules/net-lb-proxy-int/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-lb-proxy-int/versions.tf +++ b/modules/net-lb-proxy-int/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-swp/versions.tf b/modules/net-swp/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-swp/versions.tf +++ b/modules/net-swp/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vlan-attachment/versions.tf b/modules/net-vlan-attachment/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vlan-attachment/versions.tf +++ b/modules/net-vlan-attachment/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vpc-firewall/versions.tf b/modules/net-vpc-firewall/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vpc-firewall/versions.tf +++ b/modules/net-vpc-firewall/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vpc-peering/README.md b/modules/net-vpc-peering/README.md index 0998d7b3ed..1def1ad19a 100644 --- a/modules/net-vpc-peering/README.md +++ b/modules/net-vpc-peering/README.md @@ -82,6 +82,7 @@ module "peering" { | [peer_create_peering](variables.tf#L22) | Create the peering on the remote side. If false, only the peering from this network to the remote network is created. | bool | | true | | [prefix](variables.tf#L33) | Optional name prefix for the network peerings. | string | | null | | [routes_config](variables.tf#L43) | Control import/export for local and remote peer. Remote configuration is only used when creating remote peering. | object({…}) | | {} | +| [stack_type](variables.tf#L63) | IP version(s) of traffic and routes that are allowed to be imported or exported between peer networks. Possible values: IPV4_ONLY, IPV4_IPV6. | string | | null | ## Outputs diff --git a/modules/net-vpc-peering/main.tf b/modules/net-vpc-peering/main.tf index 2d88a0b41b..bb754ae8c2 100644 --- a/modules/net-vpc-peering/main.tf +++ b/modules/net-vpc-peering/main.tf @@ -36,6 +36,7 @@ resource "google_compute_network_peering" "local_network_peering" { import_subnet_routes_with_public_ip = try( var.routes_config.local.public_import, null ) + stack_type = var.stack_type } resource "google_compute_network_peering" "peer_network_peering" { @@ -55,5 +56,6 @@ resource "google_compute_network_peering" "peer_network_peering" { import_subnet_routes_with_public_ip = try( var.routes_config.peer.public_import, null ) + stack_type = var.stack_type depends_on = [google_compute_network_peering.local_network_peering] } diff --git a/modules/net-vpc-peering/variables.tf b/modules/net-vpc-peering/variables.tf index ade807415b..9058da1ded 100644 --- a/modules/net-vpc-peering/variables.tf +++ b/modules/net-vpc-peering/variables.tf @@ -59,3 +59,13 @@ variable "routes_config" { nullable = false default = {} } + +variable "stack_type" { + description = "IP version(s) of traffic and routes that are allowed to be imported or exported between peer networks. Possible values: IPV4_ONLY, IPV4_IPV6." + type = string + default = null + validation { + condition = var.stack_type == "IPV4_ONLY" || var.stack_type == "IPV4_IPV6" || var.stack_type == null + error_message = "The stack_type must be either 'IPV4_ONLY' or 'IPV4_IPV6'." + } +} \ No newline at end of file diff --git a/modules/net-vpc-peering/versions.tf b/modules/net-vpc-peering/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vpc-peering/versions.tf +++ b/modules/net-vpc-peering/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vpc/versions.tf b/modules/net-vpc/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vpc/versions.tf +++ b/modules/net-vpc/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vpn-dynamic/versions.tf b/modules/net-vpn-dynamic/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vpn-dynamic/versions.tf +++ b/modules/net-vpn-dynamic/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vpn-ha/versions.tf b/modules/net-vpn-ha/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vpn-ha/versions.tf +++ b/modules/net-vpn-ha/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/net-vpn-static/versions.tf b/modules/net-vpn-static/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/net-vpn-static/versions.tf +++ b/modules/net-vpn-static/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/organization/README.md b/modules/organization/README.md index 86df8ab3ba..b760f03626 100644 --- a/modules/organization/README.md +++ b/modules/organization/README.md @@ -456,7 +456,7 @@ module "org" { | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [organization_id](variables.tf#L211) | Organization id in organizations/nnnnnn format. | string | ✓ | | +| [organization_id](variables.tf#L212) | Organization id in organizations/nnnnnn format. | string | ✓ | | | [contacts](variables.tf#L17) | List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. | map(list(string)) | | {} | | [custom_roles](variables.tf#L24) | Map of role name => list of permissions to create in this project. | map(list(string)) | | {} | | [firewall_policy](variables.tf#L31) | Hierarchical firewall policies to associate to the organization. | object({…}) | | null | @@ -466,14 +466,14 @@ module "org" { | [iam_bindings_additive](variables.tf#L69) | Individual additive IAM bindings. Keys are arbitrary. | map(object({…})) | | {} | | [logging_data_access](variables.tf#L84) | Control activation of data access logs. Format is service => { log type => [exempted members]}. The special 'allServices' key denotes configuration for all services. | map(map(list(string))) | | {} | | [logging_exclusions](variables.tf#L99) | Logging exclusions for this organization in the form {NAME -> FILTER}. | map(string) | | {} | -| [logging_sinks](variables.tf#L106) | Logging sinks to create for the organization. | map(object({…})) | | {} | -| [network_tags](variables.tf#L136) | Network tags by key name. If `id` is provided, key creation is skipped. The `iam` attribute behaves like the similarly named one at module level. | map(object({…})) | | {} | -| [org_policies](variables.tf#L158) | Organization policies applied to this organization keyed by policy name. | map(object({…})) | | {} | -| [org_policies_data_path](variables.tf#L185) | Path containing org policies in YAML format. | string | | null | -| [org_policy_custom_constraints](variables.tf#L191) | Organization policy custom constraints keyed by constraint name. | map(object({…})) | | {} | -| [org_policy_custom_constraints_data_path](variables.tf#L205) | Path containing org policy custom constraints in YAML format. | string | | null | -| [tag_bindings](variables.tf#L220) | Tag bindings for this organization, in key => tag value id format. | map(string) | | null | -| [tags](variables.tf#L226) | Tags by key name. If `id` is provided, key or value creation is skipped. The `iam` attribute behaves like the similarly named one at module level. | map(object({…})) | | {} | +| [logging_sinks](variables.tf#L106) | Logging sinks to create for the organization. | map(object({…})) | | {} | +| [network_tags](variables.tf#L137) | Network tags by key name. If `id` is provided, key creation is skipped. The `iam` attribute behaves like the similarly named one at module level. | map(object({…})) | | {} | +| [org_policies](variables.tf#L159) | Organization policies applied to this organization keyed by policy name. | map(object({…})) | | {} | +| [org_policies_data_path](variables.tf#L186) | Path containing org policies in YAML format. | string | | null | +| [org_policy_custom_constraints](variables.tf#L192) | Organization policy custom constraints keyed by constraint name. | map(object({…})) | | {} | +| [org_policy_custom_constraints_data_path](variables.tf#L206) | Path containing org policy custom constraints in YAML format. | string | | null | +| [tag_bindings](variables.tf#L221) | Tag bindings for this organization, in key => tag value id format. | map(string) | | null | +| [tags](variables.tf#L227) | Tags by key name. If `id` is provided, key or value creation is skipped. The `iam` attribute behaves like the similarly named one at module level. | map(object({…})) | | {} | ## Outputs diff --git a/modules/organization/logging.tf b/modules/organization/logging.tf index 7719c0fb62..7f0665e3ab 100644 --- a/modules/organization/logging.tf +++ b/modules/organization/logging.tf @@ -21,7 +21,7 @@ locals { for type in ["bigquery", "logging", "pubsub", "storage"] : type => { for name, sink in var.logging_sinks : - name => sink if sink.type == type + name => sink if sink.iam && sink.type == type } } } diff --git a/modules/organization/variables.tf b/modules/organization/variables.tf index c9899e2e1d..f664deaef1 100644 --- a/modules/organization/variables.tf +++ b/modules/organization/variables.tf @@ -112,6 +112,7 @@ variable "logging_sinks" { disabled = optional(bool, false) exclusions = optional(map(string), {}) filter = string + iam = optional(bool, true) include_children = optional(bool, true) type = string })) diff --git a/modules/organization/versions.tf b/modules/organization/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/organization/versions.tf +++ b/modules/organization/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/project/README.md b/modules/project/README.md index 43dfbc7044..9d94d9f2d1 100644 --- a/modules/project/README.md +++ b/modules/project/README.md @@ -11,15 +11,16 @@ This module implements the creation and management of one GCP project including - [Authoritative IAM](#authoritative-iam) - [Additive IAM](#additive-iam) - [Service Identities and Authoritative IAM](#service-identities-and-authoritative-iam) - - [Service Identities Requiring Manual Iam Grants](#service-identities-requiring-manual-iam-grants) + - [Service Identities Requiring Manual IAM Grants](#service-identities-requiring-manual-iam-grants) - [Shared VPC](#shared-vpc) - [Organization Policies](#organization-policies) - [Organization Policy Factory](#organization-policy-factory) - [Log Sinks](#log-sinks) - [Data Access Logs](#data-access-logs) -- [Cloud Kms Encryption Keys](#cloud-kms-encryption-keys) +- [Cloud KMS Encryption Keys](#cloud-kms-encryption-keys) - [Tags](#tags) - [Outputs](#outputs) + - [Managing project related configuration without creating it](#managing-project-related-configuration-without-creating-it) - [Files](#files) - [Variables](#variables) - [Outputs](#outputs) @@ -30,16 +31,16 @@ This module implements the creation and management of one GCP project including ```hcl module "project" { source = "./fabric/modules/project" - billing_account = "123456-123456-123456" - name = "myproject" - parent = "folders/1234567890" - prefix = "foo" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix services = [ "container.googleapis.com", "stackdriver.googleapis.com" ] } -# tftest modules=1 resources=3 inventory=basic.yaml +# tftest modules=1 resources=3 inventory=basic.yaml e2e ``` ## IAM @@ -65,10 +66,10 @@ locals { module "project" { source = "./fabric/modules/project" - billing_account = "123456-123456-123456" - name = "project-example" - parent = "folders/1234567890" - prefix = "foo" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix services = [ "container.googleapis.com", "stackdriver.googleapis.com" @@ -87,12 +88,12 @@ The `group_iam` variable uses group email addresses as keys and is a convenient ```hcl module "project" { source = "./fabric/modules/project" - billing_account = "123456-123456-123456" - name = "project-example" - parent = "folders/1234567890" - prefix = "foo" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix group_iam = { - "gcp-security-admins@example.com" = [ + (var.group_email) = [ "roles/cloudasset.owner", "roles/cloudsupport.techSupportEditor", "roles/iam.securityReviewer", @@ -100,7 +101,7 @@ module "project" { ] } } -# tftest modules=1 resources=5 inventory=iam-group.yaml +# tftest modules=1 resources=5 inventory=iam-group.yaml e2e ``` The `iam_bindings` variable behaves like a more verbose version of `iam`, and allows setting binding-level IAM conditions. @@ -108,10 +109,10 @@ The `iam_bindings` variable behaves like a more verbose version of `iam`, and al ```hcl module "project" { source = "./fabric/modules/project" - billing_account = "123456-123456-123456" - name = "project-example" - parent = "folders/1234567890" - prefix = "foo" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix services = [ "container.googleapis.com", "stackdriver.googleapis.com" @@ -119,7 +120,7 @@ module "project" { iam_bindings = { iam_admin_conditional = { members = [ - "group:test-admins@example.org" + "group:${var.group_email}" ] role = "roles/resourcemanager.projectIamAdmin" condition = { @@ -135,7 +136,7 @@ module "project" { } } } -# tftest modules=1 resources=4 inventory=iam-bindings.yaml +# tftest modules=1 resources=4 inventory=iam-bindings.yaml e2e ``` ### Additive IAM @@ -146,19 +147,22 @@ The `iam_bindings_additive` variable allows setting individual role/principal bi ```hcl module "project" { - source = "./fabric/modules/project" - name = "project-1" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix services = [ "compute.googleapis.com" ] iam_bindings_additive = { group-owner = { - member = "group:p1-owners@example.org" + member = "group:${var.group_email}" role = "roles/owner" } } } -# tftest modules=1 resources=3 inventory=iam-bindings-additive.yaml +# tftest modules=1 resources=3 inventory=iam-bindings-additive.yaml e2e ``` ### Service Identities and Authoritative IAM @@ -167,23 +171,21 @@ As mentioned above, there are cases where authoritative management of specific I ```hcl module "project" { - source = "./fabric/modules/project" - name = "project-example" - group_iam = { - "foo@example.com" = [ - "roles/editor" - ] - } + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix iam = { "roles/editor" = [ "serviceAccount:${module.project.service_accounts.cloud_services}" ] } } -# tftest modules=1 resources=2 +# tftest modules=1 resources=2 e2e ``` -### Service Identities Requiring Manual Iam Grants +### Service Identities Requiring Manual IAM Grants The module will create service identities at project creation instead of creating of them at the time of first use. This allows granting these service identities roles in other projects, something which is usually necessary in a Shared VPC context. @@ -191,15 +193,21 @@ You can grant roles to service identities using the following construct: ```hcl module "project" { - source = "./fabric/modules/project" - name = "project-example" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix + services = [ + "apigee.googleapis.com", + ] iam = { "roles/apigee.serviceAgent" = [ "serviceAccount:${module.project.service_accounts.robots.apigee}" ] } } -# tftest modules=1 resources=2 +# tftest modules=1 resources=4 e2e ``` This table lists all affected services and roles that you need to grant to service identities @@ -225,16 +233,26 @@ You can enable Shared VPC Host at the project level and manage project service a ```hcl module "host-project" { - source = "./fabric/modules/project" - name = "my-host-project" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "host" + parent = var.folder_id + prefix = var.prefix shared_vpc_host_config = { enabled = true } } module "service-project" { - source = "./fabric/modules/project" - name = "my-service-project" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "service" + parent = var.folder_id + prefix = var.prefix + services = [ + "container.googleapis.com", + "run.googleapis.com" + ] shared_vpc_service_config = { host_project = module.host-project.project_id service_identity_iam = { @@ -250,23 +268,29 @@ module "service-project" { } } } -# tftest modules=2 resources=8 inventory=shared-vpc.yaml +# tftest modules=2 resources=10 inventory=shared-vpc.yaml e2e ``` The module allows also granting necessary permissions in host project to service identities by specifying which services will be used in service project in `grant_iam_for_services`. ```hcl module "host-project" { - source = "./fabric/modules/project" - name = "my-host-project" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "host" + parent = var.folder_id + prefix = var.prefix shared_vpc_host_config = { enabled = true } } module "service-project" { - source = "./fabric/modules/project" - name = "my-service-project" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "service" + parent = var.folder_id + prefix = var.prefix services = [ "container.googleapis.com", ] @@ -275,7 +299,7 @@ module "service-project" { service_iam_grants = module.service-project.services } } -# tftest modules=2 resources=9 inventory=shared-vpc-auto-grants.yaml +# tftest modules=2 resources=9 inventory=shared-vpc-auto-grants.yaml e2e ``` ## Organization Policies @@ -285,10 +309,10 @@ To manage organization policies, the `orgpolicy.googleapis.com` service should b ```hcl module "project" { source = "./fabric/modules/project" - billing_account = "123456-123456-123456" - name = "project-example" - parent = "folders/1234567890" - prefix = "foo" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix org_policies = { "compute.disableGuestAttributesAccess" = { rules = [{ enforce = true }] @@ -334,7 +358,7 @@ module "project" { } } } -# tftest modules=1 resources=8 inventory=org-policies.yaml +# tftest modules=1 resources=8 inventory=org-policies.yaml e2e ``` ### Organization Policy Factory @@ -348,13 +372,13 @@ The example below deploys a few organization policies split between two YAML fil ```hcl module "project" { source = "./fabric/modules/project" - billing_account = "123456-123456-123456" - name = "project-example" - parent = "folders/1234567890" - prefix = "foo" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix org_policies_data_path = "configs/org-policies/" } -# tftest modules=1 resources=8 files=boolean,list inventory=org-policies.yaml +# tftest modules=1 resources=8 files=boolean,list inventory=org-policies.yaml e2e ``` ```yaml @@ -415,6 +439,7 @@ module "gcs" { source = "./fabric/modules/gcs" project_id = var.project_id name = "gcs_sink" + prefix = var.prefix force_destroy = true } @@ -422,6 +447,7 @@ module "dataset" { source = "./fabric/modules/bigquery-dataset" project_id = var.project_id id = "bq_sink" + options = { delete_contents_on_destroy = true } } module "pubsub" { @@ -433,15 +459,19 @@ module "pubsub" { module "bucket" { source = "./fabric/modules/logging-bucket" parent_type = "project" - parent = "my-project" + parent = var.project_id id = "bucket" } module "project-host" { source = "./fabric/modules/project" - name = "my-project" - billing_account = "123456-123456-123456" - parent = "folders/1234567890" + name = "project" + billing_account = var.billing_account_id + parent = var.folder_id + prefix = var.prefix + services = [ + "logging.googleapis.com" + ] logging_sinks = { warnings = { destination = module.gcs.id @@ -471,7 +501,7 @@ module "project-host" { no-gce-instances = "resource.type=gce_instance" } } -# tftest modules=5 resources=14 inventory=logging.yaml +# tftest modules=5 resources=15 inventory=logging.yaml e2e ``` ## Data Access Logs @@ -483,13 +513,14 @@ This example shows how to set a non-authoritative access log configuration: ```hcl module "project" { source = "./fabric/modules/project" - name = "my-project" - billing_account = "123456-123456-123456" - parent = "folders/1234567890" + name = "project" + billing_account = var.billing_account_id + parent = var.folder_id + prefix = var.prefix logging_data_access = { allServices = { # logs for principals listed here will be excluded - ADMIN_READ = ["group:organization-admins@example.org"] + ADMIN_READ = ["group:${var.group_email}"] } "storage.googleapis.com" = { DATA_READ = [] @@ -497,33 +528,34 @@ module "project" { } } } -# tftest modules=1 resources=3 inventory=logging-data-access.yaml +# tftest modules=1 resources=3 inventory=logging-data-access.yaml e2e ``` -## Cloud Kms Encryption Keys +## Cloud KMS Encryption Keys The module offers a simple, centralized way to assign `roles/cloudkms.cryptoKeyEncrypterDecrypter` to service identities. ```hcl module "project" { - source = "./fabric/modules/project" - name = "my-project" - prefix = "foo" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "project" + prefix = var.prefix + parent = var.folder_id services = [ "compute.googleapis.com", "storage.googleapis.com" ] service_encryption_key_ids = { compute = [ - "projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce", - "projects/kms-central-prj/locations/europe-west4/keyRings/my-keyring/cryptoKeys/europe4-gce" + var.kms_key.id ] storage = [ - "projects/kms-central-prj/locations/europe/keyRings/my-keyring/cryptoKeys/europe-gcs" + var.kms_key.id ] } } -# tftest modules=1 resources=7 +# tftest modules=1 resources=6 e2e ``` ## Tags @@ -548,7 +580,8 @@ module "org" { module "project" { source = "./fabric/modules/project" - name = "test-project" + name = "project" + parent = var.folder_id tag_bindings = { env-prod = module.org.tag_values["environment/prod"].id foo = "tagValues/12345678" @@ -565,8 +598,11 @@ One non-obvious output is `service_accounts`, which offers a simple way to disco ```hcl module "project" { - source = "./fabric/modules/project" - name = "project-example" + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "project" + prefix = var.prefix + parent = var.folder_id services = [ "compute.googleapis.com" ] @@ -575,9 +611,222 @@ module "project" { output "compute_robot" { value = module.project.service_accounts.robots.compute } -# tftest modules=1 resources=2 inventory:outputs.yaml +# tftest modules=1 resources=2 inventory:outputs.yaml e2e ``` +### Managing project related configuration without creating it + +The module offers managing all related resources without ever touching the project itself by using `project_create = false` + +```hcl +module "create-project" { + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix +} + +module "project" { + source = "./fabric/modules/project" + depends_on = [module.create-project] + billing_account = var.billing_account_id + name = "project" + parent = var.folder_id + prefix = var.prefix + project_create = false + + group_iam = { + (var.group_email) = [ + "roles/cloudasset.owner", + "roles/cloudsupport.techSupportEditor", + "roles/iam.securityReviewer", + "roles/logging.admin", + ] + } + iam_bindings = { + iam_admin_conditional = { + members = [ + "group:${var.group_email}" + ] + role = "roles/resourcemanager.projectIamAdmin" + condition = { + title = "delegated_network_user_one" + expression = <<-END + api.getAttribute( + 'iam.googleapis.com/modifiedGrantsByRole', [] + ).hasOnly([ + 'roles/compute.networkAdmin' + ]) + END + } + } + } + iam_bindings_additive = { + group-owner = { + member = "group:${var.group_email}" + role = "roles/owner" + } + } + iam = { + "roles/editor" = [ + "serviceAccount:${module.project.service_accounts.cloud_services}" + ] + "roles/apigee.serviceAgent" = [ + "serviceAccount:${module.project.service_accounts.robots.apigee}" + ] + } + logging_data_access = { + allServices = { + # logs for principals listed here will be excluded + ADMIN_READ = ["group:${var.group_email}"] + } + "storage.googleapis.com" = { + DATA_READ = [] + DATA_WRITE = [] + } + } + logging_sinks = { + warnings = { + destination = module.gcs.id + filter = "severity=WARNING" + type = "storage" + } + info = { + destination = module.dataset.id + filter = "severity=INFO" + type = "bigquery" + } + notice = { + destination = module.pubsub.id + filter = "severity=NOTICE" + type = "pubsub" + } + debug = { + destination = module.bucket.id + filter = "severity=DEBUG" + exclusions = { + no-compute = "logName:compute" + } + type = "logging" + } + } + logging_exclusions = { + no-gce-instances = "resource.type=gce_instance" + } + org_policies = { + "compute.disableGuestAttributesAccess" = { + rules = [{ enforce = true }] + } + "compute.skipDefaultNetworkCreation" = { + rules = [{ enforce = true }] + } + "iam.disableServiceAccountKeyCreation" = { + rules = [{ enforce = true }] + } + "iam.disableServiceAccountKeyUpload" = { + rules = [ + { + condition = { + expression = "resource.matchTagId('tagKeys/1234', 'tagValues/1234')" + title = "condition" + description = "test condition" + location = "somewhere" + } + enforce = true + }, + { + enforce = false + } + ] + } + "iam.allowedPolicyMemberDomains" = { + rules = [{ + allow = { + values = ["C0xxxxxxx", "C0yyyyyyy"] + } + }] + } + "compute.trustedImageProjects" = { + rules = [{ + allow = { + values = ["projects/my-project"] + } + }] + } + "compute.vmExternalIpAccess" = { + rules = [{ deny = { all = true } }] + } + } + shared_vpc_service_config = { + host_project = module.host-project.project_id + service_iam_grants = module.project.services + service_identity_iam = { + "roles/cloudasset.owner" = [ + "cloudservices", "container-engine" + ] + } + } + services = [ + "apigee.googleapis.com", + "bigquery.googleapis.com", + "container.googleapis.com", + "logging.googleapis.com", + "run.googleapis.com", + "storage.googleapis.com", + ] + service_encryption_key_ids = { + compute = [ + var.kms_key.id + ] + storage = [ + var.kms_key.id + ] + } +} + +module "host-project" { + source = "./fabric/modules/project" + billing_account = var.billing_account_id + name = "host" + parent = var.folder_id + prefix = var.prefix + shared_vpc_host_config = { + enabled = true + } +} + +module "gcs" { + source = "./fabric/modules/gcs" + project_id = var.project_id + name = "gcs_sink" + prefix = var.prefix + force_destroy = true +} + +module "dataset" { + source = "./fabric/modules/bigquery-dataset" + project_id = var.project_id + id = "bq_sink" + options = { delete_contents_on_destroy = true } +} + +module "pubsub" { + source = "./fabric/modules/pubsub" + project_id = var.project_id + name = "pubsub_sink" +} + +module "bucket" { + source = "./fabric/modules/logging-bucket" + parent_type = "project" + parent = var.project_id + id = "bucket" +} +# tftest modules=7 resources=53 inventory=data.yaml e2e +``` + + ## Files diff --git a/modules/project/logging.tf b/modules/project/logging.tf index 0181f04f56..6694c176a2 100644 --- a/modules/project/logging.tf +++ b/modules/project/logging.tf @@ -69,7 +69,8 @@ resource "google_logging_project_sink" "sink" { depends_on = [ google_project_iam_binding.authoritative, google_project_iam_binding.bindings, - google_project_iam_member.bindings + google_project_iam_member.bindings, + google_project_service.project_services ] } diff --git a/modules/project/service-agents.yaml b/modules/project/service-agents.yaml index c8eff2df97..eb38dc4cc9 100644 --- a/modules/project/service-agents.yaml +++ b/modules/project/service-agents.yaml @@ -18,7 +18,7 @@ service_agent: "service-%s@gcp-sa-adsdatahub.iam.gserviceaccount.com" - name: "aiplatform" service_agent: "service-%s@gcp-sa-aiplatform.iam.gserviceaccount.com" - jit: true + jit: true # roles/aiplatform.customCodeServiceAgent - name: "aiplatform-cc" service_agent: "service-%s@gcp-sa-aiplatform-cc.iam.gserviceaccount.com" - name: "alloydb" @@ -37,7 +37,7 @@ service_agent: "service-%s@gcp-sa-apigateway-mgmt.iam.gserviceaccount.com" - name: "apigee" service_agent: "service-%s@gcp-sa-apigee.iam.gserviceaccount.com" - jit: true #roles/apigee.serviceAgent + jit: true # roles/apigee.serviceAgent - name: "apigeeregistry" service_agent: "service-%s@gcp-sa-apigeeregistry.iam.gserviceaccount.com" - name: "appdevelopmentexperience" @@ -221,7 +221,7 @@ service_agent: "service-%s@gcp-sa-healthcare.iam.gserviceaccount.com" - name: "iap" service_agent: "service-%s@gcp-sa-iap.iam.gserviceaccount.com" - jit: true + jit: true # none - name: "identitytoolkit" service_agent: "service-%s@gcp-sa-identitytoolkit.iam.gserviceaccount.com" - name: "ids" @@ -314,7 +314,7 @@ service_agent: "service-%s@gcp-sa-spectrumsas.iam.gserviceaccount.com" - name: "secretmanager" service_agent: "service-%s@gcp-sa-secretmanager.iam.gserviceaccount.com" - jit: true + jit: true # none - name: "securedlandingzone" service_agent: "service-%s@gcp-sa-slz.iam.gserviceaccount.com" - name: "securitycenter-notification" diff --git a/modules/project/versions.tf b/modules/project/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/project/versions.tf +++ b/modules/project/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/projects-data-source/versions.tf b/modules/projects-data-source/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/projects-data-source/versions.tf +++ b/modules/projects-data-source/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/pubsub/versions.tf b/modules/pubsub/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/pubsub/versions.tf +++ b/modules/pubsub/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/secret-manager/README.md b/modules/secret-manager/README.md index dc0929832f..8d1d8b110b 100644 --- a/modules/secret-manager/README.md +++ b/modules/secret-manager/README.md @@ -17,8 +17,10 @@ module "secret-manager" { source = "./fabric/modules/secret-manager" project_id = "my-project" secrets = { - test-auto = null - test-manual = ["europe-west1", "europe-west4"] + test-auto = {} + test-manual = { + locations = ["europe-west1", "europe-west4"] + } } } # tftest modules=1 resources=2 @@ -33,8 +35,10 @@ module "secret-manager" { source = "./fabric/modules/secret-manager" project_id = "my-project" secrets = { - test-auto = null - test-manual = ["europe-west1", "europe-west4"] + test-auto = {} + test-manual = { + locations = ["europe-west1", "europe-west4"] + } } iam = { test-auto = { @@ -57,8 +61,10 @@ module "secret-manager" { source = "./fabric/modules/secret-manager" project_id = "my-project" secrets = { - test-auto = null - test-manual = ["europe-west1", "europe-west4"] + test-auto = {} + test-manual = { + locations = ["europe-west1", "europe-west4"] + } } versions = { test-auto = { @@ -75,34 +81,40 @@ module "secret-manager" { ### Secret with customer managed encryption key -Secrets will be used if an encryption key is set in the `encryption_key` variable for the secret region. +CMEK will be used if an encryption key is set in the `keys` field of `secrets` object for the secret region. For secrets with auto-replication, a global key must be specified. ```hcl module "secret-manager" { source = "./fabric/modules/secret-manager" project_id = "my-project" secrets = { - test-encryption = ["europe-west1", "europe-west4"] - } - encryption_key = { - europe-west1 = "projects/PROJECT_ID/locations/europe-west1/keyRings/KEYRING/cryptoKeys/KEY" - europe-west4 = "projects/PROJECT_ID/locations/europe-west4/keyRings/KEYRING/cryptoKeys/KEY" + test-auto = { + keys = { + global = "projects/PROJECT_ID/locations/global/keyRings/KEYRING/cryptoKeys/KEY" + } + } + test-auto-nokeys = {} + test-manual = { + locations = ["europe-west1", "europe-west4"] + keys = { + europe-west1 = "projects/PROJECT_ID/locations/europe-west1/keyRings/KEYRING/cryptoKeys/KEY" + europe-west4 = "projects/PROJECT_ID/locations/europe-west4/keyRings/KEYRING/cryptoKeys/KEY" + } + } } } -# tftest modules=1 resources=1 +# tftest modules=1 resources=3 ``` - ## Variables | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [project_id](variables.tf#L35) | Project id where the keyring will be created. | string | ✓ | | -| [encryption_key](variables.tf#L17) | Self link of the KMS keys in {LOCATION => KEY} format. A key must be provided for all replica locations. | map(string) | | null | -| [iam](variables.tf#L23) | IAM bindings in {SECRET => {ROLE => [MEMBERS]}} format. | map(map(list(string))) | | {} | -| [labels](variables.tf#L29) | Optional labels for each secret. | map(map(string)) | | {} | -| [secrets](variables.tf#L40) | Map of secrets to manage and their locations. If locations is null, automatic management will be set. | map(list(string)) | | {} | -| [versions](variables.tf#L46) | Optional versions to manage for each secret. Version names are only used internally to track individual versions. | map(map(object({…}))) | | {} | +| [project_id](variables.tf#L29) | Project id where the keyring will be created. | string | ✓ | | +| [iam](variables.tf#L17) | IAM bindings in {SECRET => {ROLE => [MEMBERS]}} format. | map(map(list(string))) | | {} | +| [labels](variables.tf#L23) | Optional labels for each secret. | map(map(string)) | | {} | +| [secrets](variables.tf#L34) | Map of secrets to manage, their locations and KMS keys in {LOCATION => KEY} format. {GLOBAL => KEY} format enables CMEK for automatic managed secrets. If locations is null, automatic management will be set. | map(object({…})) | | {} | +| [versions](variables.tf#L43) | Optional versions to manage for each secret. Version names are only used internally to track individual versions. | map(map(object({…}))) | | {} | ## Outputs @@ -112,7 +124,6 @@ module "secret-manager" { | [secrets](outputs.tf#L24) | Secret resources. | | | [version_ids](outputs.tf#L29) | Version ids keyed by secret name : version name. | | | [versions](outputs.tf#L36) | Secret versions. | ✓ | - ## Requirements diff --git a/modules/secret-manager/main.tf b/modules/secret-manager/main.tf index 1a17ac21cf..61f4d5efe2 100644 --- a/modules/secret-manager/main.tf +++ b/modules/secret-manager/main.tf @@ -42,27 +42,32 @@ resource "google_secret_manager_secret" "default" { labels = lookup(var.labels, each.key, null) dynamic "replication" { - for_each = each.value == null ? [""] : [] + for_each = each.value.locations == null ? [""] : [] content { - # TODO(jccb): support custom keys inside auto - auto {} + auto { + dynamic "customer_managed_encryption" { + for_each = try(lookup(each.value.keys, "global", null) == null ? [] : [""], []) + content { + kms_key_name = each.value.keys["global"] + } + } + } } } dynamic "replication" { - for_each = each.value == null ? [] : [each.value] - iterator = locations + for_each = each.value.locations == null ? [] : [""] content { user_managed { dynamic "replicas" { - for_each = locations.value + for_each = each.value.locations iterator = location content { location = location.value dynamic "customer_managed_encryption" { - for_each = try(var.encryption_key[location.value] != null ? [""] : [], []) + for_each = try(lookup(each.value.keys, location.value, null) == null ? [] : [""], []) content { - kms_key_name = var.encryption_key[location.value] + kms_key_name = each.value.keys[location.value] } } } @@ -88,4 +93,4 @@ resource "google_secret_manager_secret_iam_binding" "default" { role = each.value.role secret_id = google_secret_manager_secret.default[each.value.secret].id members = each.value.members -} +} \ No newline at end of file diff --git a/modules/secret-manager/variables.tf b/modules/secret-manager/variables.tf index 7d7b528484..089f2a69b2 100644 --- a/modules/secret-manager/variables.tf +++ b/modules/secret-manager/variables.tf @@ -1,5 +1,5 @@ /** - * Copyright 2022 Google LLC + * Copyright 2023 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -14,12 +14,6 @@ * limitations under the License. */ -variable "encryption_key" { - description = "Self link of the KMS keys in {LOCATION => KEY} format. A key must be provided for all replica locations." - type = map(string) - default = null -} - variable "iam" { description = "IAM bindings in {SECRET => {ROLE => [MEMBERS]}} format." type = map(map(list(string))) @@ -38,9 +32,12 @@ variable "project_id" { } variable "secrets" { - description = "Map of secrets to manage and their locations. If locations is null, automatic management will be set." - type = map(list(string)) - default = {} + description = "Map of secrets to manage, their locations and KMS keys in {LOCATION => KEY} format. {GLOBAL => KEY} format enables CMEK for automatic managed secrets. If locations is null, automatic management will be set." + type = map(object({ + locations = optional(list(string), null) + keys = optional(map(string), null) + })) + default = {} } variable "versions" { @@ -50,4 +47,4 @@ variable "versions" { data = string }))) default = {} -} +} \ No newline at end of file diff --git a/modules/secret-manager/versions.tf b/modules/secret-manager/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/secret-manager/versions.tf +++ b/modules/secret-manager/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/service-directory/versions.tf b/modules/service-directory/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/service-directory/versions.tf +++ b/modules/service-directory/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/source-repository/versions.tf b/modules/source-repository/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/source-repository/versions.tf +++ b/modules/source-repository/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/modules/vpc-sc/README.md b/modules/vpc-sc/README.md index 91d90d466d..ee0d63a30f 100644 --- a/modules/vpc-sc/README.md +++ b/modules/vpc-sc/README.md @@ -34,6 +34,8 @@ module "test" { # tftest modules=1 resources=1 inventory=access-policy.yaml ``` +#### Scoped policy + If you need the module to create a scoped policy for you, specify 'scopes' of the policy in the `access_policy_create` variable: ```hcl @@ -49,6 +51,23 @@ module "test" { # tftest modules=1 resources=1 inventory=scoped-access-policy.yaml ``` +#### Access policy IAM + +The usual IAM interface is also implemented here, and can be used with service accounts or user principals: + +```hcl +module "test" { + source = "./fabric/modules/vpc-sc" + access_policy = "12345678" + iam = { + "roles/accesscontextmanager.policyAdmin" = [ + "user:foo@example.org" + ] + } +} +# tftest modules=1 resources=1 +``` + ### Access levels As highlighted above, the `access_levels` type replicates the underlying resource structure. @@ -190,6 +209,7 @@ module "test" { | name | description | resources | |---|---|---| | [access-levels.tf](./access-levels.tf) | Access level resources. | google_access_context_manager_access_level | +| [iam.tf](./iam.tf) | IAM bindings | google_access_context_manager_access_policy_iam_binding · google_access_context_manager_access_policy_iam_member | | [main.tf](./main.tf) | Module-level locals and resources. | google_access_context_manager_access_policy | | [outputs.tf](./outputs.tf) | Module outputs. | | | [service-perimeters-bridge.tf](./service-perimeters-bridge.tf) | Bridge service perimeter resources. | google_access_context_manager_service_perimeter | @@ -205,9 +225,12 @@ module "test" { | [access_levels](variables.tf#L17) | Access level definitions. | map(object({…})) | | {} | | [access_policy_create](variables.tf#L61) | Access Policy configuration, fill in to create. Parent is in 'organizations/123456' format, scopes are in 'folders/456789' or 'projects/project_id' format. | object({…}) | | null | | [egress_policies](variables.tf#L71) | Egress policy definitions that can be referenced in perimeters. | map(object({…})) | | {} | -| [ingress_policies](variables.tf#L102) | Ingress policy definitions that can be referenced in perimeters. | map(object({…})) | | {} | -| [service_perimeters_bridge](variables.tf#L134) | Bridge service perimeters. | map(object({…})) | | {} | -| [service_perimeters_regular](variables.tf#L144) | Regular service perimeters. | map(object({…})) | | {} | +| [iam](variables.tf#L102) | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | +| [iam_bindings](variables.tf#L108) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | map(object({…})) | | {} | +| [iam_bindings_additive](variables.tf#L123) | Individual additive IAM bindings. Keys are arbitrary. | map(object({…})) | | {} | +| [ingress_policies](variables.tf#L138) | Ingress policy definitions that can be referenced in perimeters. | map(object({…})) | | {} | +| [service_perimeters_bridge](variables.tf#L170) | Bridge service perimeters. | map(object({…})) | | {} | +| [service_perimeters_regular](variables.tf#L180) | Regular service perimeters. | map(object({…})) | | {} | ## Outputs diff --git a/modules/vpc-sc/iam.tf b/modules/vpc-sc/iam.tf new file mode 100644 index 0000000000..9b475b68fd --- /dev/null +++ b/modules/vpc-sc/iam.tf @@ -0,0 +1,54 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description IAM bindings + +resource "google_access_context_manager_access_policy_iam_binding" "authoritative" { + for_each = var.iam + name = local.access_policy + role = each.key + members = each.value +} + +resource "google_access_context_manager_access_policy_iam_binding" "bindings" { + for_each = var.iam_bindings + name = local.access_policy + role = each.value.role + members = each.value.members + dynamic "condition" { + for_each = each.value.condition == null ? [] : [""] + content { + expression = each.value.condition.expression + title = each.value.condition.title + description = each.value.condition.description + } + } +} + +resource "google_access_context_manager_access_policy_iam_member" "bindings" { + for_each = var.iam_bindings_additive + name = local.access_policy + role = each.value.role + member = each.value.member + dynamic "condition" { + for_each = each.value.condition == null ? [] : [""] + content { + expression = each.value.condition.expression + title = each.value.condition.title + description = each.value.condition.description + } + } +} diff --git a/modules/vpc-sc/variables.tf b/modules/vpc-sc/variables.tf index 8ce4b41eb1..008d7ae215 100644 --- a/modules/vpc-sc/variables.tf +++ b/modules/vpc-sc/variables.tf @@ -99,6 +99,42 @@ variable "egress_policies" { } } +variable "iam" { + description = "IAM bindings in {ROLE => [MEMBERS]} format." + type = map(list(string)) + default = {} +} + +variable "iam_bindings" { + description = "Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary." + type = map(object({ + members = list(string) + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })) + nullable = false + default = {} +} + +variable "iam_bindings_additive" { + description = "Individual additive IAM bindings. Keys are arbitrary." + type = map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })) + nullable = false + default = {} +} + variable "ingress_policies" { description = "Ingress policy definitions that can be referenced in perimeters." type = map(object({ diff --git a/modules/vpc-sc/versions.tf b/modules/vpc-sc/versions.tf index af3463955f..3adb51d3bd 100644 --- a/modules/vpc-sc/versions.tf +++ b/modules/vpc-sc/versions.tf @@ -17,11 +17,11 @@ terraform { required_providers { google = { source = "hashicorp/google" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } google-beta = { source = "hashicorp/google-beta" - version = ">= 5.0.0, < 6.0.0" # tftest + version = ">= 5.4.0, < 6.0.0" # tftest } } } diff --git a/tests/blueprints/factories/project_factory/examples/example.yaml b/tests/blueprints/factories/project_factory/examples/example.yaml index 859ec2ee29..71391a526a 100644 --- a/tests/blueprints/factories/project_factory/examples/example.yaml +++ b/tests/blueprints/factories/project_factory/examples/example.yaml @@ -30,6 +30,10 @@ values: module.project-factory.module.projects["prj-app-1"].google_project.project[0]: auto_create_network: false billing_account: 012345-67890A-BCDEF0 + effective_labels: + app: app-1 + environment: test + team: foo folder_id: '12345678' labels: app: app-1 @@ -39,6 +43,16 @@ values: org_id: null project_id: test-pf-prj-app-1 skip_delete: false + terraform_labels: + app: app-1 + environment: test + team: foo + timeouts: null + module.project-factory.module.projects["prj-app-1"].google_project_service.project_services["container.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-1 + service: container.googleapis.com timeouts: null module.project-factory.module.projects["prj-app-1"].google_project_service.project_services["stackdriver.googleapis.com"]: disable_dependent_services: false @@ -52,6 +66,14 @@ values: project: test-pf-prj-app-1 service: storage.googleapis.com timeouts: null + module.project-factory.module.projects["prj-app-2"].data.google_storage_project_service_account.gcs_sa[0]: + project: test-pf-prj-app-2 + user_project: null + module.project-factory.module.projects["prj-app-2"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: foo-host + service_project: test-pf-prj-app-2 + timeouts: null module.project-factory.module.projects["prj-app-2"].google_essential_contacts_contact.contact["admin@example.com"]: email: admin@example.com language_tag: en @@ -59,44 +81,38 @@ values: - ALL parent: projects/test-pf-prj-app-2 timeouts: null - ? module.project-factory.module.projects["prj-app-2"].google_org_policy_policy.default["compute.disableGuestAttributesAccess"] - : name: projects/test-pf-prj-app-2/policies/compute.disableGuestAttributesAccess - parent: projects/test-pf-prj-app-2 - spec: - - inherit_from_parent: null - reset: null - rules: - - allow_all: null - condition: [] - deny_all: null - enforce: 'FALSE' - values: [] - timeouts: null - ? module.project-factory.module.projects["prj-app-2"].google_org_policy_policy.default["iam.disableServiceAccountKeyCreation"] - : name: projects/test-pf-prj-app-2/policies/iam.disableServiceAccountKeyCreation - parent: projects/test-pf-prj-app-2 - spec: - - inherit_from_parent: null - reset: null - rules: - - allow_all: null - condition: [] - deny_all: null - enforce: 'FALSE' - values: [] - timeouts: null module.project-factory.module.projects["prj-app-2"].google_project.project[0]: auto_create_network: false billing_account: 012345-67890A-ABCDEF + effective_labels: + app: app-2 + environment: test + team: foo folder_id: '12345678' labels: - app: app-1 + app: app-2 environment: test team: foo name: test-pf-prj-app-2 org_id: null project_id: test-pf-prj-app-2 skip_delete: false + terraform_labels: + app: app-2 + environment: test + team: foo + timeouts: null + module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["compute.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-2 + service: compute.googleapis.com + timeouts: null + module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["run.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-2 + service: run.googleapis.com timeouts: null module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["stackdriver.googleapis.com"]: disable_dependent_services: false @@ -104,37 +120,95 @@ values: project: test-pf-prj-app-2 service: stackdriver.googleapis.com timeouts: null + module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["storage.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-2 + service: storage.googleapis.com + timeouts: null + module.project-factory.module.projects["prj-app-3"].data.google_storage_project_service_account.gcs_sa[0]: + project: test-pf-prj-app-3 + user_project: null + module.project-factory.module.projects["prj-app-3"].google_essential_contacts_contact.contact["admin@example.com"]: + email: admin@example.com + language_tag: en + notification_category_subscriptions: + - ALL + parent: projects/test-pf-prj-app-3 + timeouts: null + module.project-factory.module.projects["prj-app-3"].google_project.project[0]: + auto_create_network: false + billing_account: 012345-67890A-ABCDEF + effective_labels: + environment: test + folder_id: '12345678' + labels: + environment: test + name: test-pf-prj-app-3 + org_id: null + project_id: test-pf-prj-app-3 + skip_delete: false + terraform_labels: + environment: test + timeouts: null + module.project-factory.module.projects["prj-app-3"].google_project_service.project_services["run.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-3 + service: run.googleapis.com + timeouts: null + module.project-factory.module.projects["prj-app-3"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-3 + service: stackdriver.googleapis.com + timeouts: null + module.project-factory.module.projects["prj-app-3"].google_project_service.project_services["storage.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-prj-app-3 + service: storage.googleapis.com + timeouts: null + ? module.project-factory.module.service-accounts["prj-app-1-app-1-be"].google_project_iam_member.project-roles["test-pf-prj-app-1-roles/logging.logWriter"] + : condition: [] + project: test-pf-prj-app-1 + role: roles/logging.logWriter + ? module.project-factory.module.service-accounts["prj-app-1-app-1-be"].google_project_iam_member.project-roles["test-pf-prj-app-1-roles/monitoring.metricWriter"] + : condition: [] + project: test-pf-prj-app-1 + role: roles/monitoring.metricWriter module.project-factory.module.service-accounts["prj-app-1-app-1-be"].google_service_account.service_account[0]: account_id: app-1-be description: null disabled: false - display_name: Terraform-managed. + display_name: null project: test-pf-prj-app-1 timeouts: null module.project-factory.module.service-accounts["prj-app-1-app-1-fe"].google_service_account.service_account[0]: account_id: app-1-fe description: null disabled: false - display_name: Terraform-managed. + display_name: Test app 1 frontend. project: test-pf-prj-app-1 timeouts: null module.project-factory.module.service-accounts["prj-app-2-app-2-be"].google_service_account.service_account[0]: account_id: app-2-be description: null disabled: false - display_name: Terraform-managed. + display_name: null project: test-pf-prj-app-2 timeouts: null counts: - google_essential_contacts_contact: 2 + google_compute_shared_vpc_service_project: 1 + google_essential_contacts_contact: 3 google_kms_crypto_key_iam_member: 1 - google_org_policy_policy: 2 - google_project: 2 - google_project_service: 3 + google_project: 3 + google_project_iam_member: 2 + google_project_service: 10 google_service_account: 3 - google_storage_project_service_account: 1 - modules: 6 - resources: 14 + google_storage_project_service_account: 3 + modules: 7 + resources: 26 outputs: {} diff --git a/tests/examples/test_plan.py b/tests/examples/test_plan.py index 3ff2992ed5..22525698ed 100644 --- a/tests/examples/test_plan.py +++ b/tests/examples/test_plan.py @@ -18,9 +18,18 @@ from pathlib import Path BASE_PATH = Path(__file__).parent -COUNT_TEST_RE = re.compile(r'# tftest +modules=(\d+) +resources=(\d+)' + - r'(?: +files=([\w@,_-]+))?' + - r'(?: +inventory=([\w\-.]+))?') +COUNT_TEST_RE = re.compile(r'# tftest +modules=(?P\d+) +resources=(?P\d+)' + + r'(?: +files=(?P[\w@,_-]+))?' + + r'(?: +inventory=(?P[\w\-.]+))?') + + +def prepare_files(example, test_path, line): + if line is not None: + requested_files = line.split(',') + for f in requested_files: + destination = test_path / example.files[f].path + destination.parent.mkdir(parents=True, exist_ok=True) + destination.write_text(example.files[f].content) def test_example(plan_validator, tmp_path, example): @@ -33,21 +42,16 @@ def test_example(plan_validator, tmp_path, example): if assets_path.exists(): (tmp_path / 'assets').symlink_to(assets_path) - expected_modules = int(match.group(1)) - expected_resources = int(match.group(2)) + expected_modules = int(match.group("modules")) + expected_resources = int(match.group("resources")) - if match.group(3) is not None: - requested_files = match.group(3).split(',') - for f in requested_files: - destination = tmp_path / example.files[f].path - destination.parent.mkdir(parents=True, exist_ok=True) - destination.write_text(example.files[f].content) + prepare_files(example, tmp_path, match.group("files")) inventory = [] - if match.group(4) is not None: + if match.group("inventory") is not None: python_test_path = str(example.module).replace('-', '_') inventory = BASE_PATH.parent / python_test_path / 'examples' - inventory = inventory / match.group(4) + inventory = inventory / match.group("inventory") # TODO: force plan_validator to never copy files (we're already # running from a temp dir) diff --git a/tests/examples/variables.tf b/tests/examples/variables.tf index 9a65aa7a52..ebdbb155cb 100644 --- a/tests/examples/variables.tf +++ b/tests/examples/variables.tf @@ -22,9 +22,13 @@ variable "billing_account_id" { default = "123456-123456-123456" } +variable "group_email" { + default = "organization-admins@example.org" +} + variable "kms_key" { default = { - self_link = "kms_key_self_link" + id = "kms_key_self_link" } } diff --git a/tests/examples_e2e/README.md b/tests/examples_e2e/README.md index 027ef21c8a..bab4337288 100644 --- a/tests/examples_e2e/README.md +++ b/tests/examples_e2e/README.md @@ -9,7 +9,7 @@ Prepare following information: * prepare service account that has necessary permissions (able to assign billing account to project, resource creation etc) # How does it work -Each test case is provided by additional environment defined in [variables.tf](./variables.tf). This simplifies writing the examples as this follows the same structure as for non-end-to-end tests, and allows multiple, independent and concurrent runs of tests. +Each test case is provided by additional environment defined in [variables.tf](../examples/variables.tf). This simplifies writing the examples as this follows the same structure as for non-end-to-end tests, and allows multiple, independent and concurrent runs of tests. The test environment can be provisioned automatically during the test run (which now takes ~2 minutes) and destroyed and the end, when of the tests (Option 1 below), which is targeting automated runs in CI/CD pipeline, or can be provisioned manually to reduce test time, which might be typical use case for tests run locally. @@ -18,6 +18,7 @@ The test environment can be provisioned automatically during the test run (which ## Create `e2e.tfvars` file ```hcl billing_account = "123456-123456-123456" # billing account id to associate projects +group_email = "group@example.org" # existing group within organization organization_id = "1234567890" # your organization id parent = "folders/1234567890" # folder under which test resources will be created prefix = "your-unique-prefix" # unique prefix for projects @@ -27,15 +28,17 @@ region = "europe-west4" # region to use ``` And set environment variable pointing to the file: ```bash +export TF_VAR_prefix="your-unique-prefix" # unique prefix for projects, no longer than 7 characters export TFTEST_E2E_SETUP_TFVARS_PATH= ``` Or set above variables in environment: ```bash export TF_VAR_billing_account="123456-123456-123456" # billing account id to associate projects +export TF_VAR_group_email="group@example.org" # existing group within organization export TF_VAR_organization_id="1234567890" # your organization id export TF_VAR_parent="folders/1234567890" # folder under which test resources will be created -export TF_VAR_prefix="your-unique-prefix" # unique prefix for projects +export TF_VAR_prefix="your-unique-prefix" # unique prefix for projects, no longer than 7 characters export TF_VAR_region="europe-west4" # region to use ``` @@ -55,6 +58,7 @@ pytest tests/examples_e2e In `tests/examples_e2e/setup_module` create `terraform.tfvars` with following values: ```hcl billing_account = "123456-123456-123456" # billing account id to associate projects +group_email = "group@example.org" # existing group within organization organization_id = "1234567890" # your organization id parent = "folders/1234567890" # folder under which test resources will be created prefix = "your-unique-prefix" # unique prefix for projects @@ -79,6 +83,7 @@ This will generate also `tests/examples_e2e/setup_module/e2e_tests.tfvars` for y ## Setup your environment ```bash export TFTEST_E2E_TFVARS_PATH=`pwd`/tests/examples_e2e/setup_module/e2e_tests.tfvars # generated above +export TF_VAR_prefix="your-unique-prefix" # unique prefix for projects, no longer than 7 characters ``` ## Run tests diff --git a/tests/examples_e2e/setup_module/e2e_tests.tfvars.tftpl b/tests/examples_e2e/setup_module/e2e_tests.tfvars.tftpl index c21d77422a..31a34d3df5 100644 --- a/tests/examples_e2e/setup_module/e2e_tests.tfvars.tftpl +++ b/tests/examples_e2e/setup_module/e2e_tests.tfvars.tftpl @@ -12,14 +12,14 @@ # See the License for the specific language governing permissions and # limitations under the License. -bucket= "${bucket}" +bucket = "${bucket}" billing_account_id = "${billing_account_id}" -kms_key= { - self_link = "kms_key_self_link" +kms_key = { + id = "${kms_key_id}" } +group_email = "${group_email}" organization_id = "${organization_id}" -folder_id = "${folder_id}" -prefix = "${prefix}" +folder_id = "folders/${folder_id}" project_id = "${project_id}" region = "${region}" service_account = { diff --git a/tests/examples_e2e/setup_module/main.tf b/tests/examples_e2e/setup_module/main.tf index 9bc5a9799c..e781d779bc 100644 --- a/tests/examples_e2e/setup_module/main.tf +++ b/tests/examples_e2e/setup_module/main.tf @@ -13,11 +13,14 @@ # limitations under the License. locals { - prefix = "${var.prefix}-${var.timestamp}-${var.suffix}" + prefix = "${var.prefix}-${var.timestamp}${var.suffix}" services = [ # trimmed down list of services, to be extended as needed + "apigee.googleapis.com", + "bigquery.googleapis.com", "cloudbuild.googleapis.com", "cloudfunctions.googleapis.com", + "cloudkms.googleapis.com", "cloudresourcemanager.googleapis.com", "compute.googleapis.com", "iam.googleapis.com", @@ -77,14 +80,28 @@ resource "google_service_account" "service_account" { depends_on = [google_project_service.project_service] } +resource "google_kms_key_ring" "keyring" { + name = "keyring" + project = google_project.project.project_id + location = var.region + depends_on = [google_project_service.project_service] +} + +resource "google_kms_crypto_key" "key" { + name = "crypto-key-example" + key_ring = google_kms_key_ring.keyring.id + rotation_period = "100000s" +} + resource "local_file" "terraform_tfvars" { filename = "e2e_tests.tfvars" content = templatefile("e2e_tests.tfvars.tftpl", { bucket = google_storage_bucket.bucket.name billing_account_id = var.billing_account - organization_id = var.organization_id folder_id = google_folder.folder.folder_id - prefix = local.prefix + group_email = var.group_email + kms_key_id = google_kms_crypto_key.key.id + organization_id = var.organization_id project_id = google_project.project.project_id region = var.region service_account = { diff --git a/tests/examples_e2e/setup_module/variables.tf b/tests/examples_e2e/setup_module/variables.tf index d936cb2080..16f110df0a 100644 --- a/tests/examples_e2e/setup_module/variables.tf +++ b/tests/examples_e2e/setup_module/variables.tf @@ -15,6 +15,9 @@ variable "billing_account" { type = string } +variable "group_email" { + type = string +} variable "organization_id" { type = string } diff --git a/tests/examples_e2e/test_plan.py b/tests/examples_e2e/test_plan.py index 84579b6b41..c9936b9bbf 100644 --- a/tests/examples_e2e/test_plan.py +++ b/tests/examples_e2e/test_plan.py @@ -14,16 +14,14 @@ import re from pathlib import Path +from ..examples.test_plan import COUNT_TEST_RE, prepare_files BASE_PATH = Path(__file__).parent -COUNT_TEST_RE = re.compile(r'# tftest +modules=(\d+) +resources=(\d+)' + - r'(?: +files=([\w@,_-]+))?' + - r'(?: +inventory=([\w\-.]+))?') def test_example(e2e_validator, tmp_path, examples_e2e, e2e_tfvars_path): (tmp_path / 'fabric').symlink_to(BASE_PATH.parents[1]) - (tmp_path / 'variables.tf').symlink_to(BASE_PATH / 'variables.tf') + (tmp_path / 'variables.tf').symlink_to(BASE_PATH.parent / 'examples' / 'variables.tf') (tmp_path / 'main.tf').write_text(examples_e2e.code) assets_path = BASE_PATH.parent / str(examples_e2e.module).replace( '-', '_') / 'assets' @@ -31,5 +29,9 @@ def test_example(e2e_validator, tmp_path, examples_e2e, e2e_tfvars_path): (tmp_path / 'assets').symlink_to(assets_path) (tmp_path / 'terraform.tfvars').symlink_to(e2e_tfvars_path) + # add files the same way as it is done for examples + if match := COUNT_TEST_RE.search(examples_e2e.code): + prepare_files(examples_e2e, tmp_path, match.group("files")) + e2e_validator(module_path=tmp_path, extra_files=[], tf_var_files=[(tmp_path / 'terraform.tfvars')]) diff --git a/tests/examples_e2e/variables.tf b/tests/examples_e2e/variables.tf deleted file mode 100644 index 9a65aa7a52..0000000000 --- a/tests/examples_e2e/variables.tf +++ /dev/null @@ -1,92 +0,0 @@ -# Copyright 2023 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# common variables used for examples - -variable "bucket" { - default = "bucket" -} - -variable "billing_account_id" { - default = "123456-123456-123456" -} - -variable "kms_key" { - default = { - self_link = "kms_key_self_link" - } -} - -variable "organization_id" { - default = "organizations/1122334455" -} - -variable "folder_id" { - default = "folders/1122334455" -} - -variable "prefix" { - default = "test" -} - -variable "project_id" { - default = "project-id" -} - -variable "region" { - default = "region" -} - -variable "service_account" { - default = { - id = "service_account_id" - email = "service_account_email" - iam_email = "service_account_iam_email" - } -} - -variable "subnet" { - default = { - name = "subnet_name" - region = "subnet_region" - cidr = "subnet_cidr" - self_link = "subnet_self_link" - } -} - -variable "vpc" { - default = { - name = "vpc_name" - self_link = "projects/xxx/global/networks/aaa" - id = "projects/xxx/global/networks/aaa" - } -} - -variable "vpc1" { - default = { - name = "vpc_name" - self_link = "projects/xxx/global/networks/bbb" - } -} - -variable "vpc2" { - default = { - name = "vpc2_name" - self_link = "projects/xxx/global/networks/ccc" - } -} - -variable "zone" { - default = "zone" -} diff --git a/tests/fixtures.py b/tests/fixtures.py index caa9c9456f..01a89f423c 100644 --- a/tests/fixtures.py +++ b/tests/fixtures.py @@ -298,9 +298,12 @@ def e2e_validator(module_path, extra_files, tf_var_files, basedir=None): tf.setup(extra_files=extra_files, upgrade=True) tf_var_files = [(basedir / x).resolve() for x in tf_var_files or []] + # to allow different tests to create projects (or other globally unique resources) with the same name + # bump prefix forward on each test execution + prefix = f'{os.environ.get("TF_VAR_prefix")}-{int(time.time())}{os.environ.get("PYTEST_XDIST_WORKER", "0")[-2:]}' try: - apply = tf.apply(tf_var_file=tf_var_files) - plan = tf.plan(output=True, tf_var_file=tf_var_files) + apply = tf.apply(tf_var_file=tf_var_files, tf_vars={"prefix": prefix}) + plan = tf.plan(output=True, tf_var_file=tf_var_files, tf_vars={"prefix": prefix}) changes = {} for resource_name, value in plan.resource_changes.items(): if value.get('change', {}).get('actions') != ['no-op']: @@ -324,7 +327,7 @@ def e2e_validator(module_path, extra_files, tf_var_files, basedir=None): # If above did not fail, this should not either, but left as a safety check assert changes == {}, f'Plan not empty for following resources: {", ".join(changes.keys())}' finally: - destroy = tf.destroy(tf_var_file=tf_var_files) + destroy = tf.destroy(tf_var_file=tf_var_files, tf_vars={'prefix': prefix}) @pytest.fixture(name='e2e_validator') @@ -371,7 +374,7 @@ def e2e_tfvars_path(): tf = tftest.TerraformTest(test_path, binary=binary) tf_vars_file = None tf_vars = { - 'suffix': os.environ.get("PYTEST_XDIST_WORKER", "0"), + 'suffix': os.environ.get("PYTEST_XDIST_WORKER", "0")[-2:], # take at most 2 last chars for suffix 'timestamp': str(int(time.time())) } if 'TFTEST_E2E_SETUP_TFVARS_PATH' in os.environ: diff --git a/tests/modules/cloudsql_instance/examples/insights.yaml b/tests/modules/cloudsql_instance/examples/insights.yaml index 1c54188352..84a4245ddf 100644 --- a/tests/modules/cloudsql_instance/examples/insights.yaml +++ b/tests/modules/cloudsql_instance/examples/insights.yaml @@ -20,18 +20,10 @@ values: region: europe-west1 settings: - activation_policy: ALWAYS - active_directory_config: [] - advanced_machine_features: [] availability_type: ZONAL - collation: null - data_cache_config: [] - database_flags: [] deletion_protection_enabled: true - deny_maintenance_period: [] disk_autoresize: true - disk_autoresize_limit: 0 disk_type: PD_SSD - edition: null insights_config: - query_insights_enabled: true query_plans_per_minute: 5 @@ -41,17 +33,9 @@ values: ip_configuration: - allocated_ip_range: null authorized_networks: [] - enable_private_path_for_google_cloud_services: null ipv4_enabled: false private_network: projects/xxx/global/networks/aaa - psc_config: [] - require_ssl: null - maintenance_window: [] - password_validation_policy: [] - pricing_plan: PER_USE - sql_server_audit_config: [] tier: db-g1-small - time_zone: null counts: google_sql_database_instance: 1 diff --git a/tests/modules/cloudsql_instance/examples/public-ip.yaml b/tests/modules/cloudsql_instance/examples/public-ip.yaml index 9ba4622033..f3ca86d3a5 100644 --- a/tests/modules/cloudsql_instance/examples/public-ip.yaml +++ b/tests/modules/cloudsql_instance/examples/public-ip.yaml @@ -18,46 +18,19 @@ values: name: db project: project-id region: europe-west1 - restore_backup_context: [] settings: - activation_policy: ALWAYS - active_directory_config: [] - advanced_machine_features: [] availability_type: ZONAL - backup_configuration: - - backup_retention_settings: - - retained_backups: 7 - retention_unit: COUNT - binary_log_enabled: true - enabled: true - location: null - point_in_time_recovery_enabled: null - start_time: '23:00' - transaction_log_retention_days: 7 - collation: null - data_cache_config: [] - database_flags: [] deletion_protection_enabled: true - deny_maintenance_period: [] disk_autoresize: true - disk_autoresize_limit: 0 disk_type: PD_SSD - edition: null insights_config: [] ip_configuration: - allocated_ip_range: null authorized_networks: [] - enable_private_path_for_google_cloud_services: null ipv4_enabled: true private_network: projects/xxx/global/networks/aaa - psc_config: [] - require_ssl: null - maintenance_window: [] - password_validation_policy: [] - pricing_plan: PER_USE - sql_server_audit_config: [] tier: db-g1-small - time_zone: null module.db.google_sql_database_instance.replicas["replica1"]: database_version: MYSQL_8_0 master_instance_name: db @@ -66,34 +39,17 @@ values: region: europe-west3 settings: - activation_policy: ALWAYS - active_directory_config: [] - advanced_machine_features: [] availability_type: ZONAL - collation: null - data_cache_config: [] - database_flags: [] deletion_protection_enabled: true - deny_maintenance_period: [] disk_autoresize: true - disk_autoresize_limit: 0 disk_type: PD_SSD - edition: null insights_config: [] ip_configuration: - allocated_ip_range: null authorized_networks: [] - enable_private_path_for_google_cloud_services: null ipv4_enabled: true - psc_config: [] private_network: projects/xxx/global/networks/aaa - require_ssl: null - maintenance_window: [] - password_validation_policy: [] - pricing_plan: PER_USE - sql_server_audit_config: [] tier: db-g1-small - time_zone: null - timeouts: null counts: google_sql_database_instance: 2 diff --git a/tests/modules/cloudsql_instance/examples/simple.yaml b/tests/modules/cloudsql_instance/examples/simple.yaml index d4c0558d9c..fdc663e219 100644 --- a/tests/modules/cloudsql_instance/examples/simple.yaml +++ b/tests/modules/cloudsql_instance/examples/simple.yaml @@ -14,39 +14,22 @@ values: module.db.google_sql_database_instance.primary: - clone: [] database_version: POSTGRES_13 name: db project: my-db-project region: europe-west1 settings: - activation_policy: ALWAYS - active_directory_config: [] - advanced_machine_features: [] availability_type: ZONAL - collation: null - data_cache_config: [] - database_flags: [] deletion_protection_enabled: true - deny_maintenance_period: [] disk_autoresize: true - disk_autoresize_limit: 0 disk_type: PD_SSD - edition: null insights_config: [] ip_configuration: - allocated_ip_range: null authorized_networks: [] - enable_private_path_for_google_cloud_services: null ipv4_enabled: false - psc_config: [] - require_ssl: null - maintenance_window: [] - password_validation_policy: [] - pricing_plan: PER_USE - sql_server_audit_config: [] tier: db-g1-small - time_zone: null counts: google_sql_database_instance: 1 diff --git a/tests/modules/project/examples/basic.yaml b/tests/modules/project/examples/basic.yaml index 56e6ecc02f..919a0581ee 100644 --- a/tests/modules/project/examples/basic.yaml +++ b/tests/modules/project/examples/basic.yaml @@ -17,21 +17,21 @@ values: module.project.google_project.project[0]: auto_create_network: false billing_account: 123456-123456-123456 - folder_id: '1234567890' + folder_id: '1122334455' labels: null - name: foo-myproject + name: test-project org_id: null - project_id: foo-myproject + project_id: test-project skip_delete: false module.project.google_project_service.project_services["container.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: foo-myproject + project: test-project service: container.googleapis.com module.project.google_project_service.project_services["stackdriver.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: foo-myproject + project: test-project service: stackdriver.googleapis.com counts: diff --git a/tests/modules/project/examples/data.yaml b/tests/modules/project/examples/data.yaml new file mode 100644 index 0000000000..b7ec3d23aa --- /dev/null +++ b/tests/modules/project/examples/data.yaml @@ -0,0 +1,338 @@ +# Copyright 2023 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +values: + module.bucket.google_logging_project_bucket_config.bucket[0]: + bucket_id: bucket + project: project-id + module.create-project.google_project.project[0]: + billing_account: 123456-123456-123456 + folder_id: '1122334455' + name: test-project + project_id: test-project + module.dataset.google_bigquery_dataset.default: + dataset_id: bq_sink + delete_contents_on_destroy: true + project: project-id + module.gcs.google_storage_bucket.bucket: + name: test-gcs_sink + project: project-id + module.host-project.google_compute_shared_vpc_host_project.shared_vpc_host[0]: + project: test-host + module.host-project.google_project.project[0]: + billing_account: 123456-123456-123456 + folder_id: '1122334455' + name: test-host + project_id: test-host + module.project.data.google_bigquery_default_service_account.bq_sa[0]: + project: test-project + module.project.data.google_project.project[0]: + project_id: test-project + module.project.data.google_storage_project_service_account.gcs_sa[0]: + project: test-project + module.project.google_bigquery_dataset_iam_member.bq-sinks-binding["info"]: + role: roles/bigquery.dataEditor + module.project.google_compute_shared_vpc_service_project.shared_vpc_service[0]: + host_project: test-host + service_project: test-project + module.project.google_kms_crypto_key_iam_member.service_identity_cmek["compute.kms_key_self_link"]: + crypto_key_id: kms_key_self_link + role: roles/cloudkms.cryptoKeyEncrypterDecrypter + module.project.google_kms_crypto_key_iam_member.service_identity_cmek["storage.kms_key_self_link"]: + crypto_key_id: kms_key_self_link + role: roles/cloudkms.cryptoKeyEncrypterDecrypter + module.project.google_logging_project_exclusion.logging-exclusion["no-gce-instances"]: + filter: resource.type=gce_instance + name: no-gce-instances + project: test-project + module.project.google_logging_project_sink.sink["debug"]: + exclusions: + - description: null + disabled: false + filter: logName:compute + name: no-compute + filter: severity=DEBUG + name: debug + project: test-project + module.project.google_logging_project_sink.sink["info"]: + exclusions: [] + filter: severity=INFO + name: info + project: test-project + module.project.google_logging_project_sink.sink["notice"]: + destination: pubsub.googleapis.com/projects/project-id/topics/pubsub_sink + disabled: false + exclusions: [] + filter: severity=NOTICE + name: notice + project: test-project + module.project.google_logging_project_sink.sink["warnings"]: + destination: storage.googleapis.com/test-gcs_sink + disabled: false + exclusions: [] + filter: severity=WARNING + name: warnings + project: test-project + module.project.google_org_policy_policy.default["compute.disableGuestAttributesAccess"]: + name: projects/test-project/policies/compute.disableGuestAttributesAccess + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: [] + deny_all: null + enforce: 'TRUE' + values: [] + module.project.google_org_policy_policy.default["compute.skipDefaultNetworkCreation"]: + name: projects/test-project/policies/compute.skipDefaultNetworkCreation + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: [] + deny_all: null + enforce: 'TRUE' + values: [] + module.project.google_org_policy_policy.default["compute.trustedImageProjects"]: + name: projects/test-project/policies/compute.trustedImageProjects + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: [] + deny_all: null + enforce: null + values: + - allowed_values: + - projects/my-project + denied_values: null + module.project.google_org_policy_policy.default["compute.vmExternalIpAccess"]: + name: projects/test-project/policies/compute.vmExternalIpAccess + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: [] + deny_all: 'TRUE' + enforce: null + values: [] + module.project.google_org_policy_policy.default["iam.allowedPolicyMemberDomains"]: + name: projects/test-project/policies/iam.allowedPolicyMemberDomains + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: [] + deny_all: null + enforce: null + values: + - allowed_values: + - C0xxxxxxx + - C0yyyyyyy + denied_values: null + module.project.google_org_policy_policy.default["iam.disableServiceAccountKeyCreation"]: + name: projects/test-project/policies/iam.disableServiceAccountKeyCreation + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: [] + deny_all: null + enforce: 'TRUE' + values: [] + module.project.google_org_policy_policy.default["iam.disableServiceAccountKeyUpload"]: + name: projects/test-project/policies/iam.disableServiceAccountKeyUpload + parent: projects/test-project + spec: + - inherit_from_parent: null + rules: + - allow_all: null + condition: + - description: test condition + expression: resource.matchTagId('tagKeys/1234', 'tagValues/1234') + location: somewhere + title: condition + deny_all: null + enforce: 'TRUE' + values: [] + - allow_all: null + condition: [] + deny_all: null + enforce: 'FALSE' + values: [] + module.project.google_project_iam_audit_config.default["allServices"]: + audit_log_config: + - exempted_members: + - group:organization-admins@example.org + log_type: ADMIN_READ + project: test-project + service: allServices + module.project.google_project_iam_audit_config.default["storage.googleapis.com"]: + audit_log_config: + - exempted_members: [] + log_type: DATA_READ + - exempted_members: [] + log_type: DATA_WRITE + project: test-project + service: storage.googleapis.com + module.project.google_project_iam_binding.authoritative["roles/apigee.serviceAgent"]: + condition: [] + project: test-project + role: roles/apigee.serviceAgent + module.project.google_project_iam_binding.authoritative["roles/cloudasset.owner"]: + condition: [] + project: test-project + role: roles/cloudasset.owner + module.project.google_project_iam_binding.authoritative["roles/cloudsupport.techSupportEditor"]: + condition: [] + project: test-project + role: roles/cloudsupport.techSupportEditor + module.project.google_project_iam_binding.authoritative["roles/editor"]: + condition: [] + project: test-project + role: roles/editor + module.project.google_project_iam_binding.authoritative["roles/iam.securityReviewer"]: + condition: [] + project: test-project + role: roles/iam.securityReviewer + module.project.google_project_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + project: test-project + role: roles/logging.admin + module.project.google_project_iam_binding.bindings["iam_admin_conditional"]: + condition: + - description: null + expression: "api.getAttribute(\n 'iam.googleapis.com/modifiedGrantsByRole',\ + \ []\n).hasOnly([\n 'roles/compute.networkAdmin'\n])\n" + title: delegated_network_user_one + members: + - group:organization-admins@example.org + project: test-project + role: roles/resourcemanager.projectIamAdmin + module.project.google_project_iam_member.bindings["group-owner"]: + condition: [] + member: group:organization-admins@example.org + project: test-project + role: roles/owner + module.project.google_project_iam_member.bucket-sinks-binding["debug"]: + condition: + - title: debug bucket writer + role: roles/logging.bucketWriter + module.project.google_project_iam_member.shared_vpc_host_robots["roles/cloudasset.owner:cloudservices"]: + condition: [] + project: test-host + role: roles/cloudasset.owner + module.project.google_project_iam_member.shared_vpc_host_robots["roles/cloudasset.owner:container-engine"]: + condition: [] + project: test-host + role: roles/cloudasset.owner + module.project.google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:cloudservices"]: + condition: [] + project: test-host + role: roles/compute.networkUser + module.project.google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:container"]: + condition: [] + project: test-host + role: roles/compute.networkUser + module.project.google_project_iam_member.shared_vpc_host_robots["roles/compute.securityAdmin:container"]: + condition: [] + project: test-host + role: roles/compute.securityAdmin + module.project.google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container"]: + condition: [] + project: test-host + role: roles/container.hostServiceAgentUser + module.project.google_project_iam_member.shared_vpc_host_robots["roles/vpcaccess.user:run"]: + condition: [] + project: test-host + role: roles/vpcaccess.user + module.project.google_project_service.project_services["apigee.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-project + service: apigee.googleapis.com + module.project.google_project_service.project_services["bigquery.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-project + service: bigquery.googleapis.com + module.project.google_project_service.project_services["container.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-project + service: container.googleapis.com + module.project.google_project_service.project_services["logging.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-project + service: logging.googleapis.com + module.project.google_project_service.project_services["run.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-project + service: run.googleapis.com + module.project.google_project_service.project_services["storage.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-project + service: storage.googleapis.com + module.project.google_project_service_identity.jit_si["apigee.googleapis.com"]: + project: test-project + service: apigee.googleapis.com + module.project.google_pubsub_topic_iam_member.pubsub-sinks-binding["notice"]: + condition: [] + project: project-id + role: roles/pubsub.publisher + topic: pubsub_sink + module.project.google_storage_bucket_iam_member.gcs-sinks-binding["warnings"]: + bucket: test-gcs_sink + condition: [] + role: roles/storage.objectCreator + module.pubsub.google_pubsub_topic.default: + name: pubsub_sink + project: project-id + +counts: + google_bigquery_dataset: 1 + google_bigquery_dataset_iam_member: 1 + google_bigquery_default_service_account: 1 + google_compute_shared_vpc_host_project: 1 + google_compute_shared_vpc_service_project: 1 + google_kms_crypto_key_iam_member: 2 + google_logging_project_bucket_config: 1 + google_logging_project_exclusion: 1 + google_logging_project_sink: 4 + google_org_policy_policy: 7 + google_project: 3 + google_project_iam_audit_config: 2 + google_project_iam_binding: 7 + google_project_iam_member: 9 + google_project_service: 6 + google_project_service_identity: 1 + google_pubsub_topic: 1 + google_pubsub_topic_iam_member: 1 + google_storage_bucket: 1 + google_storage_bucket_iam_member: 1 + google_storage_project_service_account: 1 + modules: 7 + resources: 53 + +outputs: {} diff --git a/tests/modules/project/examples/iam-authoritative.yaml b/tests/modules/project/examples/iam-authoritative.yaml index eeb77aa6e5..d889a88312 100644 --- a/tests/modules/project/examples/iam-authoritative.yaml +++ b/tests/modules/project/examples/iam-authoritative.yaml @@ -16,29 +16,29 @@ values: module.project.google_project.project[0]: auto_create_network: false billing_account: 123456-123456-123456 - folder_id: '1234567890' + folder_id: '1122334455' labels: null - name: foo-project-example + name: test-project org_id: null - project_id: foo-project-example + project_id: test-project skip_delete: false timeouts: null module.project.google_project_iam_binding.authoritative["roles/container.hostServiceAgentUser"]: condition: [] members: - serviceAccount:my_gke_service_account - project: foo-project-example + project: test-project role: roles/container.hostServiceAgentUser module.project.google_project_service.project_services["container.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: foo-project-example + project: test-project service: container.googleapis.com timeouts: null module.project.google_project_service.project_services["stackdriver.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: foo-project-example + project: test-project service: stackdriver.googleapis.com timeouts: null diff --git a/tests/modules/project/examples/iam-bindings-additive.yaml b/tests/modules/project/examples/iam-bindings-additive.yaml index 390781f97c..2276622bb3 100644 --- a/tests/modules/project/examples/iam-bindings-additive.yaml +++ b/tests/modules/project/examples/iam-bindings-additive.yaml @@ -15,23 +15,23 @@ values: module.project.google_project.project[0]: auto_create_network: false - billing_account: null - folder_id: null + billing_account: 123456-123456-123456 + folder_id: '1122334455' labels: null - name: project-1 + name: test-project org_id: null - project_id: project-1 + project_id: test-project skip_delete: false timeouts: null module.project.google_project_iam_member.bindings["group-owner"]: condition: [] - member: group:p1-owners@example.org - project: project-1 + member: group:organization-admins@example.org + project: test-project role: roles/owner module.project.google_project_service.project_services["compute.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: project-1 + project: test-project service: compute.googleapis.com timeouts: null diff --git a/tests/modules/project/examples/iam-bindings.yaml b/tests/modules/project/examples/iam-bindings.yaml index c9fee92524..eddc582083 100644 --- a/tests/modules/project/examples/iam-bindings.yaml +++ b/tests/modules/project/examples/iam-bindings.yaml @@ -16,11 +16,11 @@ values: module.project.google_project.project[0]: auto_create_network: false billing_account: 123456-123456-123456 - folder_id: '1234567890' + folder_id: '1122334455' labels: null - name: foo-project-example + name: test-project org_id: null - project_id: foo-project-example + project_id: test-project skip_delete: false timeouts: null module.project.google_project_iam_binding.bindings["iam_admin_conditional"]: @@ -30,19 +30,19 @@ values: \ []\n).hasOnly([\n 'roles/compute.networkAdmin'\n])\n" title: delegated_network_user_one members: - - group:test-admins@example.org - project: foo-project-example + - group:organization-admins@example.org + project: test-project role: roles/resourcemanager.projectIamAdmin module.project.google_project_service.project_services["container.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: foo-project-example + project: test-project service: container.googleapis.com timeouts: null module.project.google_project_service.project_services["stackdriver.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: foo-project-example + project: test-project service: stackdriver.googleapis.com timeouts: null diff --git a/tests/modules/project/examples/iam-group.yaml b/tests/modules/project/examples/iam-group.yaml index 0ca2ecdba9..73f14b80a6 100644 --- a/tests/modules/project/examples/iam-group.yaml +++ b/tests/modules/project/examples/iam-group.yaml @@ -17,26 +17,26 @@ values: module.project.google_project_iam_binding.authoritative["roles/cloudasset.owner"]: condition: [] members: - - group:gcp-security-admins@example.com - project: foo-project-example + - group:organization-admins@example.org + project: test-project role: roles/cloudasset.owner module.project.google_project_iam_binding.authoritative["roles/cloudsupport.techSupportEditor"]: condition: [] members: - - group:gcp-security-admins@example.com - project: foo-project-example + - group:organization-admins@example.org + project: test-project role: roles/cloudsupport.techSupportEditor module.project.google_project_iam_binding.authoritative["roles/iam.securityReviewer"]: condition: [] members: - - group:gcp-security-admins@example.com - project: foo-project-example + - group:organization-admins@example.org + project: test-project role: roles/iam.securityReviewer module.project.google_project_iam_binding.authoritative["roles/logging.admin"]: condition: [] members: - - group:gcp-security-admins@example.com - project: foo-project-example + - group:organization-admins@example.org + project: test-project role: roles/logging.admin counts: diff --git a/tests/modules/project/examples/logging-data-access.yaml b/tests/modules/project/examples/logging-data-access.yaml index 3b35bbdc38..50d3886450 100644 --- a/tests/modules/project/examples/logging-data-access.yaml +++ b/tests/modules/project/examples/logging-data-access.yaml @@ -16,11 +16,11 @@ values: module.project.google_project.project[0]: auto_create_network: false billing_account: 123456-123456-123456 - folder_id: '1234567890' + folder_id: '1122334455' labels: null - name: my-project + name: test-project org_id: null - project_id: my-project + project_id: test-project skip_delete: false timeouts: null module.project.google_project_iam_audit_config.default["allServices"]: @@ -28,7 +28,7 @@ values: - exempted_members: - group:organization-admins@example.org log_type: ADMIN_READ - project: my-project + project: test-project service: allServices module.project.google_project_iam_audit_config.default["storage.googleapis.com"]: audit_log_config: @@ -36,7 +36,7 @@ values: log_type: DATA_READ - exempted_members: [] log_type: DATA_WRITE - project: my-project + project: test-project service: storage.googleapis.com counts: diff --git a/tests/modules/project/examples/logging.yaml b/tests/modules/project/examples/logging.yaml index 9634d8d102..7e96968afe 100644 --- a/tests/modules/project/examples/logging.yaml +++ b/tests/modules/project/examples/logging.yaml @@ -21,7 +21,7 @@ values: disabled: null filter: resource.type=gce_instance name: no-gce-instances - project: my-project + project: test-project module.project-host.google_logging_project_sink.sink["debug"]: description: debug (Terraform-managed). disabled: false @@ -32,7 +32,7 @@ values: name: no-compute filter: severity=DEBUG name: debug - project: my-project + project: test-project unique_writer_identity: true module.project-host.google_logging_project_sink.sink["info"]: description: info (Terraform-managed). @@ -40,7 +40,7 @@ values: exclusions: [] filter: severity=INFO name: info - project: my-project + project: test-project unique_writer_identity: true module.project-host.google_logging_project_sink.sink["notice"]: description: notice (Terraform-managed). @@ -48,25 +48,25 @@ values: exclusions: [] filter: severity=NOTICE name: notice - project: my-project + project: test-project unique_writer_identity: true module.project-host.google_logging_project_sink.sink["warnings"]: description: warnings (Terraform-managed). - destination: storage.googleapis.com/gcs_sink + destination: storage.googleapis.com/test-gcs_sink disabled: false exclusions: [] filter: severity=WARNING name: warnings - project: my-project + project: test-project unique_writer_identity: true module.project-host.google_project.project[0]: auto_create_network: false billing_account: 123456-123456-123456 - folder_id: '1234567890' + folder_id: '1122334455' labels: null - name: my-project + name: test-project org_id: null - project_id: my-project + project_id: test-project skip_delete: false module.project-host.google_project_iam_member.bucket-sinks-binding["debug"]: condition: @@ -76,7 +76,7 @@ values: condition: [] role: roles/pubsub.publisher module.project-host.google_storage_bucket_iam_member.gcs-sinks-binding["warnings"]: - bucket: gcs_sink + bucket: test-gcs_sink condition: [] role: roles/storage.objectCreator diff --git a/tests/modules/project/examples/org-policies.yaml b/tests/modules/project/examples/org-policies.yaml index d4dddc75bf..6539c32c41 100644 --- a/tests/modules/project/examples/org-policies.yaml +++ b/tests/modules/project/examples/org-policies.yaml @@ -14,8 +14,8 @@ values: module.project.google_org_policy_policy.default["compute.disableGuestAttributesAccess"]: - name: projects/foo-project-example/policies/compute.disableGuestAttributesAccess - parent: projects/foo-project-example + name: projects/test-project/policies/compute.disableGuestAttributesAccess + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -26,8 +26,8 @@ values: enforce: 'TRUE' values: [] module.project.google_org_policy_policy.default["compute.skipDefaultNetworkCreation"]: - name: projects/foo-project-example/policies/compute.skipDefaultNetworkCreation - parent: projects/foo-project-example + name: projects/test-project/policies/compute.skipDefaultNetworkCreation + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -38,8 +38,8 @@ values: enforce: 'TRUE' values: [] module.project.google_org_policy_policy.default["compute.trustedImageProjects"]: - name: projects/foo-project-example/policies/compute.trustedImageProjects - parent: projects/foo-project-example + name: projects/test-project/policies/compute.trustedImageProjects + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -53,8 +53,8 @@ values: - projects/my-project denied_values: null module.project.google_org_policy_policy.default["compute.vmExternalIpAccess"]: - name: projects/foo-project-example/policies/compute.vmExternalIpAccess - parent: projects/foo-project-example + name: projects/test-project/policies/compute.vmExternalIpAccess + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -65,8 +65,8 @@ values: enforce: null values: [] module.project.google_org_policy_policy.default["iam.allowedPolicyMemberDomains"]: - name: projects/foo-project-example/policies/iam.allowedPolicyMemberDomains - parent: projects/foo-project-example + name: projects/test-project/policies/iam.allowedPolicyMemberDomains + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -81,8 +81,8 @@ values: - C0yyyyyyy denied_values: null module.project.google_org_policy_policy.default["iam.disableServiceAccountKeyCreation"]: - name: projects/foo-project-example/policies/iam.disableServiceAccountKeyCreation - parent: projects/foo-project-example + name: projects/test-project/policies/iam.disableServiceAccountKeyCreation + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -93,8 +93,8 @@ values: enforce: 'TRUE' values: [] module.project.google_org_policy_policy.default["iam.disableServiceAccountKeyUpload"]: - name: projects/foo-project-example/policies/iam.disableServiceAccountKeyUpload - parent: projects/foo-project-example + name: projects/test-project/policies/iam.disableServiceAccountKeyUpload + parent: projects/test-project spec: - inherit_from_parent: null reset: null @@ -115,10 +115,10 @@ values: values: [] module.project.google_project.project[0]: billing_account: 123456-123456-123456 - folder_id: '1234567890' - name: foo-project-example + folder_id: '1122334455' + name: test-project org_id: null - project_id: foo-project-example + project_id: test-project counts: google_org_policy_policy: 7 diff --git a/tests/modules/project/examples/shared-vpc-auto-grants.yaml b/tests/modules/project/examples/shared-vpc-auto-grants.yaml index 226612394b..4dacb377b9 100644 --- a/tests/modules/project/examples/shared-vpc-auto-grants.yaml +++ b/tests/modules/project/examples/shared-vpc-auto-grants.yaml @@ -14,25 +14,25 @@ values: module.host-project.google_compute_shared_vpc_host_project.shared_vpc_host[0]: - project: my-host-project + project: test-host module.host-project.google_project.project[0]: - project_id: my-host-project + project_id: test-host module.service-project.google_compute_shared_vpc_service_project.shared_vpc_service[0]: - host_project: my-host-project - service_project: my-service-project + host_project: test-host + service_project: test-service module.service-project.google_project.project[0]: - project_id: my-service-project + project_id: test-service module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:cloudservices"]: condition: [] - project: my-host-project + project: test-host role: roles/compute.networkUser module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:container"]: condition: [] - project: my-host-project + project: test-host role: roles/compute.networkUser module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container"]: condition: [] - project: my-host-project + project: test-host role: roles/container.hostServiceAgentUser counts: diff --git a/tests/modules/project/examples/shared-vpc.yaml b/tests/modules/project/examples/shared-vpc.yaml index 169c897de2..4277d56b3d 100644 --- a/tests/modules/project/examples/shared-vpc.yaml +++ b/tests/modules/project/examples/shared-vpc.yaml @@ -14,29 +14,29 @@ values: module.host-project.google_compute_shared_vpc_host_project.shared_vpc_host[0]: - project: my-host-project + project: test-host module.host-project.google_project.project[0]: - project_id: my-host-project + project_id: test-host module.service-project.google_compute_shared_vpc_service_project.shared_vpc_service[0]: - host_project: my-host-project - service_project: my-service-project + host_project: test-host + service_project: test-service module.service-project.google_project.project[0]: - project_id: my-service-project + project_id: test-service module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:cloudservices"]: condition: [] - project: my-host-project + project: test-host role: roles/compute.networkUser module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:container-engine"]: condition: [] - project: my-host-project + project: test-host role: roles/compute.networkUser module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container-engine"]: condition: [] - project: my-host-project + project: test-host role: roles/container.hostServiceAgentUser module.service-project.google_project_iam_member.shared_vpc_host_robots["roles/vpcaccess.user:cloudrun"]: condition: [] - project: my-host-project + project: test-host role: roles/vpcaccess.user counts: