Assorted module fixes

blueprints/README.md

@@ -8,7 +8,7 @@ Currently available blueprints:
 - **data solutions** - [GCE and GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms), [Cloud Composer version 2 private instance, supporting Shared VPC and external CMEK key](./data-solutions/composer-2), [Cloud SQL instance with multi-region read replicas](./data-solutions/cloudsql-multiregion), [Data Platform](./data-solutions/data-platform-foundations), [Spinning up a foundation data pipeline on Google Cloud using Cloud Storage, Dataflow and BigQuery](./data-solutions/gcs-to-bq-with-least-privileges), [#SQL Server Always On Groups blueprint](./data-solutions/sqlserver-alwayson), [Data Playground](./data-solutions/data-playground)
 - **factories** - [The why and the how of Resource Factories](./factories), [Google Cloud Identity Group Factory](./factories/cloud-identity-group-factory), [Google Cloud BQ Factory](./factories/bigquery-factory), [Google Cloud VPC Firewall Factory](./factories/net-vpc-firewall-yaml), [Minimal Project Factory](./factories/project-factory)
 - **GKE** - [Binary Authorization Pipeline Blueprint](./gke/binauthz), [Storage API](./gke/binauthz/image), [Multi-cluster mesh on GKE (fleet API)](./gke/multi-cluster-mesh-gke-fleet-api), [GKE Multitenant Blueprint](./gke/multitenant-fleet), [Shared VPC with GKE support](./networking/shared-vpc-gke/)
-- **networking** - [Decentralized firewall management](./networking/decentralized-firewall), [Decentralized firewall validator](./networking/decentralized-firewall/validator), [Network filtering with Squid](./networking/filtering-proxy), [Network filtering with Squid with isolated VPCs using Private Service Connect](./networking/filtering-proxy-psc), [HTTP Load Balancer with Cloud Armor](./networking/glb-and-armor), [Hub and Spoke via VPN](./networking/hub-and-spoke-vpn), [Hub and Spoke via VPC Peering](./networking/hub-and-spoke-peering), [Internal Load Balancer as Next Hop](./networking/ilb-next-hop), [On-prem DNS and Google Private Access](./networking/onprem-google-access-dns), [Calling a private Cloud Function from On-premises](./networking/private-cloud-function-from-onprem), [Hybrid connectivity to on-premise services through PSC](./networking/psc-hybrid), [PSC Producer](./networking/psc-hybrid/psc-producer), [PSC Consumer](./networking/psc-hybrid/psc-consumer), [Shared VPC with optional GKE cluster](./networking/shared-vpc-gke)
+- **networking** - [Decentralized firewall management](./networking/decentralized-firewall), [Decentralized firewall validator](./networking/decentralized-firewall/validator), [Network filtering with Squid](./networking/filtering-proxy), [Network filtering with Squid with isolated VPCs using Private Service Connect](./networking/filtering-proxy-psc), [HTTP Load Balancer with Cloud Armor](./networking/glb-and-armor), [Hub and Spoke via VPN](./networking/hub-and-spoke-vpn), [Hub and Spoke via VPC Peering](./networking/hub-and-spoke-peering), [Internal Load Balancer as Next Hop](./networking/ilb-next-hop), On-prem DNS and Google Private Access, [Calling a private Cloud Function from On-premises](./networking/private-cloud-function-from-onprem), [Hybrid connectivity to on-premise services through PSC](./networking/psc-hybrid), [PSC Producer](./networking/psc-hybrid/psc-producer), [PSC Consumer](./networking/psc-hybrid/psc-consumer), [Shared VPC with optional GKE cluster](./networking/shared-vpc-gke)
 - **serverless** - [Creating multi-region deployments for API Gateway](./serverless/api-gateway)
 - **third party solutions** - [OpenShift on GCP user-provisioned infrastructure](./third-party-solutions/openshift), [Wordpress deployment on Cloud Run](./third-party-solutions/wordpress/cloudrun)
 

blueprints/cloud-operations/unmanaged-instances-healthcheck/main.tf

@@ -146,13 +146,11 @@ module "cf-healthchecker" {
   name        = "cf-healthchecker"
   region      = var.region
   bucket_name = module.cf-restarter.bucket_name
-
   bundle_config = {
     source_dir  = "${path.module}/function/healthchecker"
     output_path = "healthchecker.zip"
   }
   service_account = module.service-account-healthchecker.email
-
   function_config = {
     entry_point      = "HealthCheck"
     ingress_settings = null
@@ -161,7 +159,6 @@ module "cf-healthchecker" {
     runtime          = "go116"
     timeout          = 300
   }
-
   environment_variables = {
     FILTER       = "name = nginx-*"
     GRACE_PERIOD = var.grace_period
@@ -171,7 +168,6 @@ module "cf-healthchecker" {
     TCP_PORT     = var.tcp_port
     TIMEOUT      = var.timeout
   }
-
   vpc_connector = {
     create          = true
     name            = "hc-connector"
@@ -230,23 +226,25 @@ resource "google_cloud_scheduler_job" "healthcheck-job" {
 
 module "cos-nginx" {
   source = "../../../modules/cloud-config-container/nginx"
-  test_instance = {
-    project_id = module.project.project_id
-    zone       = "${var.region}-b"
-    name       = "nginx-test"
-    type       = "f1-micro"
+}
+
+module "test-vm" {
+  source     = "../../../modules/compute-vm"
+  project_id = module.project.project_id
+  zone       = "${var.region}-b"
+  name       = "nginx-test"
+  boot_disk = {
+    image = "projects/cos-cloud/global/images/family/cos-stable"
+    type  = "pd-ssd"
+    size  = 10
+  }
+  metadata = {
+    user-data              = module.cos-nginx.cloud_config
+    google-logging-enabled = true
+  }
+  network_interfaces = [{
     network    = module.vpc.self_link
     subnetwork = module.vpc.subnet_self_links["${var.region}/apps"]
-  }
-  test_instance_defaults = {
-    disks    = {}
-    image    = null
-    metadata = {}
-    nat      = false
-    service_account_roles = [
-      "roles/logging.logWriter",
-      "roles/monitoring.metricWriter"
-    ]
-    tags = ["ssh"]
-  }
+  }]
+  tags = ["ssh"]
 }

blueprints/networking/README.md

@@ -49,19 +49,20 @@ The blueprint shows how to implement spoke transitivity via BGP advertisements,
 <!--
 ### Nginx-based reverse proxy cluster
 
-<a href="./_deprecated/nginx-reverse-proxy-cluster/" title="Nginx-based reverse proxy cluster"><img src="./_deprecated/nginx-reverse-proxy-cluster/reverse-proxy.png" align="left" width="280px"></a> This [blueprint](./nginx-reverse-proxy-cluster/) how to deploy an autoscaling reverse proxy cluster using Nginx, based on regional Managed Instance Groups. The autoscaling is driven by Nginx current connections metric, sent by Cloud Ops Agent.
+<a href="./__need_fixing/nginx-reverse-proxy-cluster/" title="Nginx-based reverse proxy cluster"><img src="./_deprecated/nginx-reverse-proxy-cluster/reverse-proxy.png" align="left" width="280px"></a> This [blueprint](./nginx-reverse-proxy-cluster/) how to deploy an autoscaling reverse proxy cluster using Nginx, based on regional Managed Instance Groups. The autoscaling is driven by Nginx current connections metric, sent by Cloud Ops Agent.
 
 <br clear="left">
--->
 
 ### DNS and Private Access for On-premises
 
-<a href="./onprem-google-access-dns/" title="DNS and Private Access for On-premises"><img src="./onprem-google-access-dns/diagram.png" align="left" width="280px"></a> This [blueprint](./onprem-google-access-dns/) uses an emulated on-premises environment running in Docker containers inside a GCE instance, to allow testing specific features like DNS policies, DNS forwarding zones across VPN, and Private Access for On-premises hosts.
+<a href="./__need_fixing/onprem-google-access-dns/" title="DNS and Private Access for On-premises"><img src="./onprem-google-access-dns/diagram.png" align="left" width="280px"></a> This [blueprint](./onprem-google-access-dns/) uses an emulated on-premises environment running in Docker containers inside a GCE instance, to allow testing specific features like DNS policies, DNS forwarding zones across VPN, and Private Access for On-premises hosts.
 
 The emulated on-premises environment can be used to test access to different services from outside Google Cloud, by implementing a VPN connection and BGP to Google CLoud via Strongswan and Bird.
 
 <br clear="left">
 
+-->
+
 ### Calling a private Cloud Function from on-premises
 
 <a href="./private-cloud-function-from-onprem/" title="Private Cloud Function from On-premises"><img src="./private-cloud-function-from-onprem/diagram.png" align="left" width="280px"></a> This [blueprint](./private-cloud-function-from-onprem/) shows how to invoke a [private Google Cloud Function](https://cloud.google.com/functions/docs/networking/network-settings) from the on-prem environment via a [Private Service Connect endpoint](https://cloud.google.com/vpc/docs/private-service-connect#benefits-apis).

blueprints/networking/onprem-google-access-dns/README.md → blueprints/networking/__need_fixing/onprem-google-access-dns/README.md

@@ -1,6 +1,6 @@
 # On-prem DNS and Google Private Access
 
-This blueprint leverages the [on prem in a box](../../../modules/cloud-config-container/onprem) module to bootstrap an emulated on-premises environment on GCP, then connects it via VPN and sets up BGP and DNS so that several specific features can be tested:
+This blueprint leverages the on prem in a box module to bootstrap an emulated on-premises environment on GCP, then connects it via VPN and sets up BGP and DNS so that several specific features can be tested:
 
 - [Cloud DNS forwarding zone](https://cloud.google.com/dns/docs/overview#fz-targets) to on-prem
 - DNS forwarding from on-prem via a [Cloud DNS inbound policy](https://cloud.google.com/dns/docs/policies#create-in)
@@ -30,7 +30,7 @@ The Cloud DNS inbound policy reserves an IP address in the VPC, which is used by
 
 ### Find out the forwarder entry point address
 
-Run this gcloud command to (find out the address assigned to the inbound forwarder)[https://cloud.google.com/dns/docs/policies#list-in-entrypoints]:
+Run this gcloud command to [find out the address assigned to the inbound forwarder](https://cloud.google.com/dns/docs/policies#list-in-entrypoints):
 
 ```bash
 gcloud compute addresses list --project [your project id]
@@ -199,7 +199,7 @@ curl www.onprem.example.org -s |grep h1
 
 A single pre-existing project is used in this blueprint to keep variables and complexity to a minimum, in a real world scenarios each spoke would probably use a separate project.
 
-The VPN-s used to connect to the on-premises environment do not account for HA, upgrading to use HA VPN is reasonably simple by using the relevant [module](../../../modules/net-vpn-ha).
+The VPN-s used to connect to the on-premises environment do not account for HA, upgrading to use HA VPN is reasonably simple by using the relevant [module](../../../../modules/net-vpn-ha).
 <!-- BEGIN TFDOC -->
 
 ## Variables

modules/cloud-config-container/README.md

@@ -1,6 +1,6 @@
 # Instance Configuration via `cloud-config`
 
-This set of modules creates specialized [cloud-config](https://cloud.google.com/container-optimized-os/docs/how-to/run-container-instance#starting_a_docker_container_via_cloud-config) configurations, which are designed for use with [Container Optimized OS](https://cloud.google.com/container-optimized-os/docs) (the [onprem module](./onprem/) is the only exception) but can also be used as a basis for other image types or cloud providers.
+This set of modules creates specialized [cloud-config](https://cloud.google.com/container-optimized-os/docs/how-to/run-container-instance#starting_a_docker_container_via_cloud-config) configurations, which are designed for use with [Container Optimized OS](https://cloud.google.com/container-optimized-os/docs) (the onprem module is the only exception) but can also be used as a basis for other image types or cloud providers.
 
 These modules are designed for several use cases:
 
@@ -14,17 +14,15 @@ These modules are designed for several use cases:
 - [CoreDNS](./coredns)
 - [MySQL](./mysql)
 - [Nginx](./nginx)
-- [On-prem in Docker](./onprem)
 - [Squid forward proxy](./squid)
+- On-prem in Docker (*needs fixing*)
 
 ## Using the modules
 
 All modules are designed to be as lightweight as possible, so that specialized modules like [compute-vm](../compute-vm) can be leveraged to manage instances or instance templates, and to allow simple forking to create custom derivatives.
 
 To use the modules with instances or instance templates, simply set use their `cloud_config` output for the `user-data` metadata. When updating the metadata after a variable change remember to manually restart the instances that use a module's output, or the changes won't effect the running system.
 
-For convenience when developing or prototyping infrastructure, an optional test instance is included in all modules. If it's not needed, the linked `*instance.tf` files can be removed from the modules without harm.
-
 ## TODO
 
 - [ ] convert all `xxx_config` variables to use file content instead of path

modules/cloud-config-container/onprem/README.md → modules/cloud-config-container/__need_fixing/onprem/README.md

@@ -10,18 +10,14 @@ The emulated on-premises infrastructure is composed of:
 - an Nginx container serving a simple static web page
 - a [generic Linux container](./docker-images/toolbox) used as a jump host inside the on-premises network
 
-A [complete scenario using this module](../../../blueprints/networking/onprem-google-access-dns) is available in the networking blueprints.
+A complete scenario using this module is available in the networking blueprints.
 
 The module renders the generated cloud config in the `cloud_config` output, to be used in instances or instance templates via the `user-data` metadata.
 
-For convenience during development or for simple use cases, the module can optionally manage a single instance via the `test_instance` variable. If the instance is not needed the `instance*tf` files can be safely removed. Refer to the [top-level README](../README.md) for more details on the included instance.
-
 ## Examples
 
 ### Static VPN
 
-The test instance is optional, as described above.
-
 ```hcl
 module "cloud-vpn" {
   source        = "./fabric/modules/net-vpn-static"
@@ -32,42 +28,54 @@ module "cloud-vpn" {
   remote_ranges = ["192.168.192.0/24"]
   tunnels = {
     remote-0 = {
-      peer_ip           = module.on-prem.external_address
+      peer_ip           = module.vm.external_ip
       traffic_selectors = { local = ["0.0.0.0/0"], remote = null }
     }
   }
 }
 
 module "on-prem" {
-  source = "./fabric/modules/cos-container/on-prem"
-  name       = "onprem"
+  source = "./fabric/modules/cloud-config-container/onprem"
   vpn_config = {
     type          = "static"
     peer_ip       = module.cloud-vpn.address
     shared_secret = module.cloud-vpn.random_secret
   }
-  test_instance = {
-    project_id = "my-project"
-    zone       = "europe-west1-b"
-    name       = "cos-coredns"
-    type       = "f1-micro"
+}
+
+module "vm" {
+  source     = "./fabric/modules/compute-vm"
+  project_id = "my-project"
+  zone       = "europe-west8-b"
+  name       = "cos-nginx-tls"
+  network_interfaces = [{
+    nat        = true
     network    = "default"
-    subnetwork = "https://www.googleapis.com/compute/v1/projects/my-project/regions/europe-west1/subnetworks/my-subnet"
+    subnetwork = "gce"
+  }]
+  metadata = {
+    user-data              = module.on-prem.cloud_config
+    google-logging-enabled = true
+  }
+  boot_disk = {
+    image = "projects/cos-cloud/global/images/family/cos-stable"
+    type  = "pd-ssd"
+    size  = 10
   }
+  tags = ["ssh"]
 }
+# tftest skip
 ```
 <!-- BEGIN TFDOC -->
 
 ## Variables
 
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [vpn_config](variables.tf#L35) | VPN configuration, type must be one of 'dynamic' or 'static'. | <code title="object&#40;&#123;&#10;  peer_ip        &#61; string&#10;  shared_secret  &#61; string&#10;  type           &#61; string&#10;  peer_ip2       &#61; string&#10;  shared_secret2 &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
+| [vpn_config](variables.tf#L35) | VPN configuration, type must be one of 'dynamic' or 'static'. | <code title="object&#40;&#123;&#10;  peer_ip        &#61; string&#10;  shared_secret  &#61; string&#10;  type           &#61; optional&#40;string, &#34;static&#34;&#41;&#10;  peer_ip2       &#61; optional&#40;string&#41;&#10;  shared_secret2 &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
 | [config_variables](variables.tf#L17) | Additional variables used to render the cloud-config and CoreDNS templates. | <code>map&#40;any&#41;</code> |  | <code>&#123;&#125;</code> |
 | [coredns_config](variables.tf#L23) | CoreDNS configuration path, if null default will be used. | <code>string</code> |  | <code>null</code> |
 | [local_ip_cidr_range](variables.tf#L29) | IP CIDR range used for the Docker onprem network. | <code>string</code> |  | <code>&#34;192.168.192.0&#47;24&#34;</code> |
-| [test_instance](variables-instance.tf#L17) | Test/development instance attributes, leave null to skip creation. | <code title="object&#40;&#123;&#10;  project_id &#61; string&#10;  zone       &#61; string&#10;  name       &#61; string&#10;  type       &#61; string&#10;  network    &#61; string&#10;  subnetwork &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
-| [test_instance_defaults](variables-instance.tf#L30) | Test/development instance defaults used for optional configuration. If image is null, COS stable will be used. | <code title="object&#40;&#123;&#10;  disks &#61; map&#40;object&#40;&#123;&#10;    read_only &#61; bool&#10;    size      &#61; number&#10;  &#125;&#41;&#41;&#10;  image                 &#61; string&#10;  metadata              &#61; map&#40;string&#41;&#10;  nat                   &#61; bool&#10;  service_account_roles &#61; list&#40;string&#41;&#10;  tags                  &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  disks    &#61; &#123;&#125;&#10;  image    &#61; null&#10;  metadata &#61; &#123;&#125;&#10;  nat      &#61; false&#10;  service_account_roles &#61; &#91;&#10;    &#34;roles&#47;logging.logWriter&#34;,&#10;    &#34;roles&#47;monitoring.metricWriter&#34;&#10;  &#93;&#10;  tags &#61; &#91;&#34;ssh&#34;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> |
 | [vpn_dynamic_config](variables.tf#L46) | BGP configuration for dynamic VPN, ignored if VPN type is 'static'. | <code title="object&#40;&#123;&#10;  local_bgp_asn      &#61; number&#10;  local_bgp_address  &#61; string&#10;  peer_bgp_asn       &#61; number&#10;  peer_bgp_address   &#61; string&#10;  local_bgp_asn2     &#61; number&#10;  local_bgp_address2 &#61; string&#10;  peer_bgp_asn2      &#61; number&#10;  peer_bgp_address2  &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  local_bgp_asn      &#61; 64514&#10;  local_bgp_address  &#61; &#34;169.254.1.2&#34;&#10;  peer_bgp_asn       &#61; 64513&#10;  peer_bgp_address   &#61; &#34;169.254.1.1&#34;&#10;  local_bgp_asn2     &#61; 64514&#10;  local_bgp_address2 &#61; &#34;169.254.2.2&#34;&#10;  peer_bgp_asn2      &#61; 64520&#10;  peer_bgp_address2  &#61; &#34;169.254.2.1&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
 | [vpn_static_ranges](variables.tf#L70) | Remote CIDR ranges for static VPN, ignored if VPN type is 'dynamic'. | <code>list&#40;string&#41;</code> |  | <code>&#91;&#34;10.0.0.0&#47;8&#34;&#93;</code> |
 
@@ -76,7 +84,5 @@ module "on-prem" {
 | name | description | sensitive |
 |---|---|:---:|
 | [cloud_config](outputs.tf#L17) | Rendered cloud-config file to be passed as user-data instance metadata. |  |
-| [test_instance](outputs-instance.tf#L17) | Optional test instance name and address. |  |
 
 <!-- END TFDOC -->
-

modules/cloud-config-container/onprem/variables.tf → modules/cloud-config-container/__need_fixing/onprem/variables.tf

@@ -37,9 +37,9 @@ variable "vpn_config" {
   type = object({
     peer_ip        = string
     shared_secret  = string
-    type           = string
-    peer_ip2       = string
-    shared_secret2 = string
+    type           = optional(string, "static")
+    peer_ip2       = optional(string)
+    shared_secret2 = optional(string)
   })
 }