diff --git a/infrastructure/hub-and-spoke-peering/README.md b/infrastructure/hub-and-spoke-peering/README.md new file mode 100644 index 0000000000..6e4cb87cdc --- /dev/null +++ b/infrastructure/hub-and-spoke-peering/README.md @@ -0,0 +1,75 @@ +# Hub and Spoke via VPC Peering + +This example creates a simple **Hub and Spoke** setup, where the VPC network connects satellite locations (spokes) through a single intermediary location (hub) via [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering). + +The example shows some of the limitations that need to be taken into account when using VPC Peering, mostly due to the lack of transivity between peerings: + +- no mesh networking between the spokes +- complex support for managed services hosted in tenant VPCs connected via peering (Cloud SQL, GKE, etc.) + +One possible solution to the managed service limitation above is presented here, using a static VPN to establish connectivity to the GKE masters in the tenant project ([courtesy of @drebes](https://github.com/drebes/tf-samples/blob/master/gke-master-from-hub/main.tf#L10)). Other solutions typically involve the use of proxies, as [described in this GKE article](https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies). + +One other topic that needs to be considered when using peering is the limit of 25 peerings in each peering group, which constrains the scalability of design like the one presented here. + +The example has been purposefully kept simple to show how to use and wire the VPC modules together, and so that it can be used as a basis for more complex scenarios. This is the high level diagram: + +![High-level diagram](diagram.png "High-level diagram") + +## Managed resources and services + +This sample creates several distinct groups of resources: + +- one VPC each for hub and each spoke +- one set of firewall rules for each VPC +- one Cloud NAT configuration for each spoke +- one test instance for each spoke +- one GKE cluster with a single nodepool in spoke 2 +- one service account for the GCE instances +- one service account for the GKE nodes +- one static VPN gateway in hub and spoke 2 with a single tunnel each + +## Testing GKE access from spoke 1 + +As mentioned above, a VPN tunnel is used as a workaround to avoid the peering transitivity issue that would prevent any VPC other than spoke 2 to connect to the GKE master. + +To test cluster access, first log on to the spoke 2 instance and confirm cluster and IAM roles are set up correctly: + +```bash +gcloud container clusters get-credentials cluster-1 --zone europe-west1-b +kubectl get all +``` + +The next step is to edit the peering towards the GKE master tenant VPC, and enable export routes. The peering has a name like `gke-xxxxxxxxxxxxxxxxxxxx-xxxx-xxxx-peer`, you can edit it in the Cloud Console from the *VPC network peering* page or using `gcloud`: + +``` +gcloud compute networks peerings list +# find the gke-xxxxxxxxxxxxxxxxxxxx-xxxx-xxxx-peer in the spoke-2 network +gcloud compute networks peerings update [peering name from above] \ + --network spoke-2 --export-custom-routes +``` + +Then connect via SSH to the spoke 1 instance and run the same commands you ran on the spoke 2 instance above, you should be able to run `kubectl` commands against the cluster. To test the default situation with no supporting VPN, just comment out the two VPN modules in `main.tf` and run `terraform apply` to bring down the VPN gateways and tunnels. GKE should only become accessible from spoke 2. + +## Operational considerations + +A single pre-existing project is used in this example to keep variables and complexity to a minimum, in a real world scenarios each spoke would probably use a separate project. + +The VPN used to connect the GKE masters VPC does not account for HA, upgrading to use HA VPN is reasonably simple by using the relevant [module](../../modules/net-vpn-ha). + + +## Variables + +| name | description | type | required | default | +|---|---|:---: |:---:|:---:| +| project_id | Project id for all resources. | string | ✓ | | +| *ip_ranges* | IP CIDR ranges. | map(string) | | ... | +| *ip_secondary_ranges* | Secondary IP CIDR ranges. | map(string) | | ... | +| *private_service_ranges* | Private service IP CIDR ranges. | map(string) | | ... | +| *region* | VPC regions. | string | | europe-west1 | + +## Outputs + +| name | description | sensitive | +|---|---|:---:| +| vms | GCE VMs. | | + diff --git a/infrastructure/hub-and-spoke-peerings/backend.tf.sample b/infrastructure/hub-and-spoke-peering/backend.tf.sample similarity index 100% rename from infrastructure/hub-and-spoke-peerings/backend.tf.sample rename to infrastructure/hub-and-spoke-peering/backend.tf.sample diff --git a/infrastructure/hub-and-spoke-peering/diagram.png b/infrastructure/hub-and-spoke-peering/diagram.png new file mode 100644 index 0000000000..0c598fe56e Binary files /dev/null and b/infrastructure/hub-and-spoke-peering/diagram.png differ diff --git a/infrastructure/hub-and-spoke-peering/main.tf b/infrastructure/hub-and-spoke-peering/main.tf new file mode 100644 index 0000000000..9f4b875734 --- /dev/null +++ b/infrastructure/hub-and-spoke-peering/main.tf @@ -0,0 +1,280 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +locals { + vm-instances = concat( + module.vm-spoke-1.instances, + module.vm-spoke-2.instances + ) + vm-startup-script = join("\n", [ + "#! /bin/bash", + "apt-get update && apt-get install -y bash-completion dnsutils kubectl" + ]) +} + +################################################################################ +# Hub networking # +################################################################################ + +module "vpc-hub" { + source = "../../modules/net-vpc" + project_id = var.project_id + name = "hub" + subnets = { + default = { + ip_cidr_range = var.ip_ranges.hub + region = var.region + secondary_ip_range = {} + } + } +} + +module "vpc-hub-firewall" { + source = "../../modules/net-vpc-firewall" + project_id = var.project_id + network = module.vpc-hub.name + admin_ranges_enabled = true + admin_ranges = values(var.ip_ranges) +} + +################################################################################ +# Spoke 1 networking # +################################################################################ + +module "vpc-spoke-1" { + source = "../../modules/net-vpc" + project_id = var.project_id + name = "spoke-1" + subnets = { + default = { + ip_cidr_range = var.ip_ranges.spoke-1 + region = var.region + secondary_ip_range = {} + } + } +} + +module "vpc-spoke-1-firewall" { + source = "../../modules/net-vpc-firewall" + project_id = var.project_id + network = module.vpc-spoke-1.name + admin_ranges_enabled = true + admin_ranges = values(var.ip_ranges) +} + +module "nat-spoke-1" { + source = "../../modules/net-cloudnat" + project_id = var.project_id + region = module.vpc-spoke-1.subnet_regions.default + name = "spoke-1" + router_name = "spoke-1" + router_network = module.vpc-spoke-1.self_link +} + +module "hub-to-spoke-1-peering" { + source = "../../modules/net-vpc-peering" + local_network = module.vpc-hub.self_link + peer_network = module.vpc-spoke-1.self_link + export_local_custom_routes = true + export_peer_custom_routes = false +} + +################################################################################ +# Spoke 2 networking # +################################################################################ + +module "vpc-spoke-2" { + source = "../../modules/net-vpc" + project_id = var.project_id + name = "spoke-2" + subnets = { + default = { + ip_cidr_range = var.ip_ranges.spoke-2 + region = var.region + secondary_ip_range = { + pods = var.ip_secondary_ranges.spoke-2-pods + services = var.ip_secondary_ranges.spoke-2-services + } + } + } +} + +module "vpc-spoke-2-firewall" { + source = "../../modules/net-vpc-firewall" + project_id = var.project_id + network = module.vpc-spoke-2.name + admin_ranges_enabled = true + admin_ranges = values(var.ip_ranges) +} + +module "nat-spoke-2" { + source = "../../modules/net-cloudnat" + project_id = var.project_id + region = module.vpc-spoke-2.subnet_regions.default + name = "spoke-2" + router_name = "spoke-2" + router_network = module.vpc-spoke-2.self_link +} + +module "hub-to-spoke-2-peering" { + source = "../../modules/net-vpc-peering" + local_network = module.vpc-hub.self_link + peer_network = module.vpc-spoke-2.self_link + export_local_custom_routes = true + export_peer_custom_routes = false + module_depends_on = [module.hub-to-spoke-1-peering.complete] +} + +################################################################################ +# Test VMs # +################################################################################ + +module "vm-spoke-1" { + source = "../../modules/compute-vm" + project_id = var.project_id + region = module.vpc-spoke-1.subnet_regions.default + zone = "${module.vpc-spoke-1.subnet_regions.default}-b" + name = "spoke-1-test" + network_interfaces = [{ + network = module.vpc-spoke-1.self_link, + subnetwork = module.vpc-spoke-1.subnet_self_links.default, + nat = false, + addresses = null + }] + metadata = { startup-script = local.vm-startup-script } + service_account = module.service-account-gce.email + service_account_scopes = ["https://www.googleapis.com/auth/cloud-platform"] + tags = ["ssh"] +} + +module "vm-spoke-2" { + source = "../../modules/compute-vm" + project_id = var.project_id + region = module.vpc-spoke-2.subnet_regions.default + zone = "${module.vpc-spoke-2.subnet_regions.default}-b" + name = "spoke-2-test" + network_interfaces = [{ + network = module.vpc-spoke-2.self_link, + subnetwork = module.vpc-spoke-2.subnet_self_links.default, + nat = false, + addresses = null + }] + metadata = { startup-script = local.vm-startup-script } + service_account = module.service-account-gce.email + service_account_scopes = ["https://www.googleapis.com/auth/cloud-platform"] + tags = ["ssh"] +} + +module "service-account-gce" { + source = "../../modules/iam-service-accounts" + project_id = var.project_id + names = ["gce-test"] + iam_project_roles = { + (var.project_id) = [ + "roles/container.developer", + "roles/logging.logWriter", + "roles/monitoring.metricWriter", + ] + } +} + +################################################################################ +# GKE # +################################################################################ + +module "cluster-1" { + source = "../../modules/gke-cluster" + name = "cluster-1" + project_id = var.project_id + location = "${module.vpc-spoke-2.subnet_regions.default}-b" + network = module.vpc-spoke-2.self_link + subnetwork = module.vpc-spoke-2.subnet_self_links.default + secondary_range_pods = "pods" + secondary_range_services = "services" + default_max_pods_per_node = 32 + labels = { + environment = "test" + } + master_authorized_ranges = { + for name, range in var.ip_ranges : name => range + } + private_cluster_config = { + enable_private_nodes = true + enable_private_endpoint = true + master_ipv4_cidr_block = var.private_service_ranges.spoke-2-cluster-1 + } +} + +module "cluster-1-nodepool-1" { + source = "../../modules/gke-nodepool" + name = "nodepool-1" + project_id = var.project_id + location = module.cluster-1.location + cluster_name = module.cluster-1.name + node_config_service_account = module.service-account-gke-node.email +} + +# roles assigned via this module use non-authoritative IAM bindings at the +# project level, with no risk of conflicts with pre-existing roles + +module "service-account-gke-node" { + source = "../../modules/iam-service-accounts" + project_id = var.project_id + names = ["gke-node"] + iam_project_roles = { + (var.project_id) = [ + "roles/logging.logWriter", "roles/monitoring.metricWriter", + ] + } +} + +################################################################################ +# GKE peering VPN # +################################################################################ + +module "vpn-hub" { + source = "../../modules/net-vpn-static" + project_id = var.project_id + region = var.region + network = module.vpc-hub.name + name = "hub" + remote_ranges = values(var.private_service_ranges) + tunnels = { + spoke-2 = { + ike_version = 2 + peer_ip = module.vpn-spoke-2.address + shared_secret = "" + traffic_selectors = { local = ["0.0.0.0/0"], remote = null } + } + } +} + +module "vpn-spoke-2" { + source = "../../modules/net-vpn-static" + project_id = var.project_id + region = var.region + network = module.vpc-spoke-2.name + name = "spoke-2" + # use an aggregate of the remote ranges, so as to be less specific than the + # routes exchanged via peering + remote_ranges = ["10.0.0.0/8"] + tunnels = { + spoke-2 = { + ike_version = 2 + peer_ip = module.vpn-hub.address + shared_secret = module.vpn-hub.random_secret + traffic_selectors = { local = ["0.0.0.0/0"], remote = null } + } + } +} diff --git a/infrastructure/hub-and-spoke-peering/outputs.tf b/infrastructure/hub-and-spoke-peering/outputs.tf new file mode 100644 index 0000000000..933d03cf3e --- /dev/null +++ b/infrastructure/hub-and-spoke-peering/outputs.tf @@ -0,0 +1,21 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +output "vms" { + description = "GCE VMs." + value = { + for instance in local.vm-instances : + instance.name => instance.network_interface.0.network_ip + } +} diff --git a/infrastructure/hub-and-spoke-peering/variables.tf b/infrastructure/hub-and-spoke-peering/variables.tf new file mode 100644 index 0000000000..3f1375b2e6 --- /dev/null +++ b/infrastructure/hub-and-spoke-peering/variables.tf @@ -0,0 +1,51 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +variable "ip_ranges" { + description = "IP CIDR ranges." + type = map(string) + default = { + hub = "10.0.0.0/24" + spoke-1 = "10.0.16.0/24" + spoke-2 = "10.0.32.0/24" + } +} + +variable "ip_secondary_ranges" { + description = "Secondary IP CIDR ranges." + type = map(string) + default = { + spoke-2-pods = "10.128.0.0/18" + spoke-2-services = "172.16.0.0/24" + } +} + +variable "private_service_ranges" { + description = "Private service IP CIDR ranges." + type = map(string) + default = { + spoke-2-cluster-1 = "192.168.0.0/28" + } +} + +variable "project_id" { + description = "Project id for all resources." + type = string +} + +variable "region" { + description = "VPC region." + type = string + default = "europe-west1" +} diff --git a/infrastructure/hub-and-spoke-peerings/versions.tf b/infrastructure/hub-and-spoke-peering/versions.tf similarity index 100% rename from infrastructure/hub-and-spoke-peerings/versions.tf rename to infrastructure/hub-and-spoke-peering/versions.tf diff --git a/infrastructure/hub-and-spoke-peerings/README.md b/infrastructure/hub-and-spoke-peerings/README.md deleted file mode 100644 index c3fcc01310..0000000000 --- a/infrastructure/hub-and-spoke-peerings/README.md +++ /dev/null @@ -1,71 +0,0 @@ -# Hub and Spoke via VPC Peering - -This sample creates a simple **Hub and Spoke** architecture, where the VPC network connects satellite locations (spokes) through a single intermediary location (hub) via [VPC Network Peering](https://cloud.google.com/vpc/docs/vpc-peering). -> **NOTE**: This example does not provide full mesh networking, if you need connectivity between spoke networks you will have to create a peering between every pair of spokes, or [use VPN](../hub-and-spoke-vpns). If you use additional peerings be aware of the [VPC Network Peering limits](https://cloud.google.com/vpc/docs/quota#vpc-peering). - -The benefits of this topology include: - -- Network/Security Admin manages Central Services Project (Hub). -- Central services and tools deployed in Central Services Project (Hub) for use by all Service Projects (Spokes). -- Network/Security Admin hands over spoke Projects to respective team who then have full autonomy. -- Network/Security Admin monitors spoke projects for organization security posture compliance using tools like [Forseti](https://forsetisecurity.org/), [CSCC](https://cloud.google.com/security-command-center/) etc deployed in Central Services Project (Hub). -- Spokes communicate with on-prem via VPC Network Peering to transit hub and then over Interconnect or VPN to on-premises (on-premises resources are not included in this sample for obvious reasons). -- This is a decentralized architecture where each spoke project has autonomy to manage all their GCP compute and network resources. - -The purpose of this sample is showing how to wire different [Cloud Foundation Fabric](https://github.com/search?q=topic%3Acft-fabric+org%3Aterraform-google-modules&type=Repositories) modules to create **Hub and Spoke Peerings** network architectures, and as such it is meant to be used for prototyping, or to experiment with networking configurations. Additional best practices and security considerations need to be taken into account for real world usage (eg removal of default service accounts, disabling of external IPs, firewall design, etc). - - -![High-level diagram](diagram.png "High-level diagram") - -## Managed resources and services - -This sample creates several distinct groups of resources: - -- three VPC Networks (hub network and two ppoke networks) -- VPC-level resources (VPC, subnets, firewall rules, etc.) -- one Cloud DNS Private zone in the hub project -- one Cloud DNS Forwarding zone in the hub project -- four Cloud DNS Peering zones (two per each spoke project) -- one Cloud DNS Policy for inbound forwarding -- four VPC Network Peerings (two in hub project and one per each spoke project) - -## Test resources - -A set of test resources are included for convenience, as they facilitate experimenting with different networking configurations (firewall rules, external connectivity via VPN, etc.). They are encapsulated in the `test-resources.tf` file, and can be safely removed as a single unit. - -- two virtual machine instances in hub project (one per each region) -- two virtual machine instances in spoke1 project (one per each region) -- two virtual machine instances in spoke2 project (one per each region) - -SSH access to instances is configured via [OS Login](https://cloud.google.com/compute/docs/oslogin/). External access is allowed via the default SSH rule created by the firewall module, and corresponding `ssh` tags on the instances. - -## Known issues - - It is not possible to get inbound DNS forwarding IPs in the terraform output. - - Please refer to the [bug](https://github.com/terraform-providers/terraform-provider-google/issues/3753) for more details. - - Please refer to the [documentation](https://cloud.google.com/dns/zones/#creating_a_dns_policy_that_enables_inbound_dns_forwarding) on how to get the IPs with `gcloud`. - - -## Variables - -| name | description | type | required | default | -|---|---|:---: |:---:|:---:| -| hub_project_id | Hub Project id. Same project can be used for hub and spokes. | string | ✓ | | -| spoke_1_project_id | Spoke 1 Project id. Same project can be used for hub and spokes. | string | ✓ | | -| spoke_2_project_id | Spoke 2 Project id. Same project can be used for hub and spokes. | string | ✓ | | -| *forwarding_dns_zone_domain* | Forwarding DNS Zone Domain. | string | | on-prem.local. | -| *forwarding_dns_zone_name* | Forwarding DNS Zone Name. | string | | on-prem-local | -| *hub_subnets* | Hub VPC subnets configuration. | map(object({...})) | | ... | -| *on_prem_cidr_range* | On-prem CIDR range for on-prem-in-a-box emulation. | string | | 192.168.192.0/24 | -| *private_dns_zone_domain* | Private DNS Zone Domain. | string | | gcp.local. | -| *private_dns_zone_name* | Private DNS Zone Name. | string | | gcp-local | -| *spoke_1_subnets* | Spoke 1 VPC subnets configuration. | map(object({...})) | | ... | -| *spoke_2_subnets* | Spoke 2 VPC subnets configuration. | map(object({...})) | | ... | - -## Outputs - -| name | description | sensitive | -|---|---|:---:| -| hub | Hub network resources. | | -| spoke-1 | Spoke1 network resources. | | -| spoke-2 | Spoke2 network resources. | | - diff --git a/infrastructure/hub-and-spoke-peerings/locals.tf b/infrastructure/hub-and-spoke-peerings/locals.tf deleted file mode 100644 index 16c5bc3079..0000000000 --- a/infrastructure/hub-and-spoke-peerings/locals.tf +++ /dev/null @@ -1,45 +0,0 @@ -# Copyright 2020 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -locals { - hub_subnet_cidr_ranges = ( - { - for subnet in keys(var.hub_subnets) - : var.hub_subnets[subnet]["ip_cidr_range"] => subnet - } - ) - spoke_1_subnet_cidr_ranges = ( - { - for subnet in keys(var.spoke_1_subnets) - : var.spoke_1_subnets[subnet]["ip_cidr_range"] => subnet - } - ) - spoke_2_subnet_cidr_ranges = ( - { - for subnet in keys(var.spoke_2_subnets) - : var.spoke_2_subnets[subnet]["ip_cidr_range"] => subnet - } - ) - custom_routes_to_spokes = ( - merge(local.spoke_1_subnet_cidr_ranges, local.spoke_2_subnet_cidr_ranges) - ) - all_subnet_cidrs = ( - concat( - keys(local.hub_subnet_cidr_ranges), - keys(local.spoke_1_subnet_cidr_ranges), - keys(local.spoke_2_subnet_cidr_ranges), - [var.on_prem_cidr_range] - ) - ) -} diff --git a/infrastructure/hub-and-spoke-peerings/main.tf b/infrastructure/hub-and-spoke-peerings/main.tf deleted file mode 100644 index c2fda7f8ef..0000000000 --- a/infrastructure/hub-and-spoke-peerings/main.tf +++ /dev/null @@ -1,325 +0,0 @@ -# Copyright 2020 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -############################################################## -# VPCs # -############################################################## - -module "vpc-hub" { - source = "../../modules/net-vpc" - - project_id = var.hub_project_id - name = "hub-network" - subnets = var.hub_subnets - routing_mode = "GLOBAL" -} - -module "vpc-spoke-1" { - source = "../../modules/net-vpc" - - project_id = var.spoke_1_project_id - name = "spoke-1-network" - subnets = var.spoke_1_subnets - routing_mode = "GLOBAL" -} - -module "vpc-spoke-2" { - source = "../../modules/net-vpc" - - project_id = var.spoke_2_project_id - name = "spoke-2-network" - subnets = var.spoke_2_subnets - routing_mode = "GLOBAL" -} - -############################################################## -# Network Peering # -############################################################## - -module "hub-to-spoke-1-peering" { - source = "../../modules/net-vpc-peering" - - local_network = module.vpc-hub.self_link - peer_network = module.vpc-spoke-1.self_link - export_local_custom_routes = true - export_peer_custom_routes = false -} - -module "hub-to-spoke-2-peering" { - source = "../../modules/net-vpc-peering" - - local_network = module.vpc-hub.self_link - peer_network = module.vpc-spoke-2.self_link - export_local_custom_routes = true - export_peer_custom_routes = false - - module_depends_on = [module.hub-to-spoke-1-peering.complete] -} - -############################################################## -# Firewalls # -############################################################## - -module "firewall-hub" { - source = "../../modules/net-vpc-firewall" - - project_id = var.hub_project_id - network = module.vpc-hub.name - admin_ranges_enabled = true - admin_ranges = local.all_subnet_cidrs -} - -module "firewall-spoke-1" { - source = "../../modules/net-vpc-firewall" - - project_id = var.spoke_1_project_id - network = module.vpc-spoke-1.name - admin_ranges_enabled = true - admin_ranges = local.all_subnet_cidrs -} - -module "firewall-spoke-2" { - source = "../../modules/net-vpc-firewall" - - project_id = var.spoke_2_project_id - network = module.vpc-spoke-2.name - admin_ranges_enabled = true - admin_ranges = local.all_subnet_cidrs -} - -############################################################## -# DNS Zones # -############################################################## - -module "hub-private-zone" { - source = "../../modules/dns" - - project_id = var.hub_project_id - type = "private" - name = "${var.private_dns_zone_name}-hub-private" - domain = var.private_dns_zone_domain - client_networks = [module.vpc-hub.self_link] - recordsets = [ - { name = "hub-test-vm", type = "A", ttl = 300, records = module.vm-hub.internal_ips }, - { name = "spoke-1-test-vm", type = "A", ttl = 300, records = module.vm-spoke-1.internal_ips }, - { name = "spoke-2-test-vm", type = "A", ttl = 300, records = module.vm-spoke-2.internal_ips }, - ] -} - -module "hub-forwarding-zone" { - source = "../../modules/dns" - - project_id = var.hub_project_id - type = "forwarding" - name = "${var.forwarding_dns_zone_name}-hub-forwarding" - domain = var.forwarding_dns_zone_domain - - client_networks = [module.vpc-hub.self_link] - forwarders = [module.on-prem-in-a-box.dns_ip_address] -} - -module "spoke-1-peering-zone-to-hub-private-zone" { - source = "../../modules/dns" - - project_id = var.spoke_1_project_id - type = "peering" - name = "${var.private_dns_zone_name}-spoke-1-peering-to-hub-private" - domain = var.private_dns_zone_domain - - client_networks = [module.vpc-spoke-1.self_link] - peer_network = module.vpc-hub.self_link -} - -module "spoke-2-peering-zone-to-hub-private-zone" { - source = "../../modules/dns" - - project_id = var.spoke_2_project_id - type = "peering" - name = "${var.private_dns_zone_name}-spoke-2-peering-to-hub-private" - domain = var.private_dns_zone_domain - - client_networks = [module.vpc-spoke-2.self_link] - peer_network = module.vpc-hub.self_link -} - -module "spoke-1-peering-zone-to-hub-forwarding-zone" { - source = "../../modules/dns" - - project_id = var.spoke_1_project_id - type = "peering" - name = "${var.private_dns_zone_name}-spoke-1-peering-to-hub-forwarding" - domain = var.forwarding_dns_zone_domain - - client_networks = [module.vpc-spoke-1.self_link] - peer_network = module.vpc-hub.self_link -} - -module "spoke-2-peering-zone-to-hub-forwarding-zone" { - source = "../../modules/dns" - - project_id = var.spoke_2_project_id - type = "peering" - name = "${var.private_dns_zone_name}-spoke-2-peering-to-hub-forwarding" - domain = var.forwarding_dns_zone_domain - - client_networks = [module.vpc-spoke-2.self_link] - peer_network = module.vpc-hub.self_link -} - -############################################################## -# TESTING: Inbount DNS Forwarding # -############################################################## - -# TODO Provide resolver addresses in the output once https://github.com/terraform-providers/terraform-provider-google/issues/3753 resolved. -# For now please refer to the documentation on how to get the compute addresses for the DNS Resolver https://cloud.google.com/dns/zones/#creating_a_dns_policy_that_enables_inbound_dns_forwarding -resource "google_dns_policy" "google_dns_policy" { - provider = google-beta - - project = var.hub_project_id - name = "inbound-dns-forwarding-policy" - enable_inbound_forwarding = true - - networks { - network_url = module.vpc-hub.self_link - } -} - -############################################################################### -# TESTING: Hub test VM # -############################################################################### - -module "vm-hub" { - source = "../../modules/compute-vm" - - project_id = var.hub_project_id - region = module.vpc-hub.subnet_regions[keys(var.hub_subnets).0] - zone = "${module.vpc-hub.subnet_regions[keys(var.hub_subnets).0]}-b" - name = "hub-test-vm" - network_interfaces = [{ - network = module.vpc-hub.self_link - subnetwork = module.vpc-hub.subnet_self_links[keys(var.hub_subnets).0], - nat = false, - addresses = null - }] - tags = ["ssh"] - instance_count = 1 -} - -############################################################################### -# TESTING: Spoke 1 test VM # -############################################################################### - -module "vm-spoke-1" { - source = "../../modules/compute-vm" - - project_id = var.spoke_1_project_id - region = module.vpc-spoke-1.subnet_regions[keys(var.spoke_1_subnets).0] - zone = "${module.vpc-spoke-1.subnet_regions[keys(var.spoke_1_subnets).0]}-b" - name = "spoke-1-test-vm" - network_interfaces = [{ - network = module.vpc-spoke-1.self_link - subnetwork = module.vpc-spoke-1.subnet_self_links[keys(var.spoke_1_subnets).0], - nat = false, - addresses = null - }] - tags = ["ssh"] - instance_count = 1 -} - -############################################################################### -# TESTING: Spoke 2 test VM # -############################################################################### - -module "vm-spoke-2" { - source = "../../modules/compute-vm" - - project_id = var.spoke_2_project_id - region = module.vpc-spoke-2.subnet_regions[keys(var.spoke_2_subnets).0] - zone = "${module.vpc-spoke-2.subnet_regions[keys(var.spoke_2_subnets).0]}-b" - name = "spoke-2-test-vm" - network_interfaces = [{ - network = module.vpc-spoke-2.self_link - subnetwork = module.vpc-spoke-2.subnet_self_links[keys(var.spoke_2_subnets).0], - nat = false, - addresses = null - }] - tags = ["ssh"] - instance_count = 1 -} - -############################################################################### -# TESTING: On-Prem-In-a-Box resources # -############################################################################### - -module "hub-to-onprem-cloud-vpn" { - source = "../../modules/net-vpn-dynamic/" - - project_id = var.hub_project_id - region = module.vpc-hub.subnet_regions[keys(var.hub_subnets).0] - network = module.vpc-hub.self_link - name = "hub-net-to-on-prem" - router_asn = 65001 - tunnels = { - remote-1 = { - bgp_peer = { - address = "169.254.0.2" - asn = 65002 - } - bgp_session_range = "169.254.0.1/30" - ike_version = 2 - peer_ip = module.on-prem-in-a-box.external_address - shared_secret = null - bgp_peer_options = { - advertise_groups = ["ALL_SUBNETS"] - advertise_ip_ranges = local.custom_routes_to_spokes - advertise_mode = "CUSTOM" - route_priority = 1000 - } - } - } -} - -module "vpc-on-prem" { - source = "../../modules/net-vpc" - - project_id = var.hub_project_id - name = "network-for-on-prem" - subnets = { - on-prem = { - ip_cidr_range = "172.16.1.0/28" - region = "europe-west1" - secondary_ip_range = {} - } - } -} - -module "on-prem-in-a-box" { - source = "../../modules/on-prem-in-a-box/" - - project_id = var.hub_project_id - zone = "europe-west1-b" - network = module.vpc-on-prem.name - subnet_self_link = module.vpc-on-prem.subnet_self_links["on-prem"] - vpn_gateway_type = "dynamic" - peer_ip = module.hub-to-onprem-cloud-vpn.address - local_ip_cidr_range = var.on_prem_cidr_range - shared_secret = module.hub-to-onprem-cloud-vpn.random_secret - peer_bgp_session_range = "169.254.0.1/30" - local_bgp_session_range = "169.254.0.2/30" - peer_bgp_asn = 65001 - local_bgp_asn = 65002 - on_prem_dns_zone = var.forwarding_dns_zone_domain - cloud_dns_zone = var.private_dns_zone_domain - cloud_dns_forwarder_ip = cidrhost(module.vpc-hub.subnet_ips[keys(var.hub_subnets).0], 2) -} diff --git a/infrastructure/hub-and-spoke-peerings/outputs.tf b/infrastructure/hub-and-spoke-peerings/outputs.tf deleted file mode 100644 index 8a0f89bd06..0000000000 --- a/infrastructure/hub-and-spoke-peerings/outputs.tf +++ /dev/null @@ -1,64 +0,0 @@ -# Copyright 2020 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -output "hub" { - description = "Hub network resources." - value = { - network_name = module.vpc-hub.name - subnet_ips = module.vpc-hub.subnet_ips - subnet_regions = module.vpc-hub.subnet_regions - privte_dns_zone = { - name = module.hub-private-zone.name - domain = module.hub-private-zone.domain - } - forwarding_dns_zone = { - name = module.hub-forwarding-zone.name - domain = module.hub-forwarding-zone.domain - } - } -} - -output "spoke-1" { - description = "Spoke1 network resources." - value = { - network_name = module.vpc-spoke-1.name - subnet_ips = module.vpc-spoke-1.subnet_ips - subnet_regions = module.vpc-spoke-1.subnet_regions - peering_to_hub_private_dns_zone = { - name = module.spoke-1-peering-zone-to-hub-private-zone.name - domain = module.spoke-1-peering-zone-to-hub-private-zone.domain - } - peering_to_hub_forwarding_dns_zone = { - name = module.spoke-1-peering-zone-to-hub-forwarding-zone.name - domain = module.spoke-1-peering-zone-to-hub-forwarding-zone.domain - } - } -} - -output "spoke-2" { - description = "Spoke2 network resources." - value = { - network_name = module.vpc-spoke-2.name - subnet_ips = module.vpc-spoke-2.subnet_ips - subnet_regions = module.vpc-spoke-2.subnet_regions - peering_to_hub_private_dns_zone = { - name = module.spoke-2-peering-zone-to-hub-private-zone.name - domain = module.spoke-2-peering-zone-to-hub-private-zone.domain - } - peering_to_hub_forwarding_dns_zone = { - name = module.spoke-2-peering-zone-to-hub-forwarding-zone.name - domain = module.spoke-2-peering-zone-to-hub-forwarding-zone.domain - } - } -} diff --git a/infrastructure/hub-and-spoke-peerings/terraform.tfvars.sample b/infrastructure/hub-and-spoke-peerings/terraform.tfvars.sample deleted file mode 100644 index cfe767c3d3..0000000000 --- a/infrastructure/hub-and-spoke-peerings/terraform.tfvars.sample +++ /dev/null @@ -1,3 +0,0 @@ -hub_project_id = "automation-examples" -spoke_1_project_id = "automation-examples" -spoke_2_project_id = "automation-examples" diff --git a/infrastructure/hub-and-spoke-peerings/variables.tf b/infrastructure/hub-and-spoke-peerings/variables.tf deleted file mode 100644 index 46e4417aac..0000000000 --- a/infrastructure/hub-and-spoke-peerings/variables.tf +++ /dev/null @@ -1,121 +0,0 @@ -# Copyright 2020 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -variable "hub_project_id" { - description = "Hub Project id. Same project can be used for hub and spokes." - type = string -} - -variable "spoke_1_project_id" { - description = "Spoke 1 Project id. Same project can be used for hub and spokes." - type = string -} - -variable "spoke_2_project_id" { - description = "Spoke 2 Project id. Same project can be used for hub and spokes." - type = string -} - -variable "hub_subnets" { - description = "Hub VPC subnets configuration." - type = map(object({ - ip_cidr_range = string - region = string - secondary_ip_range = map(string) - })) - default = { - hub-subnet-a = { - ip_cidr_range = "10.10.10.0/24" - region = "europe-west1" - secondary_ip_range = {} - } - hub-subnet-b = { - ip_cidr_range = "10.10.20.0/24" - region = "europe-west2" - secondary_ip_range = {} - } - } -} - -variable "spoke_1_subnets" { - description = "Spoke 1 VPC subnets configuration." - type = map(object({ - ip_cidr_range = string - region = string - secondary_ip_range = map(string) - })) - default = { - spoke-1-subnet-a = { - ip_cidr_range = "10.20.10.0/24" - region = "europe-west1" - secondary_ip_range = {} - } - spoke-1-subnet-b = { - ip_cidr_range = "10.20.20.0/24" - region = "europe-west2" - secondary_ip_range = {} - } - } -} - -variable "spoke_2_subnets" { - description = "Spoke 2 VPC subnets configuration." - type = map(object({ - ip_cidr_range = string - region = string - secondary_ip_range = map(string) - })) - default = { - spoke-2-subnet-a = { - ip_cidr_range = "10.30.10.0/24" - region = "europe-west1" - secondary_ip_range = {} - } - spoke-2-subnet-b = { - ip_cidr_range = "10.30.20.0/24" - region = "europe-west2" - secondary_ip_range = {} - } - } -} - -variable "private_dns_zone_name" { - description = "Private DNS Zone Name." - type = string - default = "gcp-local" -} - -variable "private_dns_zone_domain" { - description = "Private DNS Zone Domain." - type = string - default = "gcp.local." -} - -variable "forwarding_dns_zone_name" { - description = "Forwarding DNS Zone Name." - type = string - default = "on-prem-local" -} - -variable "forwarding_dns_zone_domain" { - description = "Forwarding DNS Zone Domain." - type = string - default = "on-prem.local." -} - -variable "on_prem_cidr_range" { - description = "On-prem CIDR range for on-prem-in-a-box emulation." - type = string - default = "192.168.192.0/24" -} diff --git a/infrastructure/hub-and-spoke-vpns/README.md b/infrastructure/hub-and-spoke-vpn/README.md similarity index 100% rename from infrastructure/hub-and-spoke-vpns/README.md rename to infrastructure/hub-and-spoke-vpn/README.md diff --git a/infrastructure/hub-and-spoke-vpns/backend.tf.sample b/infrastructure/hub-and-spoke-vpn/backend.tf.sample similarity index 100% rename from infrastructure/hub-and-spoke-vpns/backend.tf.sample rename to infrastructure/hub-and-spoke-vpn/backend.tf.sample diff --git a/infrastructure/hub-and-spoke-vpns/diagram.png b/infrastructure/hub-and-spoke-vpn/diagram.png similarity index 100% rename from infrastructure/hub-and-spoke-vpns/diagram.png rename to infrastructure/hub-and-spoke-vpn/diagram.png diff --git a/infrastructure/hub-and-spoke-vpns/main.tf b/infrastructure/hub-and-spoke-vpn/main.tf similarity index 100% rename from infrastructure/hub-and-spoke-vpns/main.tf rename to infrastructure/hub-and-spoke-vpn/main.tf diff --git a/infrastructure/hub-and-spoke-vpns/outputs.tf b/infrastructure/hub-and-spoke-vpn/outputs.tf similarity index 100% rename from infrastructure/hub-and-spoke-vpns/outputs.tf rename to infrastructure/hub-and-spoke-vpn/outputs.tf diff --git a/infrastructure/hub-and-spoke-vpns/provider.tf b/infrastructure/hub-and-spoke-vpn/provider.tf similarity index 100% rename from infrastructure/hub-and-spoke-vpns/provider.tf rename to infrastructure/hub-and-spoke-vpn/provider.tf diff --git a/infrastructure/hub-and-spoke-vpns/variables.tf b/infrastructure/hub-and-spoke-vpn/variables.tf similarity index 95% rename from infrastructure/hub-and-spoke-vpns/variables.tf rename to infrastructure/hub-and-spoke-vpn/variables.tf index 88bcbf1080..2d3feaf5fe 100644 --- a/infrastructure/hub-and-spoke-vpns/variables.tf +++ b/infrastructure/hub-and-spoke-vpn/variables.tf @@ -32,6 +32,8 @@ variable "bgp_asn" { } variable "bgp_interface_ranges" { + description = "BGP interface IP CIDR ranges." + type = map(string) default = { spoke-1 = "169.254.1.0/30" spoke-2 = "169.254.1.4/30" diff --git a/infrastructure/onprem-google-access-dns/main.tf b/infrastructure/onprem-google-access-dns/main.tf new file mode 100644 index 0000000000..ae257e683c --- /dev/null +++ b/infrastructure/onprem-google-access-dns/main.tf @@ -0,0 +1,98 @@ +/** + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + bgp_interface_gcp = "${cidrhost(var.bgp_interface_ranges.gcp, 1)}" + bgp_interface_onprem = "${cidrhost(var.bgp_interface_ranges.gcp, 2)}" +} + +data "google_netblock_ip_ranges" "private-googleapis" { + range_type = "private-googleapis" +} + +data "google_netblock_ip_ranges" "dns-forwarders" { + range_type = "dns-forwarders" +} + +module "vpc" { + source = "../../modules/net-vpc" + project_id = var.project_id + name = "to-onprem" + subnets = { + default = { + ip_cidr_range = var.ip_ranges.gcp + region = var.region + secondary_ip_range = {} + } + } +} + +module "vpc-firewall" { + source = "../../modules/net-vpc-firewall" + project_id = var.project_id + network = module.vpc.name + admin_ranges_enabled = true + admin_ranges = values(var.ip_ranges) +} + +module "vpn" { + source = "../../modules/net-vpn-dynamic" + project_id = var.project_id + region = module.vpc.subnet_regions["default"] + network = module.vpc.name + name = "to-onprem" + router_asn = var.bgp_asn.gcp + tunnels = { + onprem = { + bgp_peer = { + address = local.bgp_interface_onprem + asn = var.bgp_asn.onprem + } + bgp_peer_options = { + advertise_groups = ["ALL_SUBNETS"] + advertise_ip_ranges = { + (data.google_netblock_ip_ranges.private-googleapis.cidr_blocks_ipv4.0) = "private-googleapis" + (data.google_netblock_ip_ranges.dns-forwarders.cidr_blocks_ipv4.0) = "dns-forwarders" + } + advertise_mode = "CUSTOM" + route_priority = 1000 + } + bgp_session_range = "${local.bgp_interface_gcp}/30" + ike_version = 2 + peer_ip = module.on-prem.external_address + shared_secret = "" + } + } +} + +module "on-prem" { + source = "../../modules/on-prem-in-a-box/" + project_id = var.project_id + zone = "${var.region}-b" + network = module.vpc.name + subnet_self_link = module.vpc.subnet_self_links.default + vpn_gateway_type = "dynamic" + peer_ip = module.vpn.address + local_ip_cidr_range = var.ip_ranges.onprem + shared_secret = module.vpn.random_secret + peer_bgp_session_range = "${local.bgp_interface_gcp}/30" + local_bgp_session_range = "${local.bgp_interface_gcp}/30" + peer_bgp_asn = var.bgp_asn.gcp + local_bgp_asn = var.bgp_asn.onprem + cloud_dns_zone = var.dns_domains.gcp + on_prem_dns_zone = var.dns_domains.onprem + cloud_dns_forwarder_ip = cidrhost(module.vpc.subnet_ips.default, 2) +} diff --git a/infrastructure/onprem-google-access-dns/outputs.tf b/infrastructure/onprem-google-access-dns/outputs.tf new file mode 100644 index 0000000000..e69de29bb2 diff --git a/infrastructure/onprem-google-access-dns/variables.tf b/infrastructure/onprem-google-access-dns/variables.tf new file mode 100644 index 0000000000..5f659e87bb --- /dev/null +++ b/infrastructure/onprem-google-access-dns/variables.tf @@ -0,0 +1,62 @@ +/** + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "bgp_asn" { + description = "BGP ASNs." + type = map(number) + default = { + gcp = 64513 + onprem = 64514 + } +} + +variable "bgp_interface_ranges" { + description = "BGP interface IP CIDR ranges." + type = map(string) + default = { + gcp = "169.254.1.0/30" + } +} + +variable "dns_domains" { + description = "Private DNS domain names." + type = map(string) + default = { + gcp = "gcp.example.com" + onprem = "onprem.example.com" + } +} + +variable "ip_ranges" { + description = "IP CIDR ranges." + type = map(string) + default = { + gcp = "10.0.0.0/24" + onprem = "10.0.16.0/24" + } +} + +variable "project_id" { + description = "Project id for all resources." + type = string +} + +variable "region" { + description = "VPC region." + type = string + default = "europe-west1" +} + diff --git a/infrastructure/onprem-google-access-dns/versions.tf b/infrastructure/onprem-google-access-dns/versions.tf new file mode 100644 index 0000000000..4eb1500c5a --- /dev/null +++ b/infrastructure/onprem-google-access-dns/versions.tf @@ -0,0 +1,17 @@ +# Copyright 2019 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +terraform { + required_version = ">= 0.12" +} diff --git a/infrastructure/shared-vpc-gke/main.tf b/infrastructure/shared-vpc-gke/main.tf index 5fdb5aa13a..8ab2800ae5 100644 --- a/infrastructure/shared-vpc-gke/main.tf +++ b/infrastructure/shared-vpc-gke/main.tf @@ -195,6 +195,7 @@ module "vm-bastion" { ]) } service_account_create = true + tags = ["ssh"] } ################################################################################ diff --git a/tests/requirements.txt b/tests/requirements.txt index 078c682c71..8259965506 100644 --- a/tests/requirements.txt +++ b/tests/requirements.txt @@ -1,2 +1,2 @@ -pytest>=5.3.5 +pytest>=4.6.0 tftest>=1.5.0