Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serverless networking program #1146

Merged
merged 49 commits into from
Feb 10, 2023
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
c9b076c
Initial commit, Cloud Run service with default URL
juliodiez Jan 24, 2023
aca604e
Merge pull request #3 from GoogleCloudPlatform/master
juliodiez Jan 24, 2023
6c8c033
Add Global Load Balancer in front of Cloud Run
juliodiez Jan 25, 2023
097c9ce
HTTPS frontend with static IP
juliodiez Jan 26, 2023
417584f
Default config renamed to default-url config
juliodiez Jan 26, 2023
6039105
Get rid of default-url config and make it 'default'
juliodiez Jan 26, 2023
b25afb5
Fix format
juliodiez Jan 26, 2023
5004f17
Add custom-url configuration
juliodiez Jan 26, 2023
1d3449d
Add custom domain as variable
juliodiez Jan 26, 2023
914a8c4
Rename output URL as default URL
juliodiez Jan 30, 2023
e10c6ed
Output URL as single string
juliodiez Jan 30, 2023
cca1d9f
Output the LB IP address
juliodiez Jan 30, 2023
93276d0
Add custom domain output
juliodiez Jan 30, 2023
865a98e
Add ingress settings blocking requests not coming from the LB
juliodiez Jan 30, 2023
8a7aa39
Add purpose to config files
juliodiez Jan 30, 2023
4c40180
Add security configuration using Cloud Armor in the LB
juliodiez Jan 30, 2023
4c2e31b
Rename config file
juliodiez Jan 30, 2023
b95b801
Merge pull request #5 from GoogleCloudPlatform/master
juliodiez Feb 1, 2023
462ca56
Merge pull request #6 from juliodiez/master
juliodiez Feb 1, 2023
63bae85
Identity-Aware Proxy configuration
juliodiez Feb 2, 2023
616125c
Enable service APIs in the project
juliodiez Feb 3, 2023
54a0ff8
Info comment for support_email
juliodiez Feb 3, 2023
8899412
Separate var iap_email to override it on the CLI
juliodiez Feb 3, 2023
60c9f2c
Merge branch 'serverless-program' of https://github.com/juliodiez/clo…
juliodiez Feb 3, 2023
642fff0
Get rid of config files
juliodiez Feb 7, 2023
8b41008
Add README file with the use cases
juliodiez Feb 8, 2023
a44759d
Fix typos in README
juliodiez Feb 8, 2023
3e7d8c5
Infer LB creation from custom_domain var
juliodiez Feb 9, 2023
d9791a3
Merge pull request #7 from GoogleCloudPlatform/master
juliodiez Feb 10, 2023
e8303e1
Merge pull request #8 from juliodiez/master
juliodiez Feb 10, 2023
a199414
Add license boilerplate
juliodiez Feb 10, 2023
a8de908
Alphabetical ordering for outputs and variables
juliodiez Feb 10, 2023
3935867
Simplify conditional expressions
juliodiez Feb 10, 2023
d916632
Update URL to repo
juliodiez Feb 10, 2023
f31293b
Specify block types
juliodiez Feb 10, 2023
aca3069
Add quotes around var project_id
juliodiez Feb 10, 2023
d19d434
Add HTML tags to generate output and variable tables
juliodiez Feb 10, 2023
4aa135f
Generated output and variable tables via tfdoc.py
juliodiez Feb 10, 2023
9fa48ea
Add ending periods to descriptions
juliodiez Feb 10, 2023
1335fd6
Update output and variable tables via tfdoc.py
juliodiez Feb 10, 2023
98f9a31
Update output and variable tables via tfdoc.py
juliodiez Feb 10, 2023
54d88c7
Merge branch 'master' into serverless-program
juliodiez Feb 10, 2023
eee0a0e
Merge pull request #9 from GoogleCloudPlatform/master
juliodiez Feb 10, 2023
9dab5fa
Merge pull request #10 from juliodiez/master
juliodiez Feb 10, 2023
6c470ec
Option to create a project
juliodiez Feb 10, 2023
e72602a
Update variables table via tfdoc.py
juliodiez Feb 10, 2023
621ee6e
Update variables table via tfdoc.py
juliodiez Feb 10, 2023
5c3fd31
Add missing newlines in README
juliodiez Feb 10, 2023
0cfd8dd
Move all images to an images/ folder
juliodiez Feb 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions blueprints/serverless/cloud-run-explore/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Cloud Run Explore

## Introduction

This blueprint contains all the necessary Terraform modules to build and publicly expose a Cloud Run service in a variety of use cases.

The content of this blueprint corresponds to the chapter '_My serverless "Hello, World! - Exploring Cloud Run_' of the __Serverless Networking Guide__ (to be released soon). This guide is an easy to follow introduction to Cloud Run, where a couple of friendly characters will guide you from the basics to more advanced topics with a very practical approach and in record time! The code here complements this learning and allows you to test the scenarios presented and your knowledge.
juliodiez marked this conversation as resolved.
Show resolved Hide resolved

## Architecture

The following diagram depicts the main components that this blueprint will set up:

<p align="center"> <img src="architecture.png" width="700"> </p>

The following products or features are used to fulfill the different use cases covered in this blueprint (to learn more about them click on the hyperlinks):

* [Cloud Run](https://cloud.google.com/run/docs/overview/what-is-cloud-run) - Cloud Run is a managed compute platform that lets you run containers directly on top of Google's scalable infrastructure.
* [Cloud Run Ingress Settings](https://cloud.google.com/run/docs/securing/ingress) - feature that restricts network access to your Cloud Run service. At a network level, by default, any resource on the Internet can reach your Cloud Run service on its run.app URL or at a custom domain set up in Cloud Run. You can change this default by specifying a different setting for its ingress. All ingress paths, including the default run.app URL, are subject to your ingress setting. Ingress is set at the service level. The following settings are available:
* __Internal__: Allows requests from VPC networks that are in the same project or VPC Service Controls perimeter as your Cloud Run service.
* __Internal and Cloud Load Balancing__: Allows requests from resources allowed by the more restrictive Internal setting and an External HTTP(S) load balancer.
* __All__ (default): Allows all requests, including requests directly from the Internet to the default run.app URL.
* [Google Cloud Load Balancer](https://cloud.google.com/run/docs/mapping-custom-domains#https-load-balancer) - When an HTTP(S) load balancer is enabled for Cloud Run, you can reach your serverless app through a custom domain mapped to a single dedicated global Anycast IP address that is not shared with other services.
* [Cloud Armor](https://cloud.google.com/armor) - Google Cloud Armor is the web-application firewall (WAF) and DDoS mitigation service that helps users defend their web apps and services at Google scale at the edge of Google’s network.
* [Identity Aware Proxy](https://cloud.google.com/iap/docs/concepts-overview) - IAP lets you establish a central authorization layer for applications accessed by HTTPS, so you can use an application-level access control model instead of relying on network-level firewalls. [External Load Balancing with IAP](https://cloud.google.com/iap/docs/load-balancer-howto) is supported for Cloud Run with Serverless NEGs.
* [Cloud CDN](https://cloud.google.com/cdn) - Configure fast, reliable web and video content delivery with global scale and reach. __Note__: Cloud CDN is not part of this blueprint yet.

## Prerequisites

You will need an existing [project](https://cloud.google.com/resource-manager/docs/creating-managing-projects) with [billing enabled](https://cloud.google.com/billing/docs/how-to/modify-project) and a user with the “Project owner” [IAM](https://cloud.google.com/iam) role on that project. __Note__: to grant a user a role, take a look at the [Granting and Revoking Access](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) documentation.

## Spinning up the architecture
### General steps
juliodiez marked this conversation as resolved.
Show resolved Hide resolved

1. Clone the repo to your local machine or Cloud Shell:
```
git clone https://github.com/juliodiez/cloud-foundation-fabric
juliodiez marked this conversation as resolved.
Show resolved Hide resolved
git checkout serverless-program
```

2. Change to the directory of the blueprint:
```
cd cloud-foundation-fabric/blueprints/serverless/cloud-run-explore
```
You should see this README and some terraform files.

3. To deploy a specific use case, you will need to create a file in this directory called `terraform.tfvars` and follow the corresponding instructions to set variables. Sometimes values that are meant to be substituted will be shown inside brackets but you need to omit these brackets. E.g.:
```
project_id = [your-project_id]
```
may become
```
juliodiez marked this conversation as resolved.
Show resolved Hide resolved
project_id = spiritual-hour-331417
```

Although each use case is somehow built around the previous one they are self-contained so you can deploy any of them at will.

4. The usual terraform commands will do the work:
```
terraform init
terraform plan
terraform apply
```

The resource creation will take a few minutes but when it’s complete, you should see an output stating the command completed successfully with a list of the created resources, and some output variables with information to access your service.

__Congratulations!__ You have successfully deployed the use case you chose based on the variables configuration.

### Use case 1: Cloud Run service with default URL
This is the simplest case, the "Hello World" for Cloud Run. A Cloud Run service is deployed with a default URL based in your project, service name and cloud region where it is deployed:
juliodiez marked this conversation as resolved.
Show resolved Hide resolved

<p align="center"> <img src="use-case-1.png" width="700"> </p>

In this case the only variable that you need to set in `terraform.tfvars` is the project ID:
```
project_id = [your-project-id]
```
Alternatively you can pass this value on the command line:
```
terraform apply -var project_id=[your-project-id]
```

The default URL is automatically created and shown as a terraform output variable. It will be similar to the one shown in the picture above. Now use your browser to visit it, you should see the following:

<p align="center"> <img src="service-running.png" width="700"> </p>

### Use case 2: Cloud Run service with custom domain
If you want to use your own custom domain you need a GCLB in front of your Cloud Run app:
juliodiez marked this conversation as resolved.
Show resolved Hide resolved

<p align="center"> <img src="use-case-2.png" width="700"> </p>

The following values will need to be set in `terraform.tfvars`, replacing the custom_domain value with your own domain:
```
project_id = [your-project-id]
custom_domain = "cloud-run-explore.example.org"
```
Since it is an HTTPS connection a Google managed certificate is created, but for it to be provisioned correctly you will need to point to the load balancer IP address with an A DNS record at your registrar: [Use Google-managed SSL certificates | Load Balancing](https://cloud.google.com/load-balancing/docs/ssl-certificates/google-managed-certs#update-dns). The LB IP is shown as a terraform output.

Be aware that in this case the Cloud Run service can also be reached through the default URL. To limit access only through the custom domain see the next use case.

### Use case 3: Cloud Run service exposed only via custom domain
To block access to the default URL, you can configure Ingress Settings so that Internet requests will be accepted only if they come through the Load Balancer:

<p align="center"> <img src="use-case-3.png" width="700"> </p>

You only need to set one more value in the previous `terraform.tfvars` file:
```
project_id = [your-project-id]
custom_domain = "cloud-run-explore.example.org"
ingress_settings = "internal-and-cloud-load-balancing"
```

The default URL is still created but if you try to visit it, you should see a forbidden error:
<p align="center"> <img src="forbidden.png" width="700"> </p>

### Use case 4: Cloud Run service protected by Cloud Armor
To use Cloud Armor to protect the Cloud Run service, you need to create a security policy to enforce in the load balancer:
<p align="center"> <img src="use-case-4.png" width="700"> </p>

The code allows to block a list of IPs and a specific URL path. For example, you may want to block access to a login page to external users. To test its behavior, by default all IPs and the path `"/login.html"` are blocked, but you can override any of these settings with your own values:
```
project_id = [your-project-id]
custom_domain = "cloud-run-explore.example.org"
ingress_settings = "internal-and-cloud-load-balancing"
security_policy = {
enabled = true
ip_blacklist = ["79.149.0.0/16"]
path_blocked = "/admin.html"
}
```

Note that to avoid users to bypass the Cloud Armor policy you need to block access through the default URL. Ingress settings is configured to do that.

### Use case 5: Cloud Run service protected by Cloud Armor and Identity-Aware Proxy
You can enable IAP at the load balancer to control access using identity and context:
<p align="center"> <img src="use-case-5.png" width="700"> </p>
Use your own email as identity to access the Cloud Run service:

```
project_id = [your-project-id]
custom_domain = "cloud-run-explore.example.org"
ingress_settings = "internal-and-cloud-load-balancing"
security_policy = {
enabled = true
ip_blacklist = ["79.149.0.0/16"]
}
iap = {
enabled = true
email = "[email protected]"
}
```
When visiting it you may be redirected to login with Google. You can use an incognito window to test this behavior.

## Cleaning up your environment
The easiest way to remove all the deployed resources is to run the following command:
```
terraform destroy
```
The above command will delete the associated resources so there will be no billable charges made afterwards. IAP Brands, though, can only be created once per project and not deleted. Destroying a Terraform-managed IAP Brand will remove it from state but will not delete it from Google Cloud.
juliodiez marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
163 changes: 163 additions & 0 deletions blueprints/serverless/cloud-run-explore/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
locals {
juliodiez marked this conversation as resolved.
Show resolved Hide resolved
gclb_create = var.custom_domain == null ? false : true
}

module "project" {
source = "../../../modules/project"
project_create = false
name = var.project_id
services = [
"run.googleapis.com",
"compute.googleapis.com",
"iap.googleapis.com"
]
}

# Cloud Run service
module "cloud_run" {
source = "../../../modules/cloud-run"
project_id = module.project.project_id
name = var.run_svc_name
region = var.region
containers = [{
image = var.image
options = null
ports = null
resources = null
volume_mounts = null
}]
iam = {
"roles/run.invoker" = ["allUsers"]
}
ingress_settings = var.ingress_settings
}

# Reserved static IP for the Load Balancer
resource "google_compute_global_address" "default" {
count = local.gclb_create ? 1 : 0
project = module.project.project_id
name = "glb-ip"
}

# Global L7 HTTPS Load Balancer in front of Cloud Run
module "glb" {
source = "../../../modules/net-glb"
count = local.gclb_create ? 1 : 0
project_id = module.project.project_id
name = "glb"
address = google_compute_global_address.default[0].address
backend_service_configs = {
default = {
backends = [
{ backend = "neg-0" }
]
health_checks = []
port_name = "http"
security_policy = try(google_compute_security_policy.policy[0].name,
null)
iap_config = try({
oauth2_client_id = google_iap_client.iap_client[0].client_id,
oauth2_client_secret = google_iap_client.iap_client[0].secret
}, null)
}
}
health_check_configs = {}
neg_configs = {
neg-0 = {
cloudrun = {
region = var.region
target_service = {
name = var.run_svc_name
}
}
}
}
protocol = "HTTPS"
ssl_certificates = {
managed_configs = {
default = {
domains = [var.custom_domain]
}
}
}
}

# Cloud Armor configuration
resource "google_compute_security_policy" "policy" {
count = local.gclb_create ? (var.security_policy.enabled ? 1 : 0) : 0
name = "cloud-run-policy"
project = module.project.project_id
rule {
action = "deny(403)"
priority = 1000
match {
versioned_expr = "SRC_IPS_V1"
config {
src_ip_ranges = var.security_policy.ip_blacklist
}
}
description = "Deny access to list of IPs"
}
rule {
action = "deny(403)"
priority = 900
match {
expr {
expression = "request.path.matches(\"${var.security_policy.path_blocked}\")"
}
}
description = "Deny access to specific URL paths"
}
rule {
action = "allow"
priority = "2147483647"
match {
versioned_expr = "SRC_IPS_V1"
config {
src_ip_ranges = ["*"]
}
}
description = "Default rule"
}
}

# Identity-Aware Proxy (IAP) or OAuth brand (see OAuth consent screen)
# Note:
# Only "Organization Internal" brands can be created programmatically
# via API. To convert it into an external brand please use the GCP
# Console.
# Brands can only be created once for a Google Cloud project and the
# underlying Google API doesn't support DELETE or PATCH methods.
# Destroying a Terraform-managed Brand will remove it from state but
# will not delete it from Google Cloud.
resource "google_iap_brand" "iap_brand" {
count = local.gclb_create ? (var.iap.enabled ? 1 : 0) : 0
juliodiez marked this conversation as resolved.
Show resolved Hide resolved
project = module.project.project_id
# Support email displayed on the OAuth consent screen. The caller must be
# the user with the associated email address, or if a group email is
# specified, the caller can be either a user or a service account which
# is an owner of the specified group in Cloud Identity.
support_email = var.iap.email
application_title = var.iap.app_title
}

# IAP owned OAuth2 client
# Note:
# Only internal org clients can be created via declarative tools.
# External clients must be manually created via the GCP console.
# Warning:
# All arguments including secret will be stored in the raw state as plain-text.
resource "google_iap_client" "iap_client" {
count = local.gclb_create ? (var.iap.enabled ? 1 : 0) : 0
display_name = var.iap.oauth2_client_name
brand = google_iap_brand.iap_brand[0].name
}

# IAM policy for IAP
# For simplicity we use the same email as support_email and authorized member
resource "google_iap_web_iam_member" "iap_iam" {
count = local.gclb_create ? (var.iap.enabled ? 1 : 0) : 0
project = module.project.project_id
role = "roles/iap.httpsResourceAccessor"
member = "user:${var.iap.email}"
}
16 changes: 16 additions & 0 deletions blueprints/serverless/cloud-run-explore/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
output "default_URL" {
description = "Cloud Run service default URL"
value = module.cloud_run.service.status[0].url
}

output "load_balancer_ip" {
description = "LB IP that forwards to Cloud Run service"
value = local.gclb_create ? module.glb[0].address : "none"
}

# Custom domain for the Load Balancer. I'd prefer getting the value from the
# SSL certificate but it is not exported as output
output "custom_domain" {
description = "Custom domain for the Load Balancer"
value = local.gclb_create ? var.custom_domain : "none"
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
55 changes: 55 additions & 0 deletions blueprints/serverless/cloud-run-explore/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
variable "project_id" {
juliodiez marked this conversation as resolved.
Show resolved Hide resolved
description = "Project ID"
type = string
}

variable "region" {
description = "Cloud region where resource will be deployed"
type = string
default = "europe-west1"
}

variable "run_svc_name" {
description = "Cloud Run service name"
type = string
default = "hello"
}

variable "image" {
description = "Container image to deploy"
type = string
default = "us-docker.pkg.dev/cloudrun/container/hello"
}

variable "custom_domain" {
description = "Custom domain for the Load Balancer"
type = string
default = null
}

variable "ingress_settings" {
description = "Ingress traffic sources allowed to call the service"
type = string
default = "all"
}

variable "security_policy" {
description = "Security policy (Cloud Armor) to enforce in the LB"
type = object({
enabled = optional(bool, false)
ip_blacklist = optional(list(string), ["*"])
path_blocked = optional(string, "/login.html")
})
default = {}
}

variable "iap" {
description = "Identity-Aware Proxy for Cloud Run in the LB"
type = object({
enabled = optional(bool, false)
app_title = optional(string, "Cloud Run Explore Application")
oauth2_client_name = optional(string, "Test Client")
email = optional(string)
})
default = {}
}