Skip to content

Commit

Permalink
Merge pull request #1375 from bluPhy/master
Browse files Browse the repository at this point in the history
Several updates
  • Loading branch information
juliocc authored May 15, 2023
2 parents 8540710 + 11f7f3e commit 5bd3aa4
Show file tree
Hide file tree
Showing 60 changed files with 78 additions and 78 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -981,7 +981,7 @@ tests:
# run a test named `test-plan`, load the specified tfvars files
# use the default inventory file of `test-plan.yaml`
test-plan:
tfvars: # if ommited, we load test-plan.tfvars by default
tfvars: # if omitted, we load test-plan.tfvars by default
- test-plan.tfvars
- test-plan-extra.tfvars
inventory:
Expand All @@ -991,7 +991,7 @@ tests:
# extra_files:
# - ../plugin-x/*.tf

# You can ommit the tfvars and inventory sections and they will
# You can omit the tfvars and inventory sections and they will
# default to the name of the test. The following two examples are equivalent:
#
# test-plan2:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The architecture is the one depicted below.

![Diagram](diagram.png)

To emulate an service deployed on-premise, we have used a managed instance group of instances running Nginx exposed via a regional internalload balancer (L7). The service is accesible through VPN.
To emulate an service deployed on-premise, we have used a managed instance group of instances running Nginx exposed via a regional internalload balancer (L7). The service is accessible through VPN.

## Running the blueprint

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ A few configuration values for the function which are relevant to this example c

## Discovery configuration

Discovery configuration is done via the `discovery_config` variable, which mimicks the set of options available when running the discovery tool in cli mode. Pay particular care in defining the right top-level scope via the `discovery_root` attribute, as this is the root of the hierarchy used to discover Compute resources and it needs to include the individual folders and projects that needs to be monitored, which are defined via the `monitored_folders` and `monitored_projects` attributes.
Discovery configuration is done via the `discovery_config` variable, which mimics the set of options available when running the discovery tool in cli mode. Pay particular care in defining the right top-level scope via the `discovery_root` attribute, as this is the root of the hierarchy used to discover Compute resources and it needs to include the individual folders and projects that needs to be monitored, which are defined via the `monitored_folders` and `monitored_projects` attributes.

The following schematic diagram of a resource hierarchy illustrates the interplay between root scope and monitored resources. The root scope is set to the top-level red folder and completely encloses every resource that needs to be monitored. The blue folder and project are set as monitored defining the actual perimeter used to discover resources. Note that setting the root scope to the blue folder would have resulted in the rightmost project being excluded.

Expand Down Expand Up @@ -64,7 +64,7 @@ dashboard_json_path = "../dashboards/quotas-utilization.json"

| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [discovery_config](variables.tf#L48) | Discovery configuration. Discovery root is the organization or a folder. If monitored folders and projects are empy, every project under the discovery root node will be monitored. | <code title="object&#40;&#123;&#10; discovery_root &#61; string&#10; monitored_folders &#61; list&#40;string&#41;&#10; monitored_projects &#61; list&#40;string&#41;&#10; custom_quota_file &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> || |
| [discovery_config](variables.tf#L48) | Discovery configuration. Discovery root is the organization or a folder. If monitored folders and projects are empty, every project under the discovery root node will be monitored. | <code title="object&#40;&#123;&#10; discovery_root &#61; string&#10; monitored_folders &#61; list&#40;string&#41;&#10; monitored_projects &#61; list&#40;string&#41;&#10; custom_quota_file &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> || |
| [project_id](variables.tf#L100) | Project id where the Cloud Function will be deployed. | <code>string</code> || |
| [bundle_path](variables.tf#L17) | Path used to write the intermediate Cloud Function code bundle. | <code>string</code> | | <code>&#34;.&#47;bundle.zip&#34;</code> |
| [cloud_function_config](variables.tf#L23) | Optional Cloud Function configuration. | <code title="object&#40;&#123;&#10; bucket_name &#61; optional&#40;string&#41;&#10; build_worker_pool_id &#61; optional&#40;string&#41;&#10; bundle_path &#61; optional&#40;string, &#34;.&#47;bundle.zip&#34;&#41;&#10; debug &#61; optional&#40;bool, false&#41;&#10; memory_mb &#61; optional&#40;number, 256&#41;&#10; source_dir &#61; optional&#40;string, &#34;..&#47;src&#34;&#41;&#10; timeout_seconds &#61; optional&#40;number, 540&#41;&#10; vpc_connector &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; egress_settings &#61; optional&#40;string, &#34;ALL_TRAFFIC&#34;&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ variable "dashboard_json_path" {
}

variable "discovery_config" {
description = "Discovery configuration. Discovery root is the organization or a folder. If monitored folders and projects are empy, every project under the discovery root node will be monitored."
description = "Discovery configuration. Discovery root is the organization or a folder. If monitored folders and projects are empty, every project under the discovery root node will be monitored."
type = object({
discovery_root = string
monitored_folders = list(string)
Expand Down
2 changes: 1 addition & 1 deletion blueprints/cloud-operations/quota-monitoring/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Labels are set with project id (which may differ from the monitoring workspace p

<img src="explorer.png" width="640px" alt="GCP Metrics Explorer, usage, limit and utilization view sample">

The solution can also create a basic monitoring alert policy, to demonstrate how to raise alerts when quotas utilization goes over a predefined threshold, to enable it, set variable `alert_create` to true and reapply main.tf after main.py has run at least one and quota monitoring metrics have been creaed.
The solution can also create a basic monitoring alert policy, to demonstrate how to raise alerts when quotas utilization goes over a predefined threshold, to enable it, set variable `alert_create` to true and reapply main.tf after main.py has run at least one and quota monitoring metrics have been created.

## Running the blueprint

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ module "sa-tfc" {

iam = {
# We allow only tokens generated by a specific TFC workspace impersonation of the service account,
# that way one identity pool can be used for a TFC Organization, but every workspace will be able to impersonate only a specifc SA
# that way one identity pool can be used for a TFC Organization, but every workspace will be able to impersonate only a specific SA
"roles/iam.workloadIdentityUser" = ["principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.tfc-pool.name}/attribute.terraform_workspace_id/${var.tfc_workspace_id}"]
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ func getEnv(key, fallback string) string {
return fallback
}

// GetConfiguration generates configration by reading ENV variables.
// GetConfiguration generates configuration by reading ENV variables.
func GetConfiguration() (*Configuration, error) {
timeout, err := time.ParseDuration(getEnv("TIMEOUT", "1000ms"))
if err != nil {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ This sample creates\updates several distinct groups of resources:
|---|---|:---:|:---:|:---:|
| [migration_admin_users](variables.tf#L15) | List of users authorized to create a new M4CE sources and perform all other migration operations, in IAM format. | <code>list&#40;string&#41;</code> || |
| [migration_target_projects](variables.tf#L20) | List of target projects for m4ce workload migrations. | <code>list&#40;string&#41;</code> || |
| [migration_viewer_users](variables.tf#L25) | List of users authorized to retrive information about M4CE in the Google Cloud Console, in IAM format. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [migration_viewer_users](variables.tf#L25) | List of users authorized to retrieve information about M4CE in the Google Cloud Console, in IAM format. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [project_create](variables.tf#L31) | Parameters for the creation of the new project to host the M4CE backend. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [project_name](variables.tf#L40) | Name of an existing project or of the new project assigned as M4CE host project. | <code>string</code> | | <code>&#34;m4ce-host-project-000&#34;</code> |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ variable "migration_target_projects" {
}

variable "migration_viewer_users" {
description = "List of users authorized to retrive information about M4CE in the Google Cloud Console, in IAM format."
description = "List of users authorized to retrieve information about M4CE in the Google Cloud Console, in IAM format."
type = list(string)
default = []
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ This sample creates\update several distinct groups of resources:
| [migration_admin_users](variables.tf#L15) | List of users authorized to create a new M4CE sources and perform all other migration operations, in IAM format. | <code>list&#40;string&#41;</code> || |
| [migration_target_projects](variables.tf#L20) | List of target projects for m4ce workload migrations. | <code>list&#40;string&#41;</code> || |
| [sharedvpc_host_projects](variables.tf#L45) | List of host projects that share a VPC with the selected target projects. | <code>list&#40;string&#41;</code> || |
| [migration_viewer_users](variables.tf#L25) | List of users authorized to retrive information about M4CE in the Google Cloud Console, in IAM format. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [migration_viewer_users](variables.tf#L25) | List of users authorized to retrieve information about M4CE in the Google Cloud Console, in IAM format. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [project_create](variables.tf#L30) | Parameters for the creation of the new project to host the M4CE backend. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [project_name](variables.tf#L39) | Name of an existing project or of the new project assigned as M4CE host project. | <code>string</code> | | <code>&#34;m4ce-host-project-000&#34;</code> |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ variable "migration_target_projects" {
}

variable "migration_viewer_users" {
description = "List of users authorized to retrive information about M4CE in the Google Cloud Console, in IAM format."
description = "List of users authorized to retrieve information about M4CE in the Google Cloud Console, in IAM format."
type = list(string)
default = []
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ This sample creates several distinct groups of resources:
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [migration_admin_users](variables.tf#L15) | List of users authorized to create a new M4CE sources and perform all other migration operations, in IAM format. | <code>list&#40;string&#41;</code> || |
| [migration_viewer_users](variables.tf#L20) | List of users authorized to retrive information about M4CE in the Google Cloud Console, in IAM format. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [migration_viewer_users](variables.tf#L20) | List of users authorized to retrieve information about M4CE in the Google Cloud Console, in IAM format. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [project_create](variables.tf#L26) | Parameters for the creation of the new project to host the M4CE backend. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [project_name](variables.tf#L35) | Name of an existing project or of the new project assigned as M4CE host an target project. | <code>string</code> | | <code>&#34;m4ce-host-project-000&#34;</code> |
| [vpc_config](variables.tf#L41) | Parameters to create a simple VPC on the M4CE project. | <code title="object&#40;&#123;&#10; ip_cidr_range &#61; string,&#10; region &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; ip_cidr_range &#61; &#34;10.200.0.0&#47;20&#34;,&#10; region &#61; &#34;us-west2&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ variable "migration_admin_users" {
}

variable "migration_viewer_users" {
description = "List of users authorized to retrive information about M4CE in the Google Cloud Console, in IAM format."
description = "List of users authorized to retrieve information about M4CE in the Google Cloud Console, in IAM format."
type = list(string)
default = []
}
Expand Down
2 changes: 1 addition & 1 deletion blueprints/data-solutions/bq-ml/demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ In this tutorial we will also see how to make explainable predictions, in order

# Dataset

This tutorial uses a fictitious e-commerce dataset collecting programmatically generated data from the fictitious e-commerce store called The Look. The dataset is publicy available on BigQuery at this location `bigquery-public-data.thelook_ecommerce`.
This tutorial uses a fictitious e-commerce dataset collecting programmatically generated data from the fictitious e-commerce store called The Look. The dataset is publicly available on BigQuery at this location `bigquery-public-data.thelook_ecommerce`.

# Goal

Expand Down
4 changes: 2 additions & 2 deletions blueprints/data-solutions/bq-ml/demo/bmql_pipeline.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@
"outputs": [],
"source": [
"# deploy the BigQuery ML model on Vertex Endpoint\n",
"# have a coffe - this step can take up 10/15 minutes to finish\n",
"# have a coffee - this step can take up 10/15 minutes to finish\n",
"model.deploy(endpoint=endpoint, deployed_model_display_name='bqml-deployed-model')"
]
},
Expand Down Expand Up @@ -436,7 +436,7 @@
"\n",
"Thanks to this tutorial we were able to:\n",
"- Define a re-usable Vertex AI pipeline to train and evaluate BQ ML models\n",
"- Use a Vertex AI Experiment to keep track of multiple trainings for the same model with different paramenters (in this case a different split for train/test data)\n",
"- Use a Vertex AI Experiment to keep track of multiple trainings for the same model with different parameters (in this case a different split for train/test data)\n",
"- Deploy the preferred model on a Vertex AI managed Endpoint in order to serve the model for real-time use cases via API\n",
"- Make batch prediction via Big Query and see what are the top 5 features which influenced the algorithm output"
]
Expand Down
4 changes: 2 additions & 2 deletions blueprints/data-solutions/cloudsql-multiregion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ Once you have the required information, head back to the cloud shell editor. Mak

Configure the Terraform variables in your `terraform.tfvars` file. You need to specify at least the `project_id` and `prefix` variables. See [`terraform.tfvars.sample`](terraform.tfvars.sample) as starting point.

![Deploy ressources](images/image2.png)
![Deploy resources](images/image2.png)

Run Terraform init:

Expand All @@ -71,7 +71,7 @@ terraform apply

The resource creation will take a few minutes, at the end this is the output you should expect for successful completion along with a list of the created resources:

![Ressources installed](images/image3.png)
![Resources installed](images/image3.png)

## Move to real use case consideration

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# tfdoc:file:description Trasformation project and VPC.
# tfdoc:file:description Transformation project and VPC.

locals {
iam_trf = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

class ParseRow(beam.DoFn):
"""
Splits a given csv row by a seperator, validates fields and returns a dict
Splits a given csv row by a separator, validates fields and returns a dict
structure compatible with the BigQuery transform
"""

Expand Down
6 changes: 3 additions & 3 deletions blueprints/data-solutions/data-playground/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ As is often the case in real-world configurations, this blueprint accepts as inp

If the network_config variable is not provided, one VPC will be created in each project that supports network resources (load, transformation and orchestration).

## Deploy your enviroment
## Deploy your environment

We assume the identiy running the following steps has the following role:

Expand All @@ -35,7 +35,7 @@ Run Terraform init:
terraform init
```

Configure the Terraform variable in your terraform.tfvars file. You need to spefify at least the following variables:
Configure the Terraform variable in your terraform.tfvars file. You need to specify at least the following variables:

```
prefix = "prefix"
Expand All @@ -48,7 +48,7 @@ You can run now:
terraform apply
```

You can now connect to the Vertex AI notbook to perform your data analysy.
You can now connect to the Vertex AI notbook to perform your data analysis.
<!-- BEGIN TFDOC -->

## Variables
Expand Down
2 changes: 1 addition & 1 deletion blueprints/data-solutions/data-playground/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ variable "region" {
default = "europe-west1"
}

variable "service_encryption_keys" { # service encription key
variable "service_encryption_keys" { # service encryption key
description = "Cloud KMS to use to encrypt different services. Key location should match service region."
type = object({
bq = string
Expand Down
2 changes: 1 addition & 1 deletion blueprints/factories/bigquery-factory/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ You can create as many files as you like, the code will loop through it and crea

### Terraform code

In this section we show how to create tables and views from a file structure simlar to the one shown below.
In this section we show how to create tables and views from a file structure similar to the one shown below.
```bash
bigquery
Expand Down
Loading

0 comments on commit 5bd3aa4

Please sign in to comment.