Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FAST stage docs cleanup #1145

Merged
merged 12 commits into from
Feb 15, 2023
Merged

FAST stage docs cleanup #1145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
1f3b5c5
top-level and stage 0
ludoo Feb 9, 2023
e3d0719
Merge branch 'master' into ludo/fast-docs
ludoo Feb 9, 2023
c2c0d3c
stage 1
ludoo Feb 9, 2023
5973b5a
net peering
ludoo Feb 10, 2023
359fac5
networking
ludoo Feb 14, 2023
d1f1055
networking
ludoo Feb 14, 2023
b0d34b4
security
ludoo Feb 14, 2023
8864459
gke, dp
ludoo Feb 14, 2023
145ff40
Merge branch 'master' into ludo/fast-docs
ludoo Feb 14, 2023
841133b
checks
ludoo Feb 14, 2023
a1be042
Merge branch 'ludo/fast-docs' of github.com:GoogleCloudPlatform/cloud…
ludoo Feb 14, 2023
e1cb9a3
Merge branch 'master' into ludo/fast-docs
ludoo Feb 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions fast/stage-links.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ case $STAGE_NAME in
TFVARS="tfvars/0-bootstrap.auto.tfvars.json
tfvars/1-resman.auto.tfvars.json"
;;
"2-security"*)
PROVIDER="providers/2-security-providers.tf"
TFVARS="tfvars/0-bootstrap.auto.tfvars.json
tfvars/1-resman.auto.tfvars.json"
;;
*)
# check for a "dev" stage 3
echo "no stage found, trying for parent stage 3..."
Expand Down
111 changes: 62 additions & 49 deletions fast/stages/0-bootstrap/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,28 @@ Use the following diagram as a simple high level reference for the following sec
<img src="diagram.svg" alt="Organization-level diagram">
</p>

## Table of contents

- [Design overview and choices](#design-overview-and-choices)
- [User groups](#user-groups)
- [Organization-level IAM](#organization-level-iam)
- [Automation project and resources](#automation-project-and-resources)
- [Billing account](#billing-account)
- [Organization-level logging](#organization-level-logging)
- [Naming](#naming)
- [Workload Identity Federation and CI/CD](#workload-identity-federation-and-cicd)
- [How to run this stage](#how-to-run-this-stage)
- [Prerequisites](#prerequisites)
- [Output files and cross-stage variables](#output-files-and-cross-stage-variables)
- [Running the stage](#running-the-stage)
- [Customizations](#customizations)
- [Group names](#group-names)
- [IAM](#iam)
- [Log sinks and log destinations](#log-sinks-and-log-destinations)
- [Names and naming convention](#names-and-naming-convention)
- [Workload Identity Federation](#workload-identity-federation)
- [CI/CD repositories](#cicd-repositories)

## Design overview and choices

As mentioned above, this stage only does the bare minimum required to bootstrap automation, and ensure that base audit and billing exports are in place from the start to provide some measure of accountability, even before the security configurations are applied in a later stage.
Expand Down Expand Up @@ -80,7 +102,7 @@ The convention is used in its full form only for specific resources with globall

The [Customizations](#names-and-naming-convention) section on names below explains how to configure tokens, or implement a different naming convention.

## Workload Identity Federation and CI/CD
### Workload Identity Federation and CI/CD

This stage also implements initial support for two interrelated features

Expand Down Expand Up @@ -124,7 +146,7 @@ To quickly self-grant the above roles, run the following code snippet as the ini
export FAST_BU=$(gcloud config list --format 'value(core.account)')

# find and set your org id
gcloud organizations list --filter display_name:$partofyourdomain
gcloud organizations list
export FAST_ORG_ID=123456

# set needed roles
Expand All @@ -139,25 +161,6 @@ done

Then make sure the same user is also part of the `gcp-organization-admins` group so that impersonating the automation service account later on will be possible.

#### Billing account in a different organization

If you are using a billing account belonging to a different organization (e.g. in multiple organization setups), some initial configurations are needed to ensure the identities running this stage can assign billing-related roles.

If the billing organization is managed by another version of this stage, we leverage the `organizationIamAdmin` role created there, to allow restricted granting of billing roles at the organization level.

If that's not the case, an equivalent role needs to exist, or the predefined `resourcemanager.organizationAdmin` role can be used if not managed authoritatively. The role name then needs to be manually changed in the `billing.tf` file, in the `google_organization_iam_binding` resource.

The identity applying this stage for the first time also needs two roles in billing organization, they can be removed after the first `apply` completes successfully:

```bash
export FAST_BILLING_ORG_ID=789012
export FAST_ROLES=(roles/billing.admin roles/resourcemanager.organizationAdmin)
for role in $FAST_ROLES; do
gcloud organizations add-iam-policy-binding $FAST_BILLING_ORG_ID \
--member user:$FAST_BU --role $role
done
```

#### Standalone billing account

If you are using a standalone billing account, the identity applying this stage for the first time needs to be a billing account administrator:
Expand Down Expand Up @@ -187,7 +190,7 @@ Please note that FAST also supports an additional group for users with permissio
Then make sure you have configured the correct values for the following variables by providing a `terraform.tfvars` file:

- `billing_account`
an object containing `id` as the id of your billing account, derived from the Cloud Console UI or by running `gcloud beta billing accounts list`, and `organization_id` as the id of the organization owning it, or `null` to use the billing account in isolation
an object containing `id` as the id of your billing account, derived from the Cloud Console UI or by running `gcloud beta billing accounts list`, and the `is_org_level` flag that controls whether organization or account-level bindings are used, and a billing export project and dataset are created
- `groups`
the name mappings for your groups, if you're following the default convention you can leave this to the provided default
- `organization.id`, `organization.domain`, `organization.customer_id`
Expand All @@ -202,7 +205,6 @@ You can also adapt the example that follows to your needs:
# if you have too many accounts, check the Cloud Console :)
billing_account = {
id = "012345-67890A-BCDEF0"
organization_id = 1234567890
}

# use `gcloud organizations list`
Expand Down Expand Up @@ -237,18 +239,18 @@ Below is the outline of the output files generated by all stages, which is ident
```bash
[path specified in outputs_location]
├── providers
│   ├── 00-bootstrap-providers.tf
│   ├── 01-resman-providers.tf
│   ├── 02-networking-providers.tf
│   ├── 02-security-providers.tf
│   ├── 03-project-factory-dev-providers.tf
│   ├── 03-project-factory-prod-providers.tf
│   └── 99-sandbox-providers.tf
│   ├── 0-bootstrap-providers.tf
│   ├── 1-resman-providers.tf
│   ├── 2-networking-providers.tf
│   ├── 2-security-providers.tf
│   ├── 3-project-factory-dev-providers.tf
│   ├── 3-project-factory-prod-providers.tf
│   └── 9-sandbox-providers.tf
└── tfvars
│ ├── 00-bootstrap.auto.tfvars.json
│ ├── 01-resman.auto.tfvars.json
│ ├── 02-networking.auto.tfvars.json
│ └── 02-security.auto.tfvars.json
│ ├── 0-bootstrap.auto.tfvars.json
│ ├── 1-resman.auto.tfvars.json
│ ├── 2-networking.auto.tfvars.json
│ └── 2-security.auto.tfvars.json
└── workflows
└── [optional depending on the configured CI/CD repositories]
```
Expand All @@ -267,17 +269,34 @@ terraform apply \

> If you see an error related to project name already exists, please make sure the project name is unique or the project was not deleted recently

Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file if you have configured output files as described above, or extract its contents from Terraform's output, then migrate state with `terraform init`:
Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file from either

- the local filesystem if you have configured output files as described above
- the GCS bucket where output files are always stored
- Terraform outputs (not recommended as it's more complex)

The following two snippets show how to leverage the `stage-links.sh` script in the root FAST folder to fetch the commands required for output files linking or copying, using either the local output folder configured via Terraform variables, or the GCS bucket which can be derived from the `automation` output.

```bash
../../stage-links.sh ~/fast-config

# copy and paste the following commands for '0-bootstrap'

ln -s ~/fast-config/providers/0-bootstrap-providers.tf ./
```

```bash
../../stage-links.sh gs://xxx-prod-iac-core-outputs-0

# copy and paste the following commands for '0-bootstrap'

gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/providers/0-bootstrap-providers.tf ./
```

Copy/paste the command returned by the script to link or copy the provider file, then migrate state with `terraform init` and run `terraform apply`:

```bash
# if using output files via the outputs_location and set to `~/fast-config`
ln -s ~/fast-config/providers/00-bootstrap* ./
# or from outputs if not using output files
terraform output -json providers | jq -r '.["00-bootstrap"]' \
> providers.tf
# migrate state to GCS bucket configured in providers file
terraform init -migrate-state
# run terraform apply to remove the bootstrap_user iam binding
terraform apply
```

Expand Down Expand Up @@ -334,7 +353,7 @@ You can customize organization-level logs through the `log_sinks` variable in tw
- creating additional log sinks to capture more logs
- changing the destination of captured logs

By default, all logs are exported to Bigquery, but FAST can create sinks to Cloud Logging Buckets, GCS, or PubSub.
By default, all logs are exported to a log bucket, but FAST can create sinks to BigQuery, GCS, or PubSub.

If you need to capture additional logs, please refer to GCP's documentation on [scenarios for exporting logging data](https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics), where you can find ready-made filter expressions for different use cases.

Expand Down Expand Up @@ -400,12 +419,6 @@ cicd_repositories = {
name = "my-gh-org/fast-bootstrap"
type = "github"
}
cicd = {
branch = null
identity_provider = "github-sample"
name = "my-gh-org/fast-cicd"
type = "github"
}
resman = {
branch = "main"
identity_provider = "github-sample"
Expand Down
74 changes: 44 additions & 30 deletions fast/stages/1-resman/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,22 @@ The following diagram is a high level reference of the resources created and man
<img src="diagram.svg" alt="Resource-management diagram">
</p>

## Table of contents

- [Design overview and choices](#design-overview-and-choices)
- [Multitenancy](#multitenancy)
- [Workload Identity Federation and CI/CD](#workload-identity-federation-and-cicd)
- [How to run this stage](#how-to-run-this-stage)
- [Provider and Terraform variables](#provider-and-terraform-variables)
- [Impersonating the automation service account](#impersonating-the-automation-service-account)
- [Variable configuration](#variable-configuration)
- [Running the stage](#running-the-stage)
- [Customizations](#customizations)
- [Team folders](#team-folders)
- [Organization Policies](#organization-policies)
- [IAM](#iam)
- [Additional folders](#additional-folders)

## Design overview and choices

Despite its simplicity, this stage implements the basics of a design that we've seen working well for a variety of customers, where the hierarchy is laid out following two conceptually different approaches:
Expand Down Expand Up @@ -54,51 +70,49 @@ It's of course possible to run this stage in isolation, but that's outside the s

Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.

### Providers configuration

The default way of making sure you have the right permissions, is to use the identity of the service account pre-created for this stage during bootstrap, and that you are a member of the group that can impersonate it via provider-level configuration (`gcp-devops` or `organization-admins`).
### Provider and Terraform variables

To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path.
As all other FAST stages, the [mechanism used to pass variable values and pre-built provider files from one stage to the next](../0-bootstrap/README.md#output-files-and-cross-stage-variables) is also leveraged here.

If you have set a valid value for `outputs_location` in the bootstrap stage (see the [bootstrap stage README](../0-bootstrap/#output-files-and-cross-stage-variables) for more details), simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
The commands to link or copy the provider and terraform variable files can be easily derived from the `stage-links.sh` script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run.

```bash
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/01-resman-providers.tf .
```
../../stage-links.sh ~/fast-config

If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
# copy and paste the following commands for '1-resman'

```bash
cd ../0-bootstrap
terraform output -json providers | jq -r '.["01-resman"]' \
> ../1-resman/providers.tf
ln -s ~/fast-config/providers/1-resman-providers.tf ./
ln -s ~/fast-config/tfvars/globals.auto.tfvars.json ./
ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./
```

If you want to continue to rely on `outputs_location` logic, create a `terraform.tfvars` file and configure it as described [here](../0-bootstrap/#output-files-and-cross-stage-variables).
```bash
../../stage-links.sh gs://xxx-prod-iac-core-outputs-0

### Variable configuration
# copy and paste the following commands for '1-resman'

There are two broad sets of variables you will need to fill in:
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/providers/1-resman-providers.tf ./
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/globals.auto.tfvars.json ./
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./
```

- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
- variables specific to resources managed by this stage
### Impersonating the automation service account

To avoid the tedious job of filling in the first group of variable with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The `gcp-devops` and `organization-admins` groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups.

If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `*.auto.tfvars.json` files from the outputs folder. For this stage, you need the `globals.auto.tfvars.json` file containing global values compiled manually for the bootstrap stage, and `0-bootstrap.auto.tfvars.json` containing values derived from resources managed by the bootstrap stage:
### Variable configuration

```bash
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/tfvars/globals.auto.tfvars.json .
ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json .
```
Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets:

- variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the `globals.auto.tfvars.json` file linked or copied above
- variables which refer to resources managed by previous stage, which are prepopulated here via the `0-bootstrap.auto.tfvars.json` file linked or copied above
- and finally variables that optionally control this stage's behaviour and customizations, and can to be set in a custom `terraform.tfvars` file

A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra `terraform.tfvars` file.
The latter set is explained in the [Customization](#customizations) sections below, and the full list can be found in the [Variables](#variables) table at the bottom of this document.

Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations. For billing configurations, refer to the [Bootstrap documentation on billing](../0-bootstrap/README.md#billing-account) as the `billing_account` variable is identical across all stages.
### Running the stage

Once done, you can run this stage:
Once provider and variable values are in place and the correct user is configured, the stage can be run:

```bash
terraform init
Expand Down Expand Up @@ -139,9 +153,9 @@ This allows to centralize the minimum set of resources to delegate control of ea

### Organization policies

Organization policies are laid out in an explicit manner in the `organization.tf` file, so it's fairly easy to add or remove specific policies.
Organization policies leverage -- with one exception -- the built-in factory implemented in the organization module, and configured via the yaml files in the `data` folder. To edit organization policies, check and edit the files there.

For policies where additional data is needed, a root-level `organization_policy_configs` variable allows passing in specific data. Its built-in use to add additional organizations to the [Domain Restricted Sharing](https://cloud.google.com/resource-manager/docs/organization-policy/restricting-domains) policy, can be taken as an example on how to leverage it for additional customizations.
The one exception is [Domain Restricted Sharing](https://cloud.google.com/resource-manager/docs/organization-policy/restricting-domains), which is made dynamic and implemented in code so as to auto-add the current organization's customer id. The `organization_policy_configs` variable allow to easily add ids from third party organizations if needed.

### IAM

Expand Down
Loading