Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update FAST logging #2235

Merged
merged 3 commits into from
Apr 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 11 additions & 9 deletions fast/stages/0-bootstrap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,9 @@ Because of limitations of API availability, manual steps have to be followed to

### Organization-level logging

We create organization-level log sinks early in the bootstrap process to ensure a proper audit trail is in place from the very beginning. By default, we provide log filters to capture [Cloud Audit Logs](https://cloud.google.com/logging/docs/audit), [VPC Service Controls violations](https://cloud.google.com/vpc-service-controls/docs/troubleshooting#vpc-sc-errors) and [Workspace Logs](https://cloud.google.com/logging/docs/audit/configure-gsuite-audit-logs) into logging buckets in the top-level audit logging project.
We create organization-level log sinks early in the bootstrap process to ensure a proper audit trail is in place from the very beginning. By default, we provide log filters to capture [Cloud Audit Logs](https://cloud.google.com/logging/docs/audit), [VPC Service Controls violations](https://cloud.google.com/vpc-service-controls/docs/troubleshooting#vpc-sc-errors) and [Workspace Logs](https://cloud.google.com/logging/docs/audit/configure-gsuite-audit-logs) into logging buckets in the top-level audit logging project.

An organization-level sink captures IAM data access logs, including authentication and impersonation events for service accounts. To manage logging costs, the default configuration enables IAM data access logging only within the automation project (where sensitive service accounts reside). For enhanced security across the entire organization, consider enabling these logs at the organization level.

The [Customizations](#log-sinks-and-log-destinations) section explains how to change the logs captured and their destination.

Expand Down Expand Up @@ -626,8 +628,8 @@ The `fast_features` variable consists of 4 toggles:
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| [billing_account](variables.tf#L17) | Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`. | <code title="object&#40;&#123;&#10; id &#61; string&#10; is_org_level &#61; optional&#40;bool, true&#41;&#10; no_iam &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | |
| [organization](variables.tf#L223) | Organization details. | <code title="object&#40;&#123;&#10; id &#61; number&#10; domain &#61; optional&#40;string&#41;&#10; customer_id &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | |
| [prefix](variables.tf#L238) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | |
| [organization](variables.tf#L241) | Organization details. | <code title="object&#40;&#123;&#10; id &#61; number&#10; domain &#61; optional&#40;string&#41;&#10; customer_id &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | |
| [prefix](variables.tf#L256) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | |
| [bootstrap_user](variables.tf#L27) | Email of the nominal user running this stage for the first time. | <code>string</code> | | <code>null</code> | |
| [cicd_repositories](variables.tf#L33) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object&#40;&#123;&#10; bootstrap &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; type &#61; string&#10; branch &#61; optional&#40;string&#41;&#10; identity_provider &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; resman &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; type &#61; string&#10; branch &#61; optional&#40;string&#41;&#10; identity_provider &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [custom_roles](variables.tf#L79) | Map of role names => list of permissions to additionally create at the organization level. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
Expand All @@ -639,12 +641,12 @@ The `fast_features` variable consists of 4 toggles:
| [iam_bindings_additive](variables.tf#L141) | Organization-level custom additive IAM bindings. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; member &#61; string&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| [iam_by_principals](variables.tf#L156) | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| [locations](variables.tf#L163) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; bq &#61; optional&#40;string, &#34;EU&#34;&#41;&#10; gcs &#61; optional&#40;string, &#34;EU&#34;&#41;&#10; logging &#61; optional&#40;string, &#34;global&#34;&#41;&#10; pubsub &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [log_sinks](variables.tf#L177) | Org-level log sinks, in name => {type, filter} format. | <code title="map&#40;object&#40;&#123;&#10; filter &#61; string&#10; type &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; audit-logs &#61; &#123;&#10; filter &#61; &#34;logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Factivity&#92;&#34; OR logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Fsystem_event&#92;&#34; OR protoPayload.metadata.&#64;type&#61;&#92;&#34;type.googleapis.com&#47;google.cloud.audit.TransparencyLog&#92;&#34;&#34;&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10; vpc-sc &#61; &#123;&#10; filter &#61; &#34;protoPayload.metadata.&#64;type&#61;&#92;&#34;type.googleapis.com&#47;google.cloud.audit.VpcServiceControlAuditMetadata&#92;&#34;&#34;&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10; workspace-audit-logs &#61; &#123;&#10; filter &#61; &#34;logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Fdata_access&#92;&#34; and protoPayload.serviceName:&#92;&#34;login.googleapis.com&#92;&#34;&#34;&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [org_policies_config](variables.tf#L206) | Organization policies customization. | <code title="object&#40;&#123;&#10; constraints &#61; optional&#40;object&#40;&#123;&#10; allowed_policy_member_domains &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; import_defaults &#61; optional&#40;bool, false&#41;&#10; tag_name &#61; optional&#40;string, &#34;org-policies&#34;&#41;&#10; tag_values &#61; optional&#40;map&#40;object&#40;&#123;&#10; description &#61; optional&#40;string, &#34;Managed by the Terraform organization module.&#34;&#41;&#10; iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [outputs_location](variables.tf#L232) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
| [project_parent_ids](variables.tf#L247) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent. | <code title="object&#40;&#123;&#10; automation &#61; optional&#40;string&#41;&#10; billing &#61; optional&#40;string&#41;&#10; logging &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [workforce_identity_providers](variables.tf#L258) | Workforce Identity Federation pools. | <code title="map&#40;object&#40;&#123;&#10; attribute_condition &#61; optional&#40;string&#41;&#10; issuer &#61; string&#10; display_name &#61; string&#10; description &#61; string&#10; disabled &#61; optional&#40;bool, false&#41;&#10; saml &#61; optional&#40;object&#40;&#123;&#10; idp_metadata_xml &#61; string&#10; &#125;&#41;, null&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| [workload_identity_providers](variables.tf#L274) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | <code title="map&#40;object&#40;&#123;&#10; attribute_condition &#61; optional&#40;string&#41;&#10; issuer &#61; string&#10; custom_settings &#61; optional&#40;object&#40;&#123;&#10; issuer_uri &#61; optional&#40;string&#41;&#10; audiences &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; jwks_json &#61; optional&#40;string&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| [log_sinks](variables.tf#L177) | Org-level log sinks, in name => {type, filter} format. | <code title="map&#40;object&#40;&#123;&#10; filter &#61; string&#10; type &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; audit-logs &#61; &#123;&#10; filter &#61; &#60;&#60;-FILTER&#10; log_id&#40;&#34;cloudaudit.googleapis.com&#47;activity&#34;&#41; OR&#10; log_id&#40;&#34;cloudaudit.googleapis.com&#47;system_event&#34;&#41; OR&#10; log_id&#40;&#34;cloudaudit.googleapis.com&#47;policy&#34;&#41; OR&#10; log_id&#40;&#34;cloudaudit.googleapis.com&#47;access_transparency&#34;&#41;&#10; FILTER&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10; iam &#61; &#123;&#10; filter &#61; &#60;&#60;-FILTER&#10; protoPayload.serviceName&#61;&#34;iamcredentials.googleapis.com&#34; OR&#10; protoPayload.serviceName&#61;&#34;iam.googleapis.com&#34; OR&#10; protoPayload.serviceName&#61;&#34;sts.googleapis.com&#34;&#10; FILTER&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10; vpc-sc &#61; &#123;&#10; filter &#61; &#60;&#60;-FILTER&#10; protoPayload.metadata.&#64;type:&#34;type.googleapis.com&#47;google.cloud.audit.VpcServiceControlAuditMetadata&#34;&#10; FILTER&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10; workspace-audit-logs &#61; &#123;&#10; filter &#61; &#60;&#60;-FILTER&#10; log_id&#40;&#34;cloudaudit.googleapis.com&#47;data_access&#34;&#41;&#10; protoPayload.serviceName:&#34;login.googleapis.com&#34;&#10; FILTER&#10; type &#61; &#34;logging&#34;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [org_policies_config](variables.tf#L224) | Organization policies customization. | <code title="object&#40;&#123;&#10; constraints &#61; optional&#40;object&#40;&#123;&#10; allowed_policy_member_domains &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; import_defaults &#61; optional&#40;bool, false&#41;&#10; tag_name &#61; optional&#40;string, &#34;org-policies&#34;&#41;&#10; tag_values &#61; optional&#40;map&#40;object&#40;&#123;&#10; description &#61; optional&#40;string, &#34;Managed by the Terraform organization module.&#34;&#41;&#10; iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [outputs_location](variables.tf#L250) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
| [project_parent_ids](variables.tf#L265) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent. | <code title="object&#40;&#123;&#10; automation &#61; optional&#40;string&#41;&#10; billing &#61; optional&#40;string&#41;&#10; logging &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [workforce_identity_providers](variables.tf#L276) | Workforce Identity Federation pools. | <code title="map&#40;object&#40;&#123;&#10; attribute_condition &#61; optional&#40;string&#41;&#10; issuer &#61; string&#10; display_name &#61; string&#10; description &#61; string&#10; disabled &#61; optional&#40;bool, false&#41;&#10; saml &#61; optional&#40;object&#40;&#123;&#10; idp_metadata_xml &#61; string&#10; &#125;&#41;, null&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| [workload_identity_providers](variables.tf#L292) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | <code title="map&#40;object&#40;&#123;&#10; attribute_condition &#61; optional&#40;string&#41;&#10; issuer &#61; string&#10; custom_settings &#61; optional&#40;object&#40;&#123;&#10; issuer_uri &#61; optional&#40;string&#41;&#10; audiences &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; jwks_json &#61; optional&#40;string&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |

## Outputs

Expand Down
18 changes: 18 additions & 0 deletions fast/stages/0-bootstrap/automation.tf
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,24 @@ module "automation-project" {
"container.googleapis.com",
]
)

# Enable IAM data access logs to capture impersonation and service
# account token generation events. This is implemented within the
# automation project to limit log volume. For heightened security,
# consider enabling it at the organization level. A log sink within
# the organization will collect and store these logs in a logging
# bucket. See
# https://cloud.google.com/iam/docs/audit-logging#audited_operations
logging_data_access = {
"iam.googleapis.com" = {
# ADMIN_READ captures impersonation and token generation/exchanges
ADMIN_READ = []
# enable DATA_WRITE if you want to capture configuration changes
# to IAM-related resources (roles, deny policies, service
# accounts, identity pools, etc)
# DATA_WRITE = []
}
}
}

# output files bucket
Expand Down
24 changes: 21 additions & 3 deletions fast/stages/0-bootstrap/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -182,15 +182,33 @@ variable "log_sinks" {
}))
default = {
audit-logs = {
filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\" OR protoPayload.metadata.@type=\"type.googleapis.com/google.cloud.audit.TransparencyLog\""
filter = <<-FILTER
log_id("cloudaudit.googleapis.com/activity") OR
log_id("cloudaudit.googleapis.com/system_event") OR
log_id("cloudaudit.googleapis.com/policy") OR
log_id("cloudaudit.googleapis.com/access_transparency")
FILTER
type = "logging"
}
iam = {
filter = <<-FILTER
protoPayload.serviceName="iamcredentials.googleapis.com" OR
protoPayload.serviceName="iam.googleapis.com" OR
protoPayload.serviceName="sts.googleapis.com"
FILTER
type = "logging"
}
vpc-sc = {
filter = "protoPayload.metadata.@type=\"type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata\""
filter = <<-FILTER
protoPayload.metadata.@type:"type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata"
FILTER
type = "logging"
}
workspace-audit-logs = {
filter = "logName:\"/logs/cloudaudit.googleapis.com%2Fdata_access\" and protoPayload.serviceName:\"login.googleapis.com\""
filter = <<-FILTER
log_id("cloudaudit.googleapis.com/data_access")
protoPayload.serviceName:"login.googleapis.com"
FILTER
type = "logging"
}
}
Expand Down
10 changes: 5 additions & 5 deletions tests/fast/stages/s0_bootstrap/checklist.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -360,15 +360,15 @@ counts:
google_bigquery_dataset: 1
google_bigquery_default_service_account: 3
google_essential_contacts_contact: 3
google_logging_organization_sink: 3
google_logging_project_bucket_config: 3
google_logging_organization_sink: 4
google_logging_project_bucket_config: 4
google_org_policy_policy: 22
google_organization_iam_binding: 27
google_organization_iam_custom_role: 7
google_organization_iam_member: 35
google_project: 3
google_project_iam_binding: 19
google_project_iam_member: 6
google_project_iam_member: 7
google_project_service: 31
google_project_service_identity: 4
google_service_account: 4
Expand All @@ -380,5 +380,5 @@ counts:
google_storage_project_service_account: 3
google_tags_tag_key: 1
google_tags_tag_value: 1
modules: 17
resources: 198
modules: 18
resources: 202
10 changes: 5 additions & 5 deletions tests/fast/stages/s0_bootstrap/simple.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,15 +39,15 @@ counts:
google_bigquery_dataset: 1
google_bigquery_default_service_account: 3
google_essential_contacts_contact: 3
google_logging_organization_sink: 3
google_logging_project_bucket_config: 3
google_logging_organization_sink: 4
google_logging_project_bucket_config: 4
google_org_policy_policy: 22
google_organization_iam_binding: 27
google_organization_iam_custom_role: 7
google_organization_iam_member: 22
google_project: 3
google_project_iam_binding: 19
google_project_iam_member: 6
google_project_iam_member: 7
google_project_service: 31
google_project_service_identity: 4
google_service_account: 4
Expand All @@ -60,8 +60,8 @@ counts:
google_tags_tag_key: 1
google_tags_tag_value: 1
local_file: 7
modules: 16
resources: 189
modules: 17
resources: 193

outputs:
custom_roles:
Expand Down
Loading