Skip to content

Latest commit

 

History

History
355 lines (322 loc) · 18 KB

README.md

File metadata and controls

355 lines (322 loc) · 18 KB
Raw

Google Cloud Storage Module

Simple bucket example

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  versioning = true
  labels = {
    cost-center = "devops"
  }
}
# tftest modules=1 resources=1 inventory=simple.yaml e2e

Cloud KMS

module "project" {
  source         = "./fabric/modules/project"
  name           = var.project_id
  project_create = false
  services       = ["storage.googleapis.com"]
}

module "kms" {
  source     = "./fabric/modules/kms"
  project_id = var.project_id
  keyring = {
    location = "europe" # location of the KMS must match location of the bucket
    name     = "test"
  }
  keys = {
    bucket_key = {
      iam_bindings = {
        bucket_key_iam = {
          members = [module.project.service_agents.storage.iam_email]
          role    = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
        }
      }
    }
  }
}

module "bucket" {
  source         = "./fabric/modules/gcs"
  project_id     = var.project_id
  prefix         = var.prefix
  name           = "my-bucket"
  location       = "EU"
  encryption_key = module.kms.keys.bucket_key.id
}

# tftest modules=3 skip e2e

Retention policy, soft delete policy and logging

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  retention_policy = {
    retention_period = 100
    is_locked        = true
  }
  soft_delete_retention = 7776000
  logging_config = {
    log_bucket        = "log-bucket"
    log_object_prefix = null
  }
}
# tftest modules=1 resources=1 inventory=retention-logging.yaml

Lifecycle rule

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  lifecycle_rules = {
    lr-0 = {
      action = {
        type          = "SetStorageClass"
        storage_class = "STANDARD"
      }
      condition = {
        age = 30
      }
    }
  }
}
# tftest modules=1 resources=1 inventory=lifecycle.yaml e2e

GCS notifications

module "project" {
  source         = "./fabric/modules/project"
  name           = var.project_id
  project_create = false
  services       = ["storage.googleapis.com"]
}

module "bucket-gcs-notification" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  notification_config = {
    enabled           = true
    payload_format    = "JSON_API_V1"
    sa_email          = module.project.service_agents.storage.email
    topic_name        = "gcs-notification-topic"
    event_types       = ["OBJECT_FINALIZE"]
    custom_attributes = {}
  }
}
# tftest skip e2e

Object upload

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  objects_to_upload = {
    sample-data = {
      name         = "example-file.csv"
      source       = "assets/example-file.csv"
      content_type = "text/csv"
    }
  }
}
# tftest modules=1 resources=2 inventory=object-upload.yaml e2e

IAM

IAM is managed via several variables that implement different features and levels of control:

  • iam and iam_by_principals configure authoritative bindings that manage individual roles exclusively, and are internally merged
  • iam_bindings configure authoritative bindings with optional support for conditions, and are not internally merged with the previous two variables
  • iam_bindings_additive configure additive bindings via individual role/member pairs with optional support conditions

The authoritative and additive approaches can be used together, provided different roles are managed by each. Some care must also be taken with the iam_by_principals variable to ensure that variable keys are static values, so that Terraform is able to compute the dependency graph.

Refer to the project module for examples of the IAM interface.

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  iam = {
    "roles/storage.admin" = ["group:${var.group_email}"]
  }
}
# tftest modules=1 resources=2 inventory=iam-authoritative.yaml e2e
module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  iam_bindings = {
    storage-admin-with-delegated_roles = {
      role    = "roles/storage.admin"
      members = ["group:${var.group_email}"]
      condition = {
        title = "delegated-role-grants"
        expression = format(
          "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
          join(",", formatlist("'%s'",
            [
              "roles/storage.objectAdmin",
              "roles/storage.objectViewer",
            ]
          ))
        )
      }
    }
  }
}
# tftest modules=1 resources=2 inventory=iam-bindings.yaml e2e
module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  iam_bindings_additive = {
    storage-admin-with-delegated_roles = {
      role   = "roles/storage.admin"
      member = "group:${var.group_email}"
      condition = {
        title = "delegated-role-grants"
        expression = format(
          "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
          join(",", formatlist("'%s'",
            [
              "roles/storage.objectAdmin",
              "roles/storage.objectViewer",
            ]
          ))
        )
      }
    }
  }
}
# tftest modules=1 resources=2 inventory=iam-bindings-additive.yaml e2e

Tag Bindings

Refer to the Creating and managing tags documentation for details on usage.

module "org" {
  source          = "./fabric/modules/organization"
  organization_id = var.organization_id
  tags = {
    environment = {
      description = "Environment specification."
      values = {
        dev     = {}
        prod    = {}
        sandbox = {}
      }
    }
  }
}

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  tag_bindings = {
    env-sandbox = module.org.tag_values["environment/sandbox"].id
  }
}
# tftest modules=2 resources=6

Managed Folders

module "bucket" {
  source     = "./fabric/modules/gcs"
  project_id = var.project_id
  prefix     = var.prefix
  name       = "my-bucket"
  location   = "EU"
  managed_folders = {
    folder1 = {
      iam = {
        "roles/storage.admin" = ["user:[email protected]"]
      }
    }
    "folder1/subfolder" = {
      force_destroy = true
    }
  }
}
# tftest inventory=managed-folders.yaml

Variables

name description type required default
location Bucket location. string ✓
name Bucket name suffix. string ✓
project_id Bucket project id. string ✓
autoclass Enable autoclass to automatically transition objects to appropriate storage classes based on their access pattern. If set to true, storage_class must be set to STANDARD. Defaults to false. bool null
cors CORS configuration for the bucket. Defaults to null. object({…}) null
custom_placement_config The bucket's custom location configuration, which specifies the individual regions that comprise a dual-region bucket. If the bucket is designated as REGIONAL or MULTI_REGIONAL, the parameters are empty. list(string) null
default_event_based_hold Enable event based hold to new objects added to specific bucket, defaults to false. bool null
enable_object_retention Enables object retention on a storage bucket. bool null
encryption_key KMS key that will be used for encryption. string null
force_destroy Optional map to set force destroy keyed by name, defaults to false. bool false
iam IAM bindings in {ROLE => [MEMBERS]} format. map(list(string)) {}
iam_bindings Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. map(object({…})) {}
iam_bindings_additive Individual additive IAM bindings. Keys are arbitrary. map(object({…})) {}
iam_by_principals Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the iam variable. map(list(string)) {}
labels Labels to be attached to all buckets. map(string) {}
lifecycle_rules Bucket lifecycle rule. map(object({…})) {}
logging_config Bucket logging configuration. object({…}) null
managed_folders Managed folders to create within the bucket in {PATH => CONFIG} format. map(object({…})) {}
notification_config GCS Notification configuration. object({…}) null
objects_to_upload Objects to be uploaded to bucket. map(object({…})) {}
prefix Optional prefix used to generate the bucket name. string null
public_access_prevention Prevents public access to the bucket. string null
requester_pays Enables Requester Pays on a storage bucket. bool null
retention_policy Bucket retention policy. object({…}) null
rpo Bucket recovery point objective. string null
soft_delete_retention The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Set to 0 to override the default and disable. number null
storage_class Bucket storage class. string "STANDARD"
tag_bindings Tag bindings for this folder, in key => tag value id format. map(string) {}
uniform_bucket_level_access Allow using object ACLs (false) or not (true, this is the recommended behavior) , defaults to true (which is the recommended practice, but not the behavior of storage API). bool true
versioning Enable versioning, defaults to false. bool null
website Bucket website. object({…}) null

Outputs

name description sensitive
bucket Bucket resource.
id Fully qualified bucket id.
name Bucket name.
notification GCS Notification self link.
objects Objects in GCS bucket.
topic Topic ID used by GCS.
url Bucket URL.