bigtable-instance

Google Cloud BigTable Module

This module allows managing a single BigTable instance, including access configuration and tables.

TODO

• support bigtable_app_profile
• support IAM for tables

Examples

Instance with access configuration

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone = "europe-west1-b"
    }
  }
  tables = {
    test1 = {},
    test2 = {
      split_keys = ["a", "b", "c"]
    }
  }
  iam = {
    "roles/bigtable.user" = ["user:[email protected]"]
  }
}
# tftest modules=1 resources=4 inventory=simple.yaml

Instance with tables and column families

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone = "europe-west1-b"
    }
  }
  tables = {
    test1 = {},
    test2 = {
      split_keys = ["a", "b", "c"]
      column_families = {
        cf1 = {}
        cf2 = {}
        cf3 = {}
      }
    }
    test3 = {
      column_families = {
        cf1 = {}
      }
    }
  }
}
# tftest modules=1 resources=4 inventory=columns.yaml

Instance with replication enabled

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    first-cluster = {
      zone = "europe-west1-b"
    }
    second-cluster = {
      zone = "europe-southwest1-a"
    }
    third-cluster = {
      zone = "us-central1-b"
    }
  }
}
# tftest modules=1 resources=1 inventory=replication.yaml

Instance with garbage collection policy

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone = "europe-west1-b"
    }
  }
  tables = {
    test1 = {
      column_families = {
        cf1 = {
          gc_policy = {
            deletion_policy = "ABANDON"
            max_age         = "18h"
          }
        }
        cf2 = {}
      }
    }
  }
}
# tftest modules=1 resources=3 inventory=gc.yaml

Instance with default garbage collection policy

The default garbage collection policy is applied to any column family that does not specify a gc_policy. If a column family specifies a gc_policy, the default garbage collection policy is ignored for that column family.

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone = "europe-west1-b"
    }
  }
  default_gc_policy = {
    deletion_policy = "ABANDON"
    max_age         = "18h"
    max_version     = 7
  }
  tables = {
    test1 = {
      column_families = {
        cf1 = {}
        cf2 = {}
      }
    }
  }
}
# tftest modules=1 resources=4

Instance with static number of nodes

If you are not using autoscaling settings, you must set a specific number of nodes with the variable num_nodes.

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone      = "europe-west1-b"
      num_nodes = 5
    }
  }
}
# tftest modules=1 resources=1 inventory=static.yaml

Instance with autoscaling (based on CPU only)

If you use autoscaling, you should not set the variable num_nodes.

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone = "europe-southwest1-b"
      autoscaling = {
        min_nodes  = 3
        max_nodes  = 7
        cpu_target = 70
      }
    }
  }


}
# tftest modules=1 resources=1 inventory=autoscaling1.yaml

Instance with autoscaling (based on CPU and/or storage)

module "bigtable-instance" {
  source     = "./fabric/modules/bigtable-instance"
  project_id = "my-project"
  name       = "instance"
  clusters = {
    my-cluster = {
      zone         = "europe-southwest1-a"
      storage_type = "SSD"
      autoscaling = {
        min_nodes      = 3
        max_nodes      = 7
        cpu_target     = 70
        storage_target = 4096
      }
    }
  }
}
# tftest modules=1 resources=1 inventory=autoscaling2.yaml

Variables

name	description	type	required	default
clusters	Clusters to be created in the BigTable instance. Set more than one cluster to enable replication. If you set autoscaling, num_nodes will be ignored.	map(object({…}))	✓
name	The name of the Cloud Bigtable instance.	string	✓
project_id	Id of the project where datasets will be created.	string	✓
default_autoscaling	Default settings for autoscaling of clusters. This will be the default autoscaling for any cluster not specifying any autoscaling details.	object({…})		null
default_gc_policy	Default garbage collection policy, to be applied to all column families and all tables. Can be override in the tables variable for specific column families.	object({…})		null
deletion_protection	Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail.			true
display_name	The human-readable display name of the Bigtable instance.			null
iam	IAM bindings for topic in {ROLE => [MEMBERS]} format.	map(list(string))		{}
instance_type	(deprecated) The instance type to create. One of 'DEVELOPMENT' or 'PRODUCTION'.	string		null
tables	Tables to be created in the BigTable instance.	map(object({…}))		{}

Outputs

name	description	sensitive
id	Fully qualified instance id.
instance	BigTable instance.
table_ids	Map of fully qualified table ids keyed by table name.
tables	Table resources.