Adding clustering index to existing table destroys and recreates table and does not patch it thus destroying all data

[MikeMoore63]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.12.26

Affected Resource(s)

• google_bigquery_table

Terraform Configuration Files

before

resource "google_bigquery_table" "billing_tables_region1" {
  dataset_id      = "demo"
  friendly_name   = "demo table"
  table_id        = "demo_table"
  schema          = "[ { \"mode\": \"NULLABLE\", \"name\": \"HostName\", \"type\": \"STRING\" }]"
  clustering      = []
  expiration_time = null
  project         = "methodical-bee-162815"
  description     = "demo table"
  time_partitioning   {
      type                     = "DAY"

    }
}

terraform init
terraform apply

Terraform will perform the following actions:

  # google_bigquery_table.billing_tables_region1 will be created
  + resource "google_bigquery_table" "billing_tables_region1" {
      + clustering          = []
      + creation_time       = (known after apply)
      + dataset_id          = "demo"
      + description         = "demo table"
      + etag                = (known after apply)
      + expiration_time     = (known after apply)
      + friendly_name       = "demo table"
      + id                  = (known after apply)
      + last_modified_time  = (known after apply)
      + location            = (known after apply)
      + num_bytes           = (known after apply)
      + num_long_term_bytes = (known after apply)
      + num_rows            = (known after apply)
      + project             = "methodical-bee-162815"
      + schema              = jsonencode(
            [
              + {
                  + mode = "NULLABLE"
                  + name = "HostName"
                  + type = "STRING"
                },
            ]
        )
      + self_link           = (known after apply)
      + table_id            = "demo_table"
      + type                = (known after apply)

      + time_partitioning {
          + type = "DAY"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

As expected creates table

Alter tf to

resource "google_bigquery_table" "billing_tables_region1" {
  dataset_id      = "demo"
  friendly_name   = "demo table"
  table_id        = "demo_table"
  schema          = "[ { \"mode\": \"NULLABLE\", \"name\": \"HostName\", \"type\": \"STRING\" }]"
  clustering      = ["HostName"]
  expiration_time = null
  project         = "methodical-bee-162815"
  description     = "demo table"
  time_partitioning   {
      type                     = "DAY"

    }
}

Expect just a patch as https://cloud.google.com/bigquery/docs/creating-clustered-tables#modifying-cluster-spec states can be patched

instead get

 # google_bigquery_table.billing_tables_region1 must be replaced
-/+ resource "google_bigquery_table" "billing_tables_region1" {
      ~ clustering          = [ # forces replacement
          + "HostName",
        ]
      ~ creation_time       = 1599051275866 -> (known after apply)
        dataset_id          = "demo"
        description         = "demo table"
      ~ etag                = "ZUfcv4RXkRB4JvqwNvkf+w==" -> (known after apply)
      ~ expiration_time     = 0 -> (known after apply)
        friendly_name       = "demo table"
      ~ id                  = "projects/methodical-bee-162815/datasets/demo/tables/demo_table" -> (known after apply)
      - labels              = {} -> null
      ~ last_modified_time  = 1599051275915 -> (known after apply)
      ~ location            = "US" -> (known after apply)
      ~ num_bytes           = 0 -> (known after apply)
      ~ num_long_term_bytes = 0 -> (known after apply)
      ~ num_rows            = 0 -> (known after apply)
        project             = "methodical-bee-162815"
        schema              = jsonencode(
            [
                {
                    mode = "NULLABLE"
                    name = "HostName"
                    type = "STRING"
                },
            ]
        )
      ~ self_link           = "https://bigquery.googleapis.com/bigquery/v2/projects/methodical-bee-162815/datasets/demo/tables/demo_table" -> (known after apply)
        table_id            = "demo_table"
      ~ type                = "TABLE" -> (known after apply)

      ~ time_partitioning {
          - expiration_ms            = 0 -> null
          - require_partition_filter = false -> null
            type                     = "DAY"
        }
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

This will delete all data making terraform impossible to be used for tables that involve changes to clustering.

Expected Behavior

Should have patched the table not destroyed and recreated

Actual Behavior

Destroyed and recreated

Steps to Reproduce

See attached

1. terraform apply

Important Factoids

No

References

See

https://cloud.google.com/bigquery/docs/creating-clustered-tables#modifying-cluster-spec

• #0000

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!