google_pubsub_subscription with bigquery_config created with wrong 'Delivery type'

[etalaevsky]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

terraform -v
Terraform v1.2.7
on darwin_arm64

Affected Resource(s)

google provider version 4.32.0

google_pubsub_subscription

Terraform Configuration Files

Configuration is taken from official docs

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/pubsub_subscription#example-usage---pubsub-subscription-push-bq

Expected Behavior

Create new subscription with Delivery type Write to BigQuery (Push)

Actual Behavior

Create new subscription with Delivery type Pull

Steps to Reproduce

1. Create new deployment based on official docs provided above
2. terraform apply

[edwardmedia]

@etalaevsky can you share your config and the debug log so I can take a close look?

[etalaevsky]

@edwardmedia the config is exactly as in official provider documentation. Please refer to the link in the bug.

Moreover, if I change the delivery type manually from 'Pull' to 'BigQuery (Push)' via the gcp console and re-run terraform plan, the plan does not show any significant changes apart from IDs changes.

I believe it has to be easy to reproduce.

[richmcw]

I believe the issue to be the project delimiter in bigquery_config.table. I was able to manually create one; import into Terraform and examine the state between the two resources.

The docs show the following for the BigQuery config block:

  bigquery_config {
    table = "${google_bigquery_table.test.project}.${google_bigquery_table.test.dataset_id}.${google_bigquery_table.test.table_id}"
  }

but should be:

  bigquery_config {
    table = "${google_bigquery_table.test.project}:${google_bigquery_table.test.dataset_id}.${google_bigquery_table.test.table_id}"
  }

[etalaevsky]

@richmcw there is a chance you're right about ':'
BTW, I was also looking at the description in the code. It says to use the '.'

Description: `The name of the table to which to write data, of the form {projectId}.{datasetId}.{tableId}

https://github.com/hashicorp/terraform-provider-google/blob/v4.32.0/google/resource_pubsub_subscription.go#L106

[richmcw]

@etalaevsky The linked PR addresses that too.

When I examine the state of the resource created via copy/paste from documentation:

    bigquery_config {
        drop_unknown_fields = false
        table               = "rmcw0-e378ed1c.example_dataset.example_table"
        use_topic_schema    = false
        write_metadata      = false
    }    

vs when I examine the state of a manually created resource after importing to Terraform:

    bigquery_config {
        table               = "rmcw0-e378ed1c:example_dataset.example_table"
        use_topic_schema    = false
        write_metadata      = false
    }

Notice the ':' between project and dataset and not a '.'

[etalaevsky]

@richmcw noticed. Replacing it with ':' does the trick.
I am wondering there is no validation. It should've been fail on terraform apply.
Cause gcloud documentation states ':'
https://cloud.google.com/pubsub/docs/create-subscription#create_a_subscription

gcloud pubsub subscriptions create SUBSCRIPTION_ID \
  --topic=TOPIC_ID \
  --bigquery-table=PROJECT_ID:DATASET_ID.TABLE_ID

From the other side, Python and Java uses "." in example
"your-project.your-dataset.your-table"

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.