Add support for setting Pub/Sub subscription filename_datetime_format

[lahuang4]

Fixes hashicorp/terraform-provider-google#17911.

Release Note Template for Downstream PRs (will be copied)

pubsub: added `cloud_storage_config.filename_datetime_format` field to `google_pubsub_subscription` resource

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@zli82016, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 40 insertions(+), 10 deletions(-))
google-beta provider: Diff ( 3 files changed, 40 insertions(+), 10 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 2 files changed, 7 insertions(+), 5 deletions(-))

[modular-magician]

Tests analytics

Total tests: 44
Passed tests: 41
Skipped tests: 1
Affected tests: 2

Click here to see the affected service packages

• pubsub

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccPubsubSubscription_pubsubSubscriptionPushCloudstorageAvroExample|TestAccPubsubSubscription_pubsubSubscriptionPushCloudstorageExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccPubsubSubscription_pubsubSubscriptionPushCloudstorageAvroExample[Debug log]
TestAccPubsubSubscription_pubsubSubscriptionPushCloudstorageExample[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[zli82016]

@lahuang4 , thanks for adding the new field.

Can you please add the update test for this new field in the file https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/third_party/terraform/services/pubsub/resource_pubsub_subscription_test.go?

This is the documentation how to add update test.

[zli82016]

Update test is needed.

[lahuang4]

Update test is needed.

Done, thanks for linking to the docs for how to add them!

Ran the new test following https://googlecloudplatform.github.io/magic-modules/develop/test/run-tests/:

$ make testacc TEST=./google/services/pubsub TESTARGS='-run=TestAccPubsubSubscriptionCloudStorage_update'
TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google/services/pubsub -v -run=TestAccPubsubSubscriptionCloudStorage_update -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestAccPubsubSubscriptionCloudStorage_update
=== PAUSE TestAccPubsubSubscriptionCloudStorage_update
=== CONT  TestAccPubsubSubscriptionCloudStorage_update
--- PASS: TestAccPubsubSubscriptionCloudStorage_update (41.50s)
PASS
ok  	github.com/hashicorp/terraform-provider-google/google/services/pubsub	41.622s

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 105 insertions(+), 10 deletions(-))
google-beta provider: Diff ( 4 files changed, 105 insertions(+), 10 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 2 files changed, 7 insertions(+), 5 deletions(-))

[modular-magician]

Tests analytics

Total tests: 45
Passed tests: 43
Skipped tests: 1
Affected tests: 1

Click here to see the affected service packages

• pubsub

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccPubsubSubscriptionCloudStorage_update

Get to know how VCR tests work

[modular-magician]

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccPubsubSubscriptionCloudStorage_update[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[lahuang4]

Hm, the VCR test failed the new update test with:

    vcr_utils.go:152: Step 1/4 error: Error running apply: exit status 1
        
        Error: Error creating Subscription: googleapi: Error 400: Cloud Pub/Sub did not have the necessary permissions configured to access the provided bucket tf-test-bucket-ypgcey1jbd (or the bucket may not exist). Please verify that the service account [email protected] was granted the Storage Legacy Bucket Reader and Storage Object Creator roles for the provided bucket. These roles must be granted at the bucket level, not the project level. If you must grant roles at the project level, you may instead grant the Storage Admin role for the project containing the provided bucket.
        
          with google_pubsub_subscription.foo,
          on terraform_plugin_test.tf line 15, in resource "google_pubsub_subscription" "foo":
          15: resource "google_pubsub_subscription" "foo" {

I thought that providing the Storage Admin permission at the project level would do it:

resource "google_project_iam_member" "admin" {
  project = data.google_project.project.project_id
  role   = "roles/storage.admin"
  member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
}

I see that some of the other tests grant the permission at the bucket level. I'll give that a try.

[zli82016]

Hm, the VCR test failed the new update test with:

    vcr_utils.go:152: Step 1/4 error: Error running apply: exit status 1
        
        Error: Error creating Subscription: googleapi: Error 400: Cloud Pub/Sub did not have the necessary permissions configured to access the provided bucket tf-test-bucket-ypgcey1jbd (or the bucket may not exist). Please verify that the service account [email protected] was granted the Storage Legacy Bucket Reader and Storage Object Creator roles for the provided bucket. These roles must be granted at the bucket level, not the project level. If you must grant roles at the project level, you may instead grant the Storage Admin role for the project containing the provided bucket.
        
          with google_pubsub_subscription.foo,
          on terraform_plugin_test.tf line 15, in resource "google_pubsub_subscription" "foo":
          15: resource "google_pubsub_subscription" "foo" {

I thought that providing the Storage Admin permission at the project level would do it:

resource "google_project_iam_member" "admin" {
  project = data.google_project.project.project_id
  role   = "roles/storage.admin"
  member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
}

I see that some of the other tests grant the permission at the bucket level. I'll give that a try.

Yes, you can copy the generated test example here

[lahuang4]

Thanks for the tip! Looking at the generated test example, I think the issue might have been that I missed the depends_on clause. Updated the test and also added coverage for the other Cloud Storage subscription fields. Fingers crossed.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 134 insertions(+), 10 deletions(-))
google-beta provider: Diff ( 4 files changed, 134 insertions(+), 10 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 2 files changed, 7 insertions(+), 5 deletions(-))

[modular-magician]

Tests analytics

Total tests: 45
Passed tests: 43
Skipped tests: 1
Affected tests: 1

Click here to see the affected service packages

• pubsub

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccPubsubSubscriptionCloudStorage_update

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccPubsubSubscriptionCloudStorage_update[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[zli82016]

Yeah, the test passed. Thanks for adding the test. LGTM.