Fix nightly test failure of `billing_project` related acceptance tests #12122

SarahFrench · 2024-10-24T19:44:44Z

This PR updates the 'usage' test cases in TestAccSdkProvider_billing_project so that the newly provisioned project always has the Service Usage API enabled. This is needed to address failures in nightly tests.

This doesn't impact the acceptance tests' validity, as they only rely on the PubSub API being dis/enabled in the new project. Service Usage API doesn't impact that.

Here's the error being solved:

------- Stdout: -------
=== RUN   TestAccSdkProvider_billing_project
=== RUN   TestAccSdkProvider_billing_project/config_takes_precedence_over_environment_variables
=== RUN   TestAccSdkProvider_billing_project/when_billing_project_is_unset_in_the_config,_environment_variables_are_used_in_a_given_order
=== RUN   TestAccSdkProvider_billing_project/when_billing_project_is_set_to_an_empty_string_in_the_config_the_value_isn't_ignored_and_results_in_an_error
=== RUN   TestAccSdkProvider_billing_project/GOOGLE_CLOUD_QUOTA_PROJECT_environment_variable_interferes_with_the_billing_account_value_used
    testing_new.go:90: Error running post-test destroy, there may be dangling resources: exit status 1
        Error: Error when reading or editing Project Service tf-test-3svo0ueyzk/pubsub.googleapis.com: Error disabling service "pubsub.googleapis.com" for project "tf-test-3svo0ueyzk": googleapi: Error 403: Service Usage API has not been used in project tf-test-3svo0ueyzk before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=tf-test-3svo0ueyzk then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
        Details:
        [
          {
            "@type": "type.googleapis.com/google.rpc.Help",
            "links": [
              {
                "description": "Google developers console API activation",
                "url": "https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=tf-test-3svo0ueyzk
              }
            ]
          },
          {
            "@type": "type.googleapis.com/google.rpc.ErrorInfo",
            "domain": "googleapis.com",
            "metadata": {
              "consumer": "projects/tf-test-3svo0ueyzk",
              "service": "serviceusage.googleapis.com"
            },
            "reason": "SERVICE_DISABLED"
          }
        ]
        , accessNotConfigured
=== RUN   TestAccSdkProvider_billing_project/using_billing_account_alone_doesn't_impact_provisioning,_but_using_together_with_user_project_override_does
--- FAIL: TestAccSdkProvider_billing_project (341.69s)
    --- PASS: TestAccSdkProvider_billing_project/config_takes_precedence_over_environment_variables (10.50s)
    --- PASS: TestAccSdkProvider_billing_project/when_billing_project_is_unset_in_the_config,_environment_variables_are_used_in_a_given_order (16.22s)
    --- PASS: TestAccSdkProvider_billing_project/when_billing_project_is_set_to_an_empty_string_in_the_config_the_value_isn't_ignored_and_results_in_an_error (0.92s)
    --- FAIL: TestAccSdkProvider_billing_project/GOOGLE_CLOUD_QUOTA_PROJECT_environment_variable_interferes_with_the_billing_account_value_used (127.14s)
    --- PASS: TestAccSdkProvider_billing_project/using_billing_account_alone_doesn't_impact_provisioning,_but_using_together_with_user_project_override_does (186.92s)
FAIL

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

…of post-test destroy

SarahFrench · 2024-10-24T19:47:02Z

mmv1/third_party/terraform/provider/provider_billing_project_test.go

@@ -293,16 +300,6 @@ resource "google_pubsub_topic" "example-resource-in" {
 // the PubSub API. This allows the second apply step to succeed in a test, if needed.
 func testAccSdkProvider_billing_project_useBillingProject_setupWithApiEnabled(context map[string]interface{}) string {
 	return testAccSdkProvider_billing_project_useBillingProject_setup(context) + acctest.Nprintf(`
-# Needed for post test cleanup


This has been moved 'up' into testAccSdkProvider_billing_project_useBillingProject_setup, so it's still present in testAccSdkProvider_billing_project_useBillingProject_setupWithApiEnabled.

modular-magician · 2024-10-24T19:55:21Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 17 insertions(+), 16 deletions(-))
google-beta provider: Diff ( 2 files changed, 17 insertions(+), 16 deletions(-))

SarahFrench · 2024-10-24T20:56:53Z

TeamCity: https://hashicorp.teamcity.com/buildConfiguration/TerraformProviders_GoogleCloud_GOOGLE_MMUPSTREAMTESTS_GOOGLE_PACKAGE_PROVIDER/255120?buildTab=overview&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandBuildChangesSection=true

modular-magician · 2024-10-24T21:08:28Z

Tests analytics

Total tests: 4212
Passed tests: 3791
Skipped tests: 420
Affected tests: 1

Click here to see the affected service packages

All service packages are affected

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccSqlDatabaseInstance_Edition_Downgrade

Get to know how VCR tests work

modular-magician · 2024-10-24T21:19:25Z

🔴 Tests failed during RECORDING mode:
TestAccSqlDatabaseInstance_Edition_Downgrade [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

BBBmau · 2024-10-24T21:24:54Z

🔴 Tests failed during RECORDING mode: TestAccSqlDatabaseInstance_Edition_Downgrade [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

unrelated to PR

2024/10/24 21:17:42 [DEBUG] [transport] [server-transport 0xc002694d00] loopyWriter exiting with error: transport closed by client 
    resource_sql_database_instance_test.go:1818: Step 3/4 error: Error running apply: exit status 1
        
        Error: Error, failed to update instance settings for : googleapi: Error 400: Invalid request: Only ENTERPRISE PLUS edition supports data cache.., invalid
        
          with google_sql_database_instance.instance,
          on terraform_plugin_test.tf line 3, in resource "google_sql_database_instance" "instance":
           3: resource "google_sql_database_instance" "instance" {
        
2024/10/24 21:17:42 [DEBUG] [transport] [server-transport 0xc002a01860] Closing: EOF 
2024/10/24 21:17:42 [DEBUG] [transport] [server-transport 0xc002a01a00] Closing: EOF 
2024/10/24 21:17:42 [DEBUG] [transport] [server-transport 0xc002a01a00] loopyWriter exiting with error: transport closed by client 
2024/10/24 21:17:43 [DEBUG] [transport] [server-transport 0xc0024ccd00] Closing: EOF 
2024/10/24 21:17:43 [DEBUG] [transport] [server-transport 0xc0031809c0] Closing: EOF 
2024/10/24 21:17:43 [DEBUG] [transport] [server-transport 0xc003180b60] Closing: EOF 
2024/10/24 21:17:43 [DEBUG] [transport] [server-transport 0xc001d83ba0] Closing: EOF 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc0024ccea0] Closing: EOF 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc003180d00] Closing: EOF 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc00112ed00] Closing: EOF 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc001d83d40] Closing: EOF 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc0024ccea0] loopyWriter exiting with error: transport closed by client 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc001d83d40] loopyWriter exiting with error: transport closed by client 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc003180d00] loopyWriter exiting with error: transport closed by client 
2024/10/24 21:19:16 [DEBUG] [transport] [server-transport 0xc00112ed00] loopyWriter exiting with error: transport closed by client 
--- FAIL: TestAccSqlDatabaseInstance_Edition_Downgrade (583.02s)

issue opened here: hashicorp/terraform-provider-google#20010

BBBmau

looks good. Thanks for including the link to the teamcity test!

SarahFrench · 2024-10-25T09:41:39Z

This fix meant the test passed in the nightlies last night:

GoogleCloudPlatform#12122)

SarahFrench added 2 commits October 24, 2024 20:15

Update comments to be more clear about why skipping VCR is necessary

587e3f0

Ensure Service Usage API is always usable in new project at the time …

893f75a

…of post-test destroy

SarahFrench commented Oct 24, 2024

View reviewed changes

SarahFrench marked this pull request as ready for review October 24, 2024 21:10

SarahFrench requested a review from BBBmau October 24, 2024 21:11

BBBmau approved these changes Oct 24, 2024

View reviewed changes

SarahFrench merged commit ca3f2ed into GoogleCloudPlatform:main Oct 24, 2024
14 of 15 checks passed

BBBmau pushed a commit to BBBmau/magic-modules that referenced this pull request Nov 5, 2024

Fix nightly test failure of billing_project related acceptance tests (

be216d2

GoogleCloudPlatform#12122)

akshat-jindal-nit pushed a commit to akshat-jindal-nit/magic-modules that referenced this pull request Nov 18, 2024

Fix nightly test failure of billing_project related acceptance tests (

fa90b1e

GoogleCloudPlatform#12122)

amanMahendroo pushed a commit to amanMahendroo/magic-modules that referenced this pull request Dec 17, 2024

Fix nightly test failure of billing_project related acceptance tests (

f74b8d5

GoogleCloudPlatform#12122)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix nightly test failure of `billing_project` related acceptance tests #12122

Fix nightly test failure of `billing_project` related acceptance tests #12122

SarahFrench commented Oct 24, 2024 •

edited

Loading

SarahFrench Oct 24, 2024

modular-magician commented Oct 24, 2024

SarahFrench commented Oct 24, 2024

modular-magician commented Oct 24, 2024

modular-magician commented Oct 24, 2024

BBBmau commented Oct 24, 2024

BBBmau left a comment

SarahFrench commented Oct 25, 2024

Fix nightly test failure of billing_project related acceptance tests #12122

Fix nightly test failure of billing_project related acceptance tests #12122

Conversation

SarahFrench commented Oct 24, 2024 • edited Loading

SarahFrench Oct 24, 2024

Choose a reason for hiding this comment

modular-magician commented Oct 24, 2024

Diff report

SarahFrench commented Oct 24, 2024

modular-magician commented Oct 24, 2024

Tests analytics

Action taken

modular-magician commented Oct 24, 2024

BBBmau commented Oct 24, 2024

BBBmau left a comment

Choose a reason for hiding this comment

SarahFrench commented Oct 25, 2024

Fix nightly test failure of `billing_project` related acceptance tests #12122

Fix nightly test failure of `billing_project` related acceptance tests #12122

SarahFrench commented Oct 24, 2024 •

edited

Loading