Allow ENTERPRISE_PLUS as default Edition for POSTGRES_16

[skysarthak]

Current default: ENTERPRISE for settings.edition is not consistent with the API which treats EDITION_UNSPECIFIED as the default (refer link) when customer doesn't provide any. With PostgreSQL version 16, the default Edition will be ENTERPRISE_PLUS as documented in the PostgreSQL June 07, 2024 release notes. Thus, unset the default edition so that it can instead be computed in the API, allowing the API to use ENTERPRISE_PLUS as the default for POSTGRES_16.

b/338258663

I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
• Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

sql: removed the client-side default of `ENTERPRISE` for `edition` in `google_sql_database_instance` so that `edition` is determined by the API when unset. This will cause new instances to use `ENTERPRISE_PLUS` as the default for POSTGRES_16.

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@rileykarson, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[rileykarson]

Will a null response (including complete omission) still always imply ENTERPRISE? That's generally how these kind of cases work (where there was a fixed default that starts to vary on the API side). We should preserve the flattening logic if so, that'll keep the values recorded in state consistent.

[skysarthak]

The CloudSQL API actually uses default Edition as EDITION_UNSPECIFIED (refer: type Settings) when the customer doesn't specify the edition. Today a EDITION_UNSPECIFIED instance is configured similar to a ENTERPRISE instance but this could change within the CloudSQL API which will create inconsistent behavior for the customers when dealing directly with CloudSQL API vs Terraform provider.

I agree that flattening the value keeps the recorded states consistent, but it sort of goes against what we have documented in this regard (as pointed above). But if preserving the flattening logic makes this change safer than I agree that we should keep it.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 48 insertions(+), 18 deletions(-))
google-beta provider: Diff ( 2 files changed, 48 insertions(+), 18 deletions(-))

Breaking Change(s) Detected

The following breaking change(s) were detected within your pull request.

• Field settings.edition default value changed from ENTERPRISE to on google_sql_database_instance - reference

If you believe this detection to be incorrect please raise the concern with your reviewer.
If you intend to make this change you will need to wait for a major release window.
An override-breaking-change label can be added to allow merging.

[modular-magician]

Tests analytics

Total tests: 101
Passed tests: 7
Skipped tests: 15
Affected tests: 79

Click here to see the affected service packages

• sql

Action taken

Found 79 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccDataSourceGoogleSQLCaCerts_basic
• TestAccDataSourceSqlDatabaseInstanceLatestRecoveryTime_basic
• TestAccDataSourceSqlDatabaseInstance_basic
• TestAccDataSourceSqlDatabaseInstances_basic
• TestAccDataSourceSqlDatabaseInstances_databaseVersionFilter
• TestAccDataSourceSqlDatabaseInstances_regionFilter
• TestAccDataSourceSqlDatabaseInstances_tierFilter
• TestAccDataSourceSqlDatabase_basic
• TestAccDataSourceSqlDatabases_basic
• TestAccSQLDatabaseInstance_DefaultEdition
• TestAccSQLDatabaseInstance_DenyMaintenancePeriod
• TestAccSQLDatabase_sqlDatabaseBasicExample
• TestAccSQLDatabase_sqlDatabaseDeletionPolicyExample
• TestAccSqlClientCert_mysql
• TestAccSqlClientCert_postgres
• TestAccSqlDatabaseInstance_BackupRetention
• TestAccSqlDatabaseInstance_Edition
• TestAccSqlDatabaseInstance_Edition_Downgrade
• TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration
• TestAccSqlDatabaseInstance_EnableGoogleMlIntegration
• TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade
• TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled
• TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer
• TestAccSqlDatabaseInstance_ReplicaPromoteFailedWithMasterInstanceNameAndReplicaConfigurationPresent
• TestAccSqlDatabaseInstance_ReplicaPromoteFailedWithMasterInstanceNamePresent
• TestAccSqlDatabaseInstance_ReplicaPromoteFailedWithReplicaConfigurationPresent
• TestAccSqlDatabaseInstance_ReplicaPromoteSkippedWithNoMasterInstanceNameAndNoReplicaConfigurationPresent
• TestAccSqlDatabaseInstance_ReplicaPromoteSuccessful
• TestAccSqlDatabaseInstance_Smt
• TestAccSqlDatabaseInstance_SqlServerAuditOptionalBucket
• TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate
• TestAccSqlDatabaseInstance_Timezone
• TestAccSqlDatabaseInstance_activationPolicy
• TestAccSqlDatabaseInstance_authNets
• TestAccSqlDatabaseInstance_basicInstance_thenPSCEnabled
• TestAccSqlDatabaseInstance_basicMSSQL
• TestAccSqlDatabaseInstance_basicSecondGen
• TestAccSqlDatabaseInstance_basic_with_user_labels
• TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls
• TestAccSqlDatabaseInstance_diskspecs
• TestAccSqlDatabaseInstance_dontDeleteDefaultUserOnReplica
• TestAccSqlDatabaseInstance_encryptionKey
• TestAccSqlDatabaseInstance_encryptionKey_replicaInDifferentRegion
• TestAccSqlDatabaseInstance_highAvailability
• TestAccSqlDatabaseInstance_insights
• TestAccSqlDatabaseInstance_maintenance
• TestAccSqlDatabaseInstance_maintenanceVersion
• TestAccSqlDatabaseInstance_maintenance_update_track_week5
• TestAccSqlDatabaseInstance_multipleOperations
• TestAccSqlDatabaseInstance_replica
• TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable
• TestAccSqlDatabaseInstance_settingsDowngrade
• TestAccSqlDatabaseInstance_settings_basic
• TestAccSqlDatabaseInstance_settings_deletionProtection
• TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled
• TestAccSqlDatabaseInstance_settings_secondary
• TestAccSqlDatabaseInstance_settings_upgrade
• TestAccSqlDatabaseInstance_slave
• TestAccSqlDatabaseInstance_sqlMysqlInstancePvpExample
• TestAccSqlDatabaseInstance_updateDifferentFlagOrder
• TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled
• TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL
• TestAccSqlDatabaseInstance_useCasBasedServerCa
• TestAccSqlDatabaseInstance_useInternalCaByDefault
• TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject
• TestAccSqlDatabaseInstance_withPSCEnabled_withAllowedConsumerProjects
• TestAccSqlDatabaseInstance_withPSCEnabled_withEmptyAllowedConsumerProjects
• TestAccSqlDatabaseInstance_withPSCEnabled_withIpV4Enabled
• TestAccSqlDatabaseInstance_withPSCEnabled_withoutAllowedConsumerProjects
• TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange
• TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone
• TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica
• TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange
• TestAccSqlDatabase_basic
• TestAccSqlDatabase_instanceWithActivationPolicy
• TestAccSqlDatabase_update
• TestAccSqlUser_postgres
• TestAccSqlUser_postgresAbandon
• TestAccSqlUser_postgresIAM

Get to know how VCR tests work

[rileykarson]

Override reason: Default -> Computed can change the value received by end user configs, but this matches new API behaviour and the API behaviour is guarded on new product-internal versions (SQL database versions here). We've discussed this as acceptable offline, although would consider batching in a major release if we were closer to one.

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccDataSourceGoogleSQLCaCerts_basic [Debug log]
TestAccDataSourceSqlDatabaseInstanceLatestRecoveryTime_basic [Debug log]
TestAccDataSourceSqlDatabaseInstance_basic [Debug log]
TestAccDataSourceSqlDatabaseInstances_basic [Debug log]
TestAccDataSourceSqlDatabaseInstances_databaseVersionFilter [Debug log]
TestAccDataSourceSqlDatabaseInstances_regionFilter [Debug log]
TestAccDataSourceSqlDatabaseInstances_tierFilter [Debug log]
TestAccDataSourceSqlDatabase_basic [Debug log]
TestAccDataSourceSqlDatabases_basic [Debug log]
TestAccSQLDatabaseInstance_DenyMaintenancePeriod [Debug log]
TestAccSQLDatabase_sqlDatabaseBasicExample [Debug log]
TestAccSQLDatabase_sqlDatabaseDeletionPolicyExample [Debug log]
TestAccSqlClientCert_mysql [Debug log]
TestAccSqlClientCert_postgres [Debug log]
TestAccSqlDatabaseInstance_BackupRetention [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleMlIntegration [Debug log]
TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer [Debug log]
TestAccSqlDatabaseInstance_ReplicaPromoteFailedWithMasterInstanceNameAndReplicaConfigurationPresent [Debug log]
TestAccSqlDatabaseInstance_ReplicaPromoteFailedWithMasterInstanceNamePresent [Debug log]
TestAccSqlDatabaseInstance_ReplicaPromoteFailedWithReplicaConfigurationPresent [Debug log]
TestAccSqlDatabaseInstance_ReplicaPromoteSkippedWithNoMasterInstanceNameAndNoReplicaConfigurationPresent [Debug log]
TestAccSqlDatabaseInstance_ReplicaPromoteSuccessful [Debug log]
TestAccSqlDatabaseInstance_Smt [Debug log]
TestAccSqlDatabaseInstance_SqlServerAuditOptionalBucket [Debug log]
TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate [Debug log]
TestAccSqlDatabaseInstance_Timezone [Debug log]
TestAccSqlDatabaseInstance_activationPolicy [Debug log]
TestAccSqlDatabaseInstance_authNets [Debug log]
TestAccSqlDatabaseInstance_basicInstance_thenPSCEnabled [Debug log]
TestAccSqlDatabaseInstance_basicMSSQL [Debug log]
TestAccSqlDatabaseInstance_basicSecondGen [Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels [Debug log]
TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls [Debug log]
TestAccSqlDatabaseInstance_diskspecs [Debug log]
TestAccSqlDatabaseInstance_dontDeleteDefaultUserOnReplica [Debug log]
TestAccSqlDatabaseInstance_encryptionKey [Debug log]
TestAccSqlDatabaseInstance_encryptionKey_replicaInDifferentRegion [Debug log]
TestAccSqlDatabaseInstance_highAvailability [Debug log]
TestAccSqlDatabaseInstance_insights [Debug log]
TestAccSqlDatabaseInstance_maintenance [Debug log]
TestAccSqlDatabaseInstance_maintenanceVersion [Debug log]
TestAccSqlDatabaseInstance_maintenance_update_track_week5 [Debug log]
TestAccSqlDatabaseInstance_multipleOperations [Debug log]
TestAccSqlDatabaseInstance_replica [Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable [Debug log]
TestAccSqlDatabaseInstance_settingsDowngrade [Debug log]
TestAccSqlDatabaseInstance_settings_basic [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled [Debug log]
TestAccSqlDatabaseInstance_settings_secondary [Debug log]
TestAccSqlDatabaseInstance_settings_upgrade [Debug log]
TestAccSqlDatabaseInstance_slave [Debug log]
TestAccSqlDatabaseInstance_sqlMysqlInstancePvpExample [Debug log]
TestAccSqlDatabaseInstance_updateDifferentFlagOrder [Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled [Debug log]
TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL [Debug log]
TestAccSqlDatabaseInstance_useCasBasedServerCa [Debug log]
TestAccSqlDatabaseInstance_useInternalCaByDefault [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_withAllowedConsumerProjects [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_withEmptyAllowedConsumerProjects [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_withIpV4Enabled [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_withoutAllowedConsumerProjects [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange [Debug log]
TestAccSqlDatabase_basic [Debug log]
TestAccSqlDatabase_instanceWithActivationPolicy [Debug log]
TestAccSqlDatabase_update [Debug log]
TestAccSqlUser_postgres [Debug log]
TestAccSqlUser_postgresAbandon [Debug log]
TestAccSqlUser_postgresIAM [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

---

🔴 Tests failed during RECORDING mode:
TestAccSQLDatabaseInstance_DefaultEdition [Error message] [Debug log]
TestAccSqlDatabaseInstance_Edition [Error message] [Debug log]
TestAccSqlDatabaseInstance_Edition_Downgrade [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[rileykarson]

Test failures are due to missing business logic, and the fix is obvious given how the product behaves. I'll get that merged as #12096, any push of yours after that happens will integrate that change and tests will move forward.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 48 insertions(+), 18 deletions(-))
google-beta provider: Diff ( 2 files changed, 48 insertions(+), 18 deletions(-))

Breaking Change(s) Detected

The following breaking change(s) were detected within your pull request.

• Field settings.edition default value changed from ENTERPRISE to on google_sql_database_instance - reference

If you believe this detection to be incorrect please raise the concern with your reviewer.
If you intend to make this change you will need to wait for a major release window.
An override-breaking-change label can be added to allow merging.

[modular-magician]

Tests analytics

Total tests: 101
Passed tests: 83
Skipped tests: 15
Affected tests: 3

Click here to see the affected service packages

• sql

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccSQLDatabaseInstance_DefaultEdition
• TestAccSqlDatabaseInstance_Edition
• TestAccSqlDatabaseInstance_Edition_Downgrade

Get to know how VCR tests work

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccSQLDatabaseInstance_DefaultEdition [Debug log]
TestAccSqlDatabaseInstance_Edition [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

---

🔴 Tests failed during RECORDING mode:
TestAccSqlDatabaseInstance_Edition_Downgrade [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 45 insertions(+), 6 deletions(-))
google-beta provider: Diff ( 2 files changed, 45 insertions(+), 6 deletions(-))

Breaking Change(s) Detected

The following breaking change(s) were detected within your pull request.

• Field settings.edition default value changed from ENTERPRISE to on google_sql_database_instance - reference

If you believe this detection to be incorrect please raise the concern with your reviewer.
If you intend to make this change you will need to wait for a major release window.
An override-breaking-change label can be added to allow merging.

[modular-magician]

Tests analytics

Total tests: 101
Passed tests: 63
Skipped tests: 15
Affected tests: 23

Click here to see the affected service packages

• sql

Action taken

Found 23 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccSqlDatabaseInstance_Edition_Downgrade
• TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration
• TestAccSqlDatabaseInstance_EnableGoogleMlIntegration
• TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled
• TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer
• TestAccSqlDatabaseInstance_Smt
• TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate
• TestAccSqlDatabaseInstance_activationPolicy
• TestAccSqlDatabaseInstance_authNets
• TestAccSqlDatabaseInstance_basicInstance_thenPSCEnabled
• TestAccSqlDatabaseInstance_basicMSSQL
• TestAccSqlDatabaseInstance_basic_with_user_labels
• TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls
• TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable
• TestAccSqlDatabaseInstance_settings_deletionProtection
• TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled
• TestAccSqlDatabaseInstance_settings_upgrade
• TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled
• TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL
• TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject
• TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange
• TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange
• TestAccSqlDatabase_instanceWithActivationPolicy

Get to know how VCR tests work

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleMlIntegration [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer [Debug log]
TestAccSqlDatabaseInstance_Smt [Debug log]
TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate [Debug log]
TestAccSqlDatabaseInstance_activationPolicy [Debug log]
TestAccSqlDatabaseInstance_authNets [Debug log]
TestAccSqlDatabaseInstance_basicInstance_thenPSCEnabled [Debug log]
TestAccSqlDatabaseInstance_basicMSSQL [Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels [Debug log]
TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls [Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled [Debug log]
TestAccSqlDatabaseInstance_settings_upgrade [Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled [Debug log]
TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange [Debug log]
TestAccSqlDatabase_instanceWithActivationPolicy [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

---

🔴 Tests failed during RECORDING mode:
TestAccSqlDatabaseInstance_Edition_Downgrade [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[rileykarson]

TestAccSqlDatabaseInstance_Edition_Downgrade is failing because the test was changed in this PR, but would have failed before this change (and before #12096) if the test was written as it now is, and does represent a fiddly interaction. I'll follow up tomorrow to determine how we want to handle it. I don't think that this PR or #12096 needs to block on a solution.

Specifically when the edition is downgraded, the data cache feature must simultaneously not appear in the request / be disabled.

• Prior to Mark google_sql_database_instance.settings.data_cache_config Computed #12096, users had to specify the correct data_cache_config setting if they used ENTERPRISE_PLUS, i.e. that it was enabled. On downgrading, they had to remove or set the value to false.
• After Mark google_sql_database_instance.settings.data_cache_config Computed #12096, the field no longer needs to be specified to use ENTERPRISE_PLUS. However, on downgrade, we'll preserve the old data_cache_config value and send a request asking for it to be enabled to the server. Users need to specify false explicitly.

Given that downgrades are an unlikely flow and the field had a permadiff before, I think that's acceptable to move forward with and just fix-forward in a later release (or cherrypick a fix sooner if this assessment is wrong).