Add support for provider-defined functions #10013

SarahFrench · 2024-02-20T14:24:52Z

This PR is step two of decomposing #9990 into separate PRs.
To be merged following #10009

Release Note Template for Downstream PRs (will be copied)

provider: introduced support for provider-defined functions

modular-magician · 2024-02-20T14:35:30Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 111 insertions(+), 142 deletions(-))
Terraform Beta: Diff ( 4 files changed, 111 insertions(+), 142 deletions(-))

modular-magician · 2024-02-20T14:38:07Z

Tests analytics

Total tests: 0
Passed tests: 0
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

all service packages are affected

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$
View the build log

modular-magician · 2024-02-20T16:40:13Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 110 insertions(+), 142 deletions(-))
Terraform Beta: Diff ( 4 files changed, 110 insertions(+), 142 deletions(-))

modular-magician · 2024-02-20T17:40:46Z

Tests analytics

Total tests: 3452
Passed tests: 3101
Skipped tests: 350
Affected tests: 1

Click here to see the affected service packages

all service packages are affected

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccContainerCluster_withGatewayApiConfig

Get to know how VCR tests work

modular-magician · 2024-02-20T17:41:56Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccContainerCluster_withGatewayApiConfig[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

mmv1/provider/terraform/common~compile.yaml

SarahFrench · 2024-02-22T16:02:05Z

I'm going to hold off merging until I:

sync the feature branch with main via Sync main branch into FEATURE-BRANCH-provider-functions #10032
rebase this branch to be based off the synced feature branch

modular-magician · 2024-02-22T16:09:19Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 110 insertions(+), 142 deletions(-))
Terraform Beta: Diff ( 4 files changed, 110 insertions(+), 142 deletions(-))

modular-magician · 2024-02-22T17:11:00Z

Tests analytics

Total tests: 3457
Passed tests: 3103
Skipped tests: 350
Affected tests: 4

Click here to see the affected service packages

all service packages are affected

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccContainerCluster_withGatewayApiConfig|TestAccDataSourceGoogleServiceAccountAccessToken_basic|TestAccDataSourceGoogleServiceAccountJwt

Get to know how VCR tests work

modular-magician · 2024-02-22T17:16:55Z

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[Debug log]
TestAccDataSourceGoogleServiceAccountAccessToken_basic[Debug log]
TestAccDataSourceGoogleServiceAccountJwt[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccContainerCluster_withGatewayApiConfig[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

modular-magician · 2024-02-22T17:51:53Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 10 insertions(+))
Terraform Beta: Diff ( 2 files changed, 10 insertions(+))

modular-magician · 2024-02-22T18:53:42Z

Tests analytics

Total tests: 3462
Passed tests: 3110
Skipped tests: 350
Affected tests: 2

Click here to see the affected service packages

all service packages are affected

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccContainerCluster_withGatewayApiConfig

Get to know how VCR tests work

modular-magician · 2024-02-22T18:58:52Z

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccContainerCluster_withGatewayApiConfig[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

SarahFrench · 2024-02-22T20:05:39Z

Test failure is unrelated, see : hashicorp/terraform-provider-google#17319

* Sync main branch into FEATURE-BRANCH-provider-functions (#10032) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add support for provider-defined functions (#10013) * Update provider to support provider functions * Remove unused import * Add ability to copy .go files in functions package into downstream * Add `project_from_id` provider-defined function (#10021) * Add `location_from_id` provider-defined function (#10061) * Add `location_from_id` function, tests, docs * Fix whitespace in acc test HCL config * Sync main feature branch provider functions (#10138) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan <[email protected]> * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan <[email protected]> * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <[email protected]> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) <[email protected]> --------- Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Nick Elliot <[email protected]> * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan <[email protected]> * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick <[email protected]> * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix failing posture test (#10086) * fix failing posture test * modify posture name --------- Co-authored-by: Sneha Prasad <[email protected]> * Create support for KMS Ekmconnection resource (#10094) * Fixes two lifecycle rules with different no_age value always generates change. (#10137) * TeamCity: Add project for testing the provider functions feature branch (#10088) * Add ability to use non-default versions of Terraform in TeamCity builds * Add function to enable making build configs for single packages at a time * Add new sub project that contains 2 builds for testing provider functions the 2 builds: 1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo 2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo These builds both use an alpha release of TF 1.8.0 * Add builds for testing auto generated branches in the MM upstream repos These re-use existing VCR Roots. * Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time * Fix defect in 'Download Terraform' build step definition * Update build step to solve bug * Update build_configuration_per_package.kt * Add `ephemeral_directories` to google_workstations_workstation_config (#10042) * Add `ephemeral_directories` to google_workstations_workstation_config * Remove trailing spaces * Add a test for ephemeral_directories * Fix test * Add a test for source_image * Fix typo in test * Remove unnecessary immutable field * App Check DeviceCheck provider (#9978) * App Check DeviceCheck provider * Remove minimal example & pattern field * Add real private keys that are not useful anywhere * Limit tests to beta * Documentation (#10012) * duplicate Composer 2 argument reference * remove fields that are not supported in Composer 3 * move fields that are new in Composer 3 to Composer 3 section only * make suggested changes, add description if new versioning schema * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * add notice that composer 3 is not yet released. * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * specify composer version in section links. * specify composer version in composer 1 documentation links * add section links in composer 2 argument reference * add section links in composer 3 argument reference * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> --------- Co-authored-by: Pavel Salnikov <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Promote metric settings in compute region autoscaler to GA (#10103) * Clarified permissions for a few workflows (#10124) * Clarified permissions for a few workflows * fixed conditions for running repository-documentation workflows * Bumped actions/checkout to v4 * force downstream generation * Revert "force downstream generation" This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a. * Post statuses earlier (#10128) * Post initial statuses earlier so that we are resilient issues during cloning * force generation * Revert "force generation" This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140. * Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (#10140) --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> Co-authored-by: Yanwei Guo <[email protected]> Co-authored-by: Hamzawy63 <[email protected]> Co-authored-by: Hamza Hassan <[email protected]> Co-authored-by: Ryan Oaks <[email protected]> Co-authored-by: Sneha-at <[email protected]> Co-authored-by: Shogo Watanabe <[email protected]> Co-authored-by: Obada Alabbadi <[email protected]> Co-authored-by: NA2047 <[email protected]> Co-authored-by: Yuki Watanabe <[email protected]> Co-authored-by: Scott Suarez <[email protected]> Co-authored-by: askubis <[email protected]> Co-authored-by: Pavan Kumar Sunkara <[email protected]> Co-authored-by: ajaybgunjal1 <[email protected]> Co-authored-by: Hossein Golestani <[email protected]> Co-authored-by: rahul2393 <[email protected]> Co-authored-by: Chris <[email protected]> Co-authored-by: bcreddy-gcp <[email protected]> Co-authored-by: Matt Santa <[email protected]> Co-authored-by: Sam Levenick <[email protected]> Co-authored-by: Nick Elliot <[email protected]> Co-authored-by: wj-chen <[email protected]> Co-authored-by: Thomas Rodgers <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> Co-authored-by: yiyinglovecoding <[email protected]> Co-authored-by: Cameron Thornton <[email protected]> Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Francis (Feng) Liu <[email protected]> Co-authored-by: makuing <[email protected]> Co-authored-by: Lingkai Shen <[email protected]> Co-authored-by: Jared <[email protected]> Co-authored-by: Joakim Tangnes <[email protected]> Co-authored-by: Laurens Knoll <[email protected]> Co-authored-by: Rustem Bekmukhametov <[email protected]> Co-authored-by: xuchenma <[email protected]> Co-authored-by: Esha Goel <[email protected]> Co-authored-by: Filip 'Cherit' Szóstak <[email protected]> Co-authored-by: Benjamin Kaplan <[email protected]> Co-authored-by: jinyangtang <[email protected]> Co-authored-by: Naitian Liu <[email protected]> Co-authored-by: Sneha Prasad <[email protected]> Co-authored-by: Sneha Prasad <[email protected]> Co-authored-by: rishamchokshi <[email protected]> Co-authored-by: kautikdk <[email protected]> Co-authored-by: kangy-google <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Pavel Salnikov <[email protected]> * Skip provider-defined functions' acc tests in VCR mode (#10156) * Add `region_from_zone` provider function (#10073) * add regionFromZone function logic * add region_from_zone_test.go * add region_from_zone_internal_test.go * add region_from_zone.html.markdown * get region_from_zone_test.go passing * fix region_from_zone_internal_test.go tests * shorten test config setup time * have more explicit error for invalid input * Update mmv1/third_party/terraform/functions/region_from_zone.go Co-authored-by: Sarah French <[email protected]> * use compute_disk for better test time * Update mmv1/third_party/terraform/website/docs/functions/region_from_zone.html.markdown Co-authored-by: Sarah French <[email protected]> * Update mmv1/third_party/terraform/functions/region_from_zone_test.go * Add `location_from_id` provider-defined function (#10061) * Add `location_from_id` function, tests, docs * Fix whitespace in acc test HCL config * fix runtime error and update internal test output * Update mmv1/third_party/terraform/functions/region_from_zone_test.go Co-authored-by: Sarah French <[email protected]> * update to use projectRegionRegex * Update mmv1/third_party/terraform/functions/region_from_zone.go --------- Co-authored-by: Sarah French <[email protected]> * Refactor `location_from_id` acc test to provide its own location value instead of using ENVs (#10166) * Refactor `project_from_id` acc test to use non-networking resources (#10165) * Add `region_from_id` and `zone_from_id` provider-defined functions (#10060) * Add `region_from_id` function, tests, docs * Add `zone_from_id` function, tests, docs * Fix whitespace in acc tests' HCL config * Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go * Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go * Stop non-acceptance test from using ENVs, use handcoded values instead * Skip all provider-defined function acceptance tests in VCR * Update `zone_from_id` acc tests to not use networking-related resources * Fix error in comment * Update `region_from_id` acc tests to not use networking-related resources * Update `region_from_id` and `zone_from_id` documentation to use provisioned resources in examples * Update mmv1/third_party/terraform/website/docs/functions/zone_from_id.html.markdown * Fix node type in region_from_id acc test (#10177) * Sync main into FEATURE-BRANCH-provider-functions (#10198) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan <[email protected]> * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan <[email protected]> * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <[email protected]> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) <[email protected]> --------- Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Nick Elliot <[email protected]> * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan <[email protected]> * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick <[email protected]> * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix faili…

* Sync main branch into FEATURE-BRANCH-provider-functions (#10032) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add support for provider-defined functions (#10013) * Update provider to support provider functions * Remove unused import * Add ability to copy .go files in functions package into downstream * Add `project_from_id` provider-defined function (#10021) * Add `location_from_id` provider-defined function (#10061) * Add `location_from_id` function, tests, docs * Fix whitespace in acc test HCL config * Sync main feature branch provider functions (#10138) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan <[email protected]> * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan <[email protected]> * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <[email protected]> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) <[email protected]> --------- Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Nick Elliot <[email protected]> * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan <[email protected]> * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick <[email protected]> * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix failing posture test (#10086) * fix failing posture test * modify posture name --------- Co-authored-by: Sneha Prasad <[email protected]> * Create support for KMS Ekmconnection resource (#10094) * Fixes two lifecycle rules with different no_age value always generates change. (#10137) * TeamCity: Add project for testing the provider functions feature branch (#10088) * Add ability to use non-default versions of Terraform in TeamCity builds * Add function to enable making build configs for single packages at a time * Add new sub project that contains 2 builds for testing provider functions the 2 builds: 1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo 2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo These builds both use an alpha release of TF 1.8.0 * Add builds for testing auto generated branches in the MM upstream repos These re-use existing VCR Roots. * Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time * Fix defect in 'Download Terraform' build step definition * Update build step to solve bug * Update build_configuration_per_package.kt * Add `ephemeral_directories` to google_workstations_workstation_config (#10042) * Add `ephemeral_directories` to google_workstations_workstation_config * Remove trailing spaces * Add a test for ephemeral_directories * Fix test * Add a test for source_image * Fix typo in test * Remove unnecessary immutable field * App Check DeviceCheck provider (#9978) * App Check DeviceCheck provider * Remove minimal example & pattern field * Add real private keys that are not useful anywhere * Limit tests to beta * Documentation (#10012) * duplicate Composer 2 argument reference * remove fields that are not supported in Composer 3 * move fields that are new in Composer 3 to Composer 3 section only * make suggested changes, add description if new versioning schema * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * add notice that composer 3 is not yet released. * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * specify composer version in section links. * specify composer version in composer 1 documentation links * add section links in composer 2 argument reference * add section links in composer 3 argument reference * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> --------- Co-authored-by: Pavel Salnikov <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Promote metric settings in compute region autoscaler to GA (#10103) * Clarified permissions for a few workflows (#10124) * Clarified permissions for a few workflows * fixed conditions for running repository-documentation workflows * Bumped actions/checkout to v4 * force downstream generation * Revert "force downstream generation" This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a. * Post statuses earlier (#10128) * Post initial statuses earlier so that we are resilient issues during cloning * force generation * Revert "force generation" This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140. * Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (#10140) --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> Co-authored-by: Yanwei Guo <[email protected]> Co-authored-by: Hamzawy63 <[email protected]> Co-authored-by: Hamza Hassan <[email protected]> Co-authored-by: Ryan Oaks <[email protected]> Co-authored-by: Sneha-at <[email protected]> Co-authored-by: Shogo Watanabe <[email protected]> Co-authored-by: Obada Alabbadi <[email protected]> Co-authored-by: NA2047 <[email protected]> Co-authored-by: Yuki Watanabe <[email protected]> Co-authored-by: Scott Suarez <[email protected]> Co-authored-by: askubis <[email protected]> Co-authored-by: Pavan Kumar Sunkara <[email protected]> Co-authored-by: ajaybgunjal1 <[email protected]> Co-authored-by: Hossein Golestani <[email protected]> Co-authored-by: rahul2393 <[email protected]> Co-authored-by: Chris <[email protected]> Co-authored-by: bcreddy-gcp <[email protected]> Co-authored-by: Matt Santa <[email protected]> Co-authored-by: Sam Levenick <[email protected]> Co-authored-by: Nick Elliot <[email protected]> Co-authored-by: wj-chen <[email protected]> Co-authored-by: Thomas Rodgers <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> Co-authored-by: yiyinglovecoding <[email protected]> Co-authored-by: Cameron Thornton <[email protected]> Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Francis (Feng) Liu <[email protected]> Co-authored-by: makuing <[email protected]> Co-authored-by: Lingkai Shen <[email protected]> Co-authored-by: Jared <[email protected]> Co-authored-by: Joakim Tangnes <[email protected]> Co-authored-by: Laurens Knoll <[email protected]> Co-authored-by: Rustem Bekmukhametov <[email protected]> Co-authored-by: xuchenma <[email protected]> Co-authored-by: Esha Goel <[email protected]> Co-authored-by: Filip 'Cherit' Szóstak <[email protected]> Co-authored-by: Benjamin Kaplan <[email protected]> Co-authored-by: jinyangtang <[email protected]> Co-authored-by: Naitian Liu <[email protected]> Co-authored-by: Sneha Prasad <[email protected]> Co-authored-by: Sneha Prasad <[email protected]> Co-authored-by: rishamchokshi <[email protected]> Co-authored-by: kautikdk <[email protected]> Co-authored-by: kangy-google <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Pavel Salnikov <[email protected]> * Skip provider-defined functions' acc tests in VCR mode (#10156) * Add `region_from_zone` provider function (#10073) * add regionFromZone function logic * add region_from_zone_test.go * add region_from_zone_internal_test.go * add region_from_zone.html.markdown * get region_from_zone_test.go passing * fix region_from_zone_internal_test.go tests * shorten test config setup time * have more explicit error for invalid input * Update mmv1/third_party/terraform/functions/region_from_zone.go Co-authored-by: Sarah French <[email protected]> * use compute_disk for better test time * Update mmv1/third_party/terraform/website/docs/functions/region_from_zone.html.markdown Co-authored-by: Sarah French <[email protected]> * Update mmv1/third_party/terraform/functions/region_from_zone_test.go * Add `location_from_id` provider-defined function (#10061) * Add `location_from_id` function, tests, docs * Fix whitespace in acc test HCL config * fix runtime error and update internal test output * Update mmv1/third_party/terraform/functions/region_from_zone_test.go Co-authored-by: Sarah French <[email protected]> * update to use projectRegionRegex * Update mmv1/third_party/terraform/functions/region_from_zone.go --------- Co-authored-by: Sarah French <[email protected]> * Refactor `location_from_id` acc test to provide its own location value instead of using ENVs (#10166) * Refactor `project_from_id` acc test to use non-networking resources (#10165) * Add `region_from_id` and `zone_from_id` provider-defined functions (#10060) * Add `region_from_id` function, tests, docs * Add `zone_from_id` function, tests, docs * Fix whitespace in acc tests' HCL config * Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go * Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go * Stop non-acceptance test from using ENVs, use handcoded values instead * Skip all provider-defined function acceptance tests in VCR * Update `zone_from_id` acc tests to not use networking-related resources * Fix error in comment * Update `region_from_id` acc tests to not use networking-related resources * Update `region_from_id` and `zone_from_id` documentation to use provisioned resources in examples * Update mmv1/third_party/terraform/website/docs/functions/zone_from_id.html.markdown * Fix node type in region_from_id acc test (#10177) * Sync main into FEATURE-BRANCH-provider-functions (#10198) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan <[email protected]> * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan <[email protected]> * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <[email protected]> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) <[email protected]> --------- Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Nick Elliot <[email protected]> * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan <[email protected]> * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick <[email protected]> * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix failing posture test (#10086) * fix failing posture test * modify posture name --------- Co-authored-by: Sneha Prasad <[email protected]> * Create support for KMS Ekmconnection resource (#10094) * Fixes two lifecycle rules with different no_age value always generates change. (#10137) * TeamCity: Add project for testing the provider functions feature branch (#10088) * Add ability to use non-default versions of Terraform in TeamCity builds * Add function to enable making build configs for single packages at a time * Add new sub project that contains 2 builds for testing provider functions the 2 builds: 1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo 2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo These builds both use an alpha release of TF 1.8.0 * Add builds for testing auto generated branches in the MM upstream repos These re-use existing VCR Roots. * Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time * Fix defect in 'Download Terraform' build step definition * Update build step to solve bug * Update build_configuration_per_package.kt * Add `ephemeral_directories` to google_workstations_workstation_config (#10042) * Add `ephemeral_directories` to google_workstations_workstation_config * Remove trailing spaces * Add a test for ephemeral_directories * Fix test * Add a test for source_image * Fix typo in test * Remove unnecessary immutable field * App Check DeviceCheck provider (#9978) * App Check DeviceCheck provider * Remove minimal example & pattern field * Add real private keys that are not useful anywhere * Limit tests to beta * Documentation (#10012) * duplicate Composer 2 argument reference * remove fields that are not supported in Composer 3 * move fields that are new in Composer 3 to Composer 3 section only * make suggested changes, add description if new versioning schema * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_envi…

* Sync main branch into FEATURE-BRANCH-provider-functions (#10032) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add support for provider-defined functions (#10013) * Update provider to support provider functions * Remove unused import * Add ability to copy .go files in functions package into downstream * Add `project_from_id` provider-defined function (#10021) * Add `location_from_id` provider-defined function (#10061) * Add `location_from_id` function, tests, docs * Fix whitespace in acc test HCL config * Sync main feature branch provider functions (#10138) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan <[email protected]> * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan <[email protected]> * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <[email protected]> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) <[email protected]> --------- Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Nick Elliot <[email protected]> * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan <[email protected]> * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick <[email protected]> * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix failing posture test (#10086) * fix failing posture test * modify posture name --------- Co-authored-by: Sneha Prasad <[email protected]> * Create support for KMS Ekmconnection resource (#10094) * Fixes two lifecycle rules with different no_age value always generates change. (#10137) * TeamCity: Add project for testing the provider functions feature branch (#10088) * Add ability to use non-default versions of Terraform in TeamCity builds * Add function to enable making build configs for single packages at a time * Add new sub project that contains 2 builds for testing provider functions the 2 builds: 1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo 2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo These builds both use an alpha release of TF 1.8.0 * Add builds for testing auto generated branches in the MM upstream repos These re-use existing VCR Roots. * Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time * Fix defect in 'Download Terraform' build step definition * Update build step to solve bug * Update build_configuration_per_package.kt * Add `ephemeral_directories` to google_workstations_workstation_config (#10042) * Add `ephemeral_directories` to google_workstations_workstation_config * Remove trailing spaces * Add a test for ephemeral_directories * Fix test * Add a test for source_image * Fix typo in test * Remove unnecessary immutable field * App Check DeviceCheck provider (#9978) * App Check DeviceCheck provider * Remove minimal example & pattern field * Add real private keys that are not useful anywhere * Limit tests to beta * Documentation (#10012) * duplicate Composer 2 argument reference * remove fields that are not supported in Composer 3 * move fields that are new in Composer 3 to Composer 3 section only * make suggested changes, add description if new versioning schema * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * add notice that composer 3 is not yet released. * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * specify composer version in section links. * specify composer version in composer 1 documentation links * add section links in composer 2 argument reference * add section links in composer 3 argument reference * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <[email protected]> --------- Co-authored-by: Pavel Salnikov <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Promote metric settings in compute region autoscaler to GA (#10103) * Clarified permissions for a few workflows (#10124) * Clarified permissions for a few workflows * fixed conditions for running repository-documentation workflows * Bumped actions/checkout to v4 * force downstream generation * Revert "force downstream generation" This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a. * Post statuses earlier (#10128) * Post initial statuses earlier so that we are resilient issues during cloning * force generation * Revert "force generation" This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140. * Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (#10140) --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> Co-authored-by: Yanwei Guo <[email protected]> Co-authored-by: Hamzawy63 <[email protected]> Co-authored-by: Hamza Hassan <[email protected]> Co-authored-by: Ryan Oaks <[email protected]> Co-authored-by: Sneha-at <[email protected]> Co-authored-by: Shogo Watanabe <[email protected]> Co-authored-by: Obada Alabbadi <[email protected]> Co-authored-by: NA2047 <[email protected]> Co-authored-by: Yuki Watanabe <[email protected]> Co-authored-by: Scott Suarez <[email protected]> Co-authored-by: askubis <[email protected]> Co-authored-by: Pavan Kumar Sunkara <[email protected]> Co-authored-by: ajaybgunjal1 <[email protected]> Co-authored-by: Hossein Golestani <[email protected]> Co-authored-by: rahul2393 <[email protected]> Co-authored-by: Chris <[email protected]> Co-authored-by: bcreddy-gcp <[email protected]> Co-authored-by: Matt Santa <[email protected]> Co-authored-by: Sam Levenick <[email protected]> Co-authored-by: Nick Elliot <[email protected]> Co-authored-by: wj-chen <[email protected]> Co-authored-by: Thomas Rodgers <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]> Co-authored-by: yiyinglovecoding <[email protected]> Co-authored-by: Cameron Thornton <[email protected]> Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Francis (Feng) Liu <[email protected]> Co-authored-by: makuing <[email protected]> Co-authored-by: Lingkai Shen <[email protected]> Co-authored-by: Jared <[email protected]> Co-authored-by: Joakim Tangnes <[email protected]> Co-authored-by: Laurens Knoll <[email protected]> Co-authored-by: Rustem Bekmukhametov <[email protected]> Co-authored-by: xuchenma <[email protected]> Co-authored-by: Esha Goel <[email protected]> Co-authored-by: Filip 'Cherit' Szóstak <[email protected]> Co-authored-by: Benjamin Kaplan <[email protected]> Co-authored-by: jinyangtang <[email protected]> Co-authored-by: Naitian Liu <[email protected]> Co-authored-by: Sneha Prasad <[email protected]> Co-authored-by: Sneha Prasad <[email protected]> Co-authored-by: rishamchokshi <[email protected]> Co-authored-by: kautikdk <[email protected]> Co-authored-by: kangy-google <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Pavel Salnikov <[email protected]> * Skip provider-defined functions' acc tests in VCR mode (#10156) * Add `region_from_zone` provider function (#10073) * add regionFromZone function logic * add region_from_zone_test.go * add region_from_zone_internal_test.go * add region_from_zone.html.markdown * get region_from_zone_test.go passing * fix region_from_zone_internal_test.go tests * shorten test config setup time * have more explicit error for invalid input * Update mmv1/third_party/terraform/functions/region_from_zone.go Co-authored-by: Sarah French <[email protected]> * use compute_disk for better test time * Update mmv1/third_party/terraform/website/docs/functions/region_from_zone.html.markdown Co-authored-by: Sarah French <[email protected]> * Update mmv1/third_party/terraform/functions/region_from_zone_test.go * Add `location_from_id` provider-defined function (#10061) * Add `location_from_id` function, tests, docs * Fix whitespace in acc test HCL config * fix runtime error and update internal test output * Update mmv1/third_party/terraform/functions/region_from_zone_test.go Co-authored-by: Sarah French <[email protected]> * update to use projectRegionRegex * Update mmv1/third_party/terraform/functions/region_from_zone.go --------- Co-authored-by: Sarah French <[email protected]> * Refactor `location_from_id` acc test to provide its own location value instead of using ENVs (#10166) * Refactor `project_from_id` acc test to use non-networking resources (#10165) * Add `region_from_id` and `zone_from_id` provider-defined functions (#10060) * Add `region_from_id` function, tests, docs * Add `zone_from_id` function, tests, docs * Fix whitespace in acc tests' HCL config * Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go * Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go * Stop non-acceptance test from using ENVs, use handcoded values instead * Skip all provider-defined function acceptance tests in VCR * Update `zone_from_id` acc tests to not use networking-related resources * Fix error in comment * Update `region_from_id` acc tests to not use networking-related resources * Update `region_from_id` and `zone_from_id` documentation to use provisioned resources in examples * Update mmv1/third_party/terraform/website/docs/functions/zone_from_id.html.markdown * Fix node type in region_from_id acc test (#10177) * Sync main into FEATURE-BRANCH-provider-functions (#10198) * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan <[email protected]> * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan <[email protected]> * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <[email protected]> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <[email protected]> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) <[email protected]> --------- Co-authored-by: Stephen Lewis (Burrows) <[email protected]> * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Nick Elliot <[email protected]> * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan <[email protected]> * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick <[email protected]> * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix faili…

SarahFrench changed the title ~~Provider functions 1~~ Add support for provider-defined functions Feb 20, 2024

SarahFrench mentioned this pull request Feb 20, 2024

INITIAL DRAFT FOR FEEDBACK: Implement provider functions #9990

Closed

SarahFrench changed the base branch from main to FEATURE-BRANCH-provider-functions February 21, 2024 11:56

SarahFrench mentioned this pull request Feb 21, 2024

Add project_from_id provider-defined function #10021

Merged

SarahFrench requested a review from rileykarson February 21, 2024 16:17

SarahFrench marked this pull request as ready for review February 21, 2024 16:17

rileykarson approved these changes Feb 21, 2024

View reviewed changes

mmv1/provider/terraform/common~compile.yaml Show resolved Hide resolved

SarahFrench added 3 commits February 22, 2024 17:40

Update provider to support provider functions

72853c0

Remove unused import

769652b

Add ability to copy .go files in functions package into downstream

1c8e3a4

SarahFrench force-pushed the provider-functions-1 branch from 501d5dc to 1c8e3a4 Compare February 22, 2024 17:41

SarahFrench merged commit 2b6ef53 into FEATURE-BRANCH-provider-functions Feb 22, 2024
12 of 13 checks passed

modular-magician mentioned this pull request Feb 28, 2024

Add project_from_id provider-defined function GoogleCloudPlatform/terraform-google-conversion#2001

Merged

SarahFrench deleted the provider-functions-1 branch March 19, 2024 21:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for provider-defined functions #10013

Add support for provider-defined functions #10013

SarahFrench commented Feb 20, 2024 •

edited

Loading

modular-magician commented Feb 20, 2024

modular-magician commented Feb 20, 2024

modular-magician commented Feb 20, 2024

modular-magician commented Feb 20, 2024

modular-magician commented Feb 20, 2024

SarahFrench commented Feb 22, 2024

modular-magician commented Feb 22, 2024

modular-magician commented Feb 22, 2024

modular-magician commented Feb 22, 2024

modular-magician commented Feb 22, 2024

modular-magician commented Feb 22, 2024

modular-magician commented Feb 22, 2024

SarahFrench commented Feb 22, 2024

Add support for provider-defined functions #10013

Add support for provider-defined functions #10013

Conversation

SarahFrench commented Feb 20, 2024 • edited Loading

modular-magician commented Feb 20, 2024

Diff report

modular-magician commented Feb 20, 2024

Tests analytics

modular-magician commented Feb 20, 2024

Diff report

modular-magician commented Feb 20, 2024

Tests analytics

Action taken

modular-magician commented Feb 20, 2024

SarahFrench commented Feb 22, 2024

modular-magician commented Feb 22, 2024

Diff report

modular-magician commented Feb 22, 2024

Tests analytics

Action taken

modular-magician commented Feb 22, 2024

modular-magician commented Feb 22, 2024

Diff report

modular-magician commented Feb 22, 2024

Tests analytics

Action taken

modular-magician commented Feb 22, 2024

SarahFrench commented Feb 22, 2024

SarahFrench commented Feb 20, 2024 •

edited

Loading