Skip to content

Commit

Permalink
Release provider-defined functions (GoogleCloudPlatform#10288)
Browse files Browse the repository at this point in the history
* Sync main branch into FEATURE-BRANCH-provider-functions (#10032)

* fixes: permadiff issue if event trigger region is not specified (#9989)

* fixes: permadiff issue if event trigger region is not specified

- fixes https://github.com/hashicorp/terraform-provider-google/issues/17161

* chore: Updated cloudfunction2 test with no explicit trigger_region

* Fix GKE front matter, bad field docs (#10018)

* Fix GKE front matter, bad field docs

* Update container_cluster.html.markdown

* added default value for minPortsPerVM field at "google_compute_router_nat" (#9712)

* added default value for statis allocation

* setting the default from API

* add newline removal bash command in guidelines (#9734)

* Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009)

* Adding Data Source Forwarding Rules (#10004)

* Fix broken terraform datasource google_compute_machine_types example (#10020)

* Add check for Environment proto field before accessing in Dataflow provider (#10016)

* Add comments to skipped sweeper (#10023)

* Add comments for skipped sweepers (#10024)

* C3 version schema (#9986)

* add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress

* make build number optional

* regroup regex, cleaner comparison of versions

* correction

* Add Resource Manager Tags support to 'google_container_cluster' (#9531)

* resourceManagerTags added to Cluster Node Config schema

* update beta tag

* add cluster and node proto tests

* add expand and flatten proto

* removed beta tag

* added to documentation

* added resource manager tags to auto pilot

* migrating resourceManagerTags tests

* migrating node_pools test

* migrating additional tests

* minor fixes

* fixing tests

* add in-place update support

* fixed tests

* fixed annotations

* validated clusters and node pools tests. Isolated node pool auto config

* isolated resource manager tags from docs

* fixed permission issue

* fixed spaces

* fixed non determinism on tag keys

* removed auto_pilot rmts

* fixed time_sleep

* add depends_on to IAM policies

* Add volume replication support for Google Cloud NetApp Volumes (#9816)

* Initial replication commit

* Cleanup work

- Renamed a lot of files to make clear which resource the belong to
- Updated documentation for resource fields
- Renamed a few resource fields and changed some types
- Disabled the custom code for now. Needs to be discussed first

* Update example file

* Updated example file

* Major updates

- Reorganisation of block
- Reorganisation of fields to match API documentation
- Updated example parameters
- Added missing API fields
- Improved descriptions
-

* For replication deletion, stop replication first

* Add support for deleting destination volume on replication delete

* Make volumes deletable in presence of snapshots.

This change will be PRed for volume resource independently. Adding it here while it is not in main.

* Improving debug error message

* yaml check and format fix

* Add wait for mirror to initialize.

Required to run destroy shortly after create.

* Wait on destroy, not on create

* Make deleting a replication more robust

- doc improvements
- started to implement stop/resume. More work required.
- renamed a few files to better reflect what they are good for

* adding support for stop/resume

* yamlformat and lint

* Add force delete to delete volumes with nested snapshots

* resource test first version

* More changes to make tests solid

- Introduced new parameter to wait for mirror_status==MIRRORED
- more mirror state reconciliation

* Test updates

* few cleanups

* Make virtual field verifies happy

* Minor test improvements

* More fine tuning

- Remove merge conflict in volume.yaml
- make generated test work
- make output field work
- ignore_read for virtual fields

* Resource name change as suggested by @slevenick

* Remove snapshot code block and fix typo

* Detect manual stop/resume actions

* Remove ignore_read for deletion_policy

* - Made destinationVolumeParameters immutable. It still requires ignore_read.
- removed ignore_read from virtual_fields

* destinationVolumeParameters are only evaluated at create. Make the immutable.

* Name cleanups and comment improvements

* removed comment

Co-authored-by: Shuya Ma <[email protected]>

* tabs to spaces in resource block

Co-authored-by: Shuya Ma <[email protected]>

* Updates to address review comments

- make wait_for_mirror also work for stop/resume, additionally to create
- convert tabs in test resource blocks to spaces
- fix typos

* Rewording of comments

Co-authored-by: Shuya Ma <[email protected]>

---------

Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>

* Ensured that beta runs in TeamCity use only beta paths (#10025)

* Ensured that beta runs in TeamCity use only beta paths

* Added tests for sweeper package path

---------

Co-authored-by: Deepak Kumar <[email protected]>
Co-authored-by: Riley Karson <[email protected]>
Co-authored-by: Max Portocarrero CI&T <[email protected]>
Co-authored-by: Mauricio Alvarez Leon <[email protected]>
Co-authored-by: Jesse Liddle <[email protected]>
Co-authored-by: Wessel Blokzijl <[email protected]>
Co-authored-by: Jack McCluskey <[email protected]>
Co-authored-by: hao-nan-li <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Oliver Krause <[email protected]>
Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Add support for provider-defined functions (#10013)

* Update provider to support provider functions

* Remove unused import

* Add ability to copy .go files in functions package into downstream

* Add `project_from_id` provider-defined function (#10021)

* Add `location_from_id` provider-defined function (#10061)

* Add `location_from_id` function, tests, docs

* Fix whitespace in acc test HCL config

* Sync main feature branch provider functions (#10138)

* fixes: permadiff issue if event trigger region is not specified (#9989)

* fixes: permadiff issue if event trigger region is not specified

- fixes https://github.com/hashicorp/terraform-provider-google/issues/17161

* chore: Updated cloudfunction2 test with no explicit trigger_region

* Fix GKE front matter, bad field docs (#10018)

* Fix GKE front matter, bad field docs

* Update container_cluster.html.markdown

* added default value for minPortsPerVM field at "google_compute_router_nat" (#9712)

* added default value for statis allocation

* setting the default from API

* add newline removal bash command in guidelines (#9734)

* Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009)

* Adding Data Source Forwarding Rules (#10004)

* Fix broken terraform datasource google_compute_machine_types example (#10020)

* Add check for Environment proto field before accessing in Dataflow provider (#10016)

* Add comments to skipped sweeper (#10023)

* Add comments for skipped sweepers (#10024)

* C3 version schema (#9986)

* add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress

* make build number optional

* regroup regex, cleaner comparison of versions

* correction

* Add Resource Manager Tags support to 'google_container_cluster' (#9531)

* resourceManagerTags added to Cluster Node Config schema

* update beta tag

* add cluster and node proto tests

* add expand and flatten proto

* removed beta tag

* added to documentation

* added resource manager tags to auto pilot

* migrating resourceManagerTags tests

* migrating node_pools test

* migrating additional tests

* minor fixes

* fixing tests

* add in-place update support

* fixed tests

* fixed annotations

* validated clusters and node pools tests. Isolated node pool auto config

* isolated resource manager tags from docs

* fixed permission issue

* fixed spaces

* fixed non determinism on tag keys

* removed auto_pilot rmts

* fixed time_sleep

* add depends_on to IAM policies

* Add volume replication support for Google Cloud NetApp Volumes (#9816)

* Initial replication commit

* Cleanup work

- Renamed a lot of files to make clear which resource the belong to
- Updated documentation for resource fields
- Renamed a few resource fields and changed some types
- Disabled the custom code for now. Needs to be discussed first

* Update example file

* Updated example file

* Major updates

- Reorganisation of block
- Reorganisation of fields to match API documentation
- Updated example parameters
- Added missing API fields
- Improved descriptions
-

* For replication deletion, stop replication first

* Add support for deleting destination volume on replication delete

* Make volumes deletable in presence of snapshots.

This change will be PRed for volume resource independently. Adding it here while it is not in main.

* Improving debug error message

* yaml check and format fix

* Add wait for mirror to initialize.

Required to run destroy shortly after create.

* Wait on destroy, not on create

* Make deleting a replication more robust

- doc improvements
- started to implement stop/resume. More work required.
- renamed a few files to better reflect what they are good for

* adding support for stop/resume

* yamlformat and lint

* Add force delete to delete volumes with nested snapshots

* resource test first version

* More changes to make tests solid

- Introduced new parameter to wait for mirror_status==MIRRORED
- more mirror state reconciliation

* Test updates

* few cleanups

* Make virtual field verifies happy

* Minor test improvements

* More fine tuning

- Remove merge conflict in volume.yaml
- make generated test work
- make output field work
- ignore_read for virtual fields

* Resource name change as suggested by @slevenick

* Remove snapshot code block and fix typo

* Detect manual stop/resume actions

* Remove ignore_read for deletion_policy

* - Made destinationVolumeParameters immutable. It still requires ignore_read.
- removed ignore_read from virtual_fields

* destinationVolumeParameters are only evaluated at create. Make the immutable.

* Name cleanups and comment improvements

* removed comment

Co-authored-by: Shuya Ma <[email protected]>

* tabs to spaces in resource block

Co-authored-by: Shuya Ma <[email protected]>

* Updates to address review comments

- make wait_for_mirror also work for stop/resume, additionally to create
- convert tabs in test resource blocks to spaces
- fix typos

* Rewording of comments

Co-authored-by: Shuya Ma <[email protected]>

---------

Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>

* Ensured that beta runs in TeamCity use only beta paths (#10025)

* Ensured that beta runs in TeamCity use only beta paths

* Added tests for sweeper package path

* Adds description for default value of `cpuidle` field (#10005)

* add description

* remove tail space

* Add location field in DNS authorization resource. (#9968)

* Add location field in DNS authorization resource

* Fixed the provider version of the updated config

---------

Co-authored-by: Hamza Hassan <[email protected]>

* Add discoveryengine service (#10017)

* Add validations for Composer 2/3 only fields (#9917)

* block upgrade to composer 3

* make isComposer3 more generic, correct imageVersionChangeValidationFunc

* added validation for Composer 2/3 specific fields

* add tests for validation

* add checks in flattenComposerEnvironmentConfig

* Update attributes of fields not used in Composer 3

* make customizeDiff functions beta only

* remove Computed from gke_cluster

* remove Optional instead of Computed

* add envCfg.PrivateEnvironmentConfig is nil check

* modify isComposer3 to take string

* minor correction to avoid merge conflicts

* Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034)

Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319

* Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011)

Co-authored-by: Hamza Hassan <[email protected]>

* Added missing quote marks to TEST_PREFIX (#10035)

Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied

* Promote enable_confidential_storage from beta to GA (#9993)

* Add IAM resources for Cloud Deploy Target (#9927)

* Add IAM resources for Cloud Deploy Target

* update

* add back import_format

* Add comments to skipped sweepers (#10028)

* Add sweepers for gkebackupPlan and restorePlan (#10026)

* add `remote_function_options` to bigquery_routine (#9893)

* Add dataGovernanceType and remoteFunctionOptions to bigquery_routine

* add function-sources.zip to biguquery fixtures

* fix resource names in TestAccBigQueryRoutine

* add bigquery routine remote function example

* Adding documentaiton build_worker_pool (#10022)

* docs: fix import example in google_project_iam (#10007)

Fix the example in the `google_project_iam` documentation.

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import

* ScottSuarez not on vacation anymore (#10052)

* I'm on vaction :)

* ScottSuarez not vaction anymore

* ScottSuarez not vaction anymore

* Promote (R)IGM.AllInstancesConfig to V1 (#9964)

* Promote metric settings in compute region autoscaler to GA (#10045)

* Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040)

* Retry delete network step while creating a google project. (#10046)

* Adding the namespace_labels field to the GKE Hub Scope resource (#9972)

* initial commit for scope-level namespace labels

* Add validation exceptions for the  field of the GKEHub Scope and Namespace resources to be of type

* Undoing unnecessary changes

* Fixing the type of Scope namespace_labels field

* Support connectivity fields for Composer 3 (#9889)

* add composer_network_attachment

* indicate conflicting configs

* commas

* no need for bidirectional conflict definition (generates double errors)

* protect nit PrivateClusterConfig

* for optimizing error messages about conflicts

* add 2 step update for composer_network_attachment

* make composer_network_attachment available in beta only

* add two step update for network and subnetwork

* corrections in 2 phase update for network/subnetwork

* remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working)

* added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains

* add docs for composer_network_attachment

* add test for network attachment

* ignore non empty plan in network attachment test

* add networkAttachment update and conflicting fields tests

* add ComputedIf for network, change isComposer3

* minor corrections

* remove computedIf

* filter equivalent values of network/subnetwork in ForceNewIf

* simplify ResourceConditionFunc, add beta/ga version conditions

* typo

* more general comparison of network references

* use tpgresource.CompareSelfLinkRelativePaths instead of custom function

* modify isComposer3 to avoid merge conflicts later.

* removing this since documentation is handled in other PR and to avoid conflicts while merging.

* replace ExpectNonEmptyPlan with lifecycle.ignore_changes

* add testcase for changing network attachment to network and subnetwork

* add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork

* modify tests to use different network for attachment

* remove unused constant

* remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes)

* Fix resource name to have sweepable prefix (#9984)

* doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049)

* Promote Cloud Deploy Automation to GA (#10043)

* Add Cloud Deploy Automation

* Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384

* Add update test for the new automation resource

* fix lint errors

* fix lint errors

* fix errors in GA provider test

* add the missing test

* add a full test

* Attempt to fix the acceptance test errors

* fix a lint error

* mark labels as default_from_api:true

* fix advance rollout rule definition

* Mark service_account as 'ignore_read: true'

* Fix test errors

* Resolve review comments

* Format test file

* promote google_clouddeploy_automation to GA

* promote google_clouddeploy_automation to GA

* fix lint errors

* Add cloudquota service (#10063)

* TeamCity: Add validating GHAs that check Kotlin files listing services (#9999)

* services diff gha

* add needs

* use setup-go@v3

* add artifacts

* remove checkout

* update directory

* update common-copy.yaml

* update common-copy.yaml

* remove data print

* add pull-request check on teamcitydiff

* update paths in GHA

* add exit code in diff_check

* update exit output

* add exit code diff_check.go

* services_beta diff check support

* beta services support args

* add arg for services kt file

* services_ga.kt in gha

* move diff_check to tools folder

* remove go command

* provide path for artifacts download

* add cd in Build Provider

* update diff_check_beta.yml

* fix paths

* yml typo beta

* directory testing

* directory testing

* fix directory issues with commands

* refactor tools/teamcity-diff-check/main.go

* directory check

* typo

* remove common copy

* generate both providers into one gha

* type artifacts name

* add -o flag into artifacts download

* use merge multiple artifacts

* use artifact@v4

* use mmv1 directory for services file

* remove .zip

* output missing services from diff

* find artifacts folder

* use beta provider as only artifact

* include google ga and beta in gha

* artifact name

* -o flag in artifacts

* output stdout from go list command

* diff test

* output beta main.go

* remove err check

* remove services print / refactor for final review

* output cleanup

* uncomment go list err

* regex assert

* add check for new services in PR / set googleServices as groundtruth in serviceDifference function

* add actions/checkout

* logic fix

* add needs: check-pr

* add mmv1/products for pull-request event

* set force depth 0

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* use text file for services, add if statement in gha

* directory typo

* typo

* typo in grep

* quotations on if statement

* unquote GITHUB_OUTPUT

* add quotes on output.services

* if test

* echo output

* github_output error fix

* services.outputs

* proper if statement syntax gpush

* invalid 0 format

* add missing assignment

* fix openfile error

* remove types in pull_request trigger

* send txt files to correct directory

* check services directory for ls command

* add weekly workflow for teamcity diff check

* change name for weekly check

---------

Co-authored-by: Sarah French <[email protected]>

* Fixed `runtime` in  app engine tests (#10047)

* Fix google_notebooks_instance labels not being able to be updated (#9933)

* add labels to runtime update test

* Add labels to ImportStateVerifyIgnore

* fix indentation

* Add IAM resources for Cloud Deploy Custom Target Type (#10051)

* Add IAM resources for Cloud Deploy Custom Target Type

* Add base url override for Cloud Deploy Custom Target Type IAM policy

* Update import format for IAM policy and primary resource name to examples for IAM generated tests

* Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075)

* Remove unneeded GHA steps - do we need to cache here?

* Whitespace

* Test change with fake new service

* Resolve build error in fake service

* Re-order GHA steps and add name

* Add name to checkout step

* Add comments, remove extra whitespace

* Remove fake new service

* Add output to identifiers, check for project as well as projectsId (#10081)

* Add output to identifiers, check for project as well as projectsId

* Correct identifier behavior

* Rubocop

* remove nickelliot/zli82016 from vacation reviewers (#10080)

* Update membership.go

* Update membership.go

* Update enrolled_teams.yml (#10082)

* Fix resource_bigquery_dataset ID validation (#10027)

* fix resource_bigquery_dataset ID validation

* skip the new validation tests for VCR

* update dataset IDs to be valid in terraform-google-conversion example files

* update more dataset IDs in the CAI files

* Prevent duplicate entries in vcr results (#10068)

* Prevent duplicate entries in vcr results

* Create empty map

* Only collect result from current test

* Split github tokens (#9988)

* Split github tokens

* Update .ci/gcb-generate-diffs-new.yml

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Remove redundant downstreams token

* Make diff processor use new token

* Update path to markdown file

* Replace GITHUB_TOKEN

* Make github tokens optional for generate downstream

* Allow either github token to be used

* Replace GITHUB_TOKEN

* Move environment variable lookup out of constructor

* Update .ci/magician/vcr/tester.go

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Add downstream token

* Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC

* Apply suggestions from code review

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

---------

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Instance lifecycle policy default action on failure implementation (#10050)

* Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076)

* Add wait in acc test to avoid 'API has not been used in project' error

* Add external provider `time`

* Fix defect when referencing google_project resource

* Set GITHUB_TOKEN environment variable for hub (#10091)

* fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)

* make go-converted YAML compatible with go compiler (#10033)

Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>

* Revert "Promote metric settings in compute region autoscaler to GA" (#10095)

This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d.

* Add type field to DNS authorization reosurce (#10030)

* Add type field to DNS authorization reosurce

* Add an example for regional DNS authorization

* Add an example for regional certs using regional DNS auth

* Fix lint errors

* Fix typo in the enum values

* Add type field in regional dns auth example

---------

Co-authored-by: Hamza Hassan <[email protected]>

* TeamCity: Refactor config tests (#9956)

* Add test util for locating a subproject 2 layers deep

* Refactor tests to use new helper function

* Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value

* Refactor new sweeper tests

* Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098)

* TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099)

* Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096)

* subnetwork and service_account_email params described (17211) (#10102)

* subnetwork and service_account_email params described (17211)

* Doc adjustment

* Update provider name in various spots (#10092)

* Go compiler skeleton and early terraform.rb provider code (#10104)

* Remove provider-related caches to address cache issues (#10097)

We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out.

* Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093)

* Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111)

This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174.

* Add support for string --> object map for DCL resources (#10039)

* Add support for string --> object map for DCL resources

* Fix whitespace in template

* Add test, need updated DCL to work

* Add override for key name

* Both overrides

* Update docs, prepend key for maps

* Update bundle descriptions

* Comment update

* fix permadiff by reading empty docker_config field (#10113)

* Import package golang.org/x/exp/slices in MMv1 go compiler (#10108)

* fix(kms): certificate chain type to array of strings (#9582)

* Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110)

* Update all uses of `actions/cache` to v3

* Update all uses of `actions/checkout` to v4

* Update all uses of `actions/upload-artifact` to v3.1.0

* Update all uses of `actions/setup-go` to v4.0.0

* Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2

[v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20

The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2

* Address dependency issues in TestAccFirestoreField_* tests (#9957)

* Add additional wait in TestAccFirestoreField_* tests

* Boost wait in test to 6 minutes

* Add dependency between database and service to control delete order

* Update dependency to explicitly include project

* Make firestore fields be removed from state when they're 'deleted'

* Add `destroy_duration`

* Remove from state after log line that uses id value

* Update destory check to accept a 403 as valid

* Remove unneeded changes in PR

* Remove call to SetId

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895)

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource

* Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes

* Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058)

* Handwrite sweepers for networkConnectivity hubs and spokes (#10069)

* Fix wrong variable in override logic (#10125)

* Make missing test detector reader into a separate module (#10115)

* Use old github token if new tokens are not available (#10114)

* Use old github token if new tokens are not available

* Add lookup function for github token or fallback

* Add fall back

* Add fallback

* Add test for PATCH environment type (#10131)

* Add new resource for Application for Apphub (#10079)

* Add new resource for Application for Apphub

* Add new resource for Application for Apphub

* Enable Apphub API in test cases

* Fix precheck error

* Fix precheck error

* Resolve comments

* Fix lint error

* Fix examples

* Remove apphub from teamcity config

This causes PR build failures, I'll add it later

---------

Co-authored-by: Sam Levenick <[email protected]>

* Add apphub (#10133)

Adds AppHub to APIs to activate

* Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031)

* Support service-level min instances in Cloud Run v2 services. (#10083)

* feat: Add CMEK support for Firestore database in Beta provider (#10044)

* Modify database.yaml to add cmek related fields

* Add two examples for firestore CMEK databases for testing

* Resolve trailing space

* Update documentation for kmsKeyName field

* Resolve trailing space

* Make field immutable

* Update field documentation

* Update field description

* Allow vcenter_network to be set (#9946)

* Allow vcenter_network to be set

* set default from api

* Sort resources and set in product in go compiler (#10135)

* fix failing posture test (#10086)

* fix failing posture test

* modify posture name

---------

Co-authored-by: Sneha Prasad <[email protected]>

* Create support for KMS Ekmconnection resource (#10094)

* Fixes two lifecycle rules with different no_age value always generates change. (#10137)

* TeamCity: Add project for testing the provider functions feature branch (#10088)

* Add ability to use non-default versions of Terraform in TeamCity builds

* Add function to enable making build configs for single packages at a time

* Add new sub project that contains 2 builds for testing provider functions

the 2 builds:
1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo
2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo

These builds both use an alpha release of TF 1.8.0

* Add builds for testing auto generated branches in the MM upstream repos

These re-use existing VCR Roots.

* Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time

* Fix defect in 'Download Terraform' build step definition

* Update build step to solve bug

* Update build_configuration_per_package.kt

* Add `ephemeral_directories` to google_workstations_workstation_config (#10042)

* Add `ephemeral_directories` to google_workstations_workstation_config

* Remove trailing spaces

* Add a test for ephemeral_directories

* Fix test

* Add a test for source_image

* Fix typo in test

* Remove unnecessary immutable field

* App Check DeviceCheck provider (#9978)

* App Check DeviceCheck provider

* Remove minimal example & pattern field

* Add real private keys that are not useful anywhere

* Limit tests to beta

* Documentation (#10012)

* duplicate Composer 2 argument reference

* remove fields that are not supported in Composer 3

* move fields that are new in Composer 3 to Composer 3 section only

* make suggested changes, add description if new versioning schema

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* add notice that composer 3 is not yet released.

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* specify composer version in section links.

* specify composer version in composer 1 documentation links

* add section links in composer 2 argument reference

* add section links in composer 3 argument reference

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

---------

Co-authored-by: Pavel Salnikov <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Promote metric settings in compute region autoscaler to GA (#10103)

* Clarified permissions for a few workflows (#10124)

* Clarified permissions for a few workflows

* fixed conditions for running repository-documentation workflows

* Bumped actions/checkout to v4

* force downstream generation

* Revert "force downstream generation"

This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a.

* Post statuses earlier (#10128)

* Post initial statuses earlier so that we are resilient issues during cloning

* force generation

* Revert "force generation"

This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140.

* Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (#10140)

---------

Co-authored-by: Deepak Kumar <[email protected]>
Co-authored-by: Riley Karson <[email protected]>
Co-authored-by: Max Portocarrero CI&T <[email protected]>
Co-authored-by: Mauricio Alvarez Leon <[email protected]>
Co-authored-by: Jesse Liddle <[email protected]>
Co-authored-by: Wessel Blokzijl <[email protected]>
Co-authored-by: Jack McCluskey <[email protected]>
Co-authored-by: hao-nan-li <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Oliver Krause <[email protected]>
Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
Co-authored-by: Yanwei Guo <[email protected]>
Co-authored-by: Hamzawy63 <[email protected]>
Co-authored-by: Hamza Hassan <[email protected]>
Co-authored-by: Ryan Oaks <[email protected]>
Co-authored-by: Sneha-at <[email protected]>
Co-authored-by: Shogo Watanabe <[email protected]>
Co-authored-by: Obada Alabbadi <[email protected]>
Co-authored-by: NA2047 <[email protected]>
Co-authored-by: Yuki Watanabe <[email protected]>
Co-authored-by: Scott Suarez <[email protected]>
Co-authored-by: askubis <[email protected]>
Co-authored-by: Pavan Kumar Sunkara <[email protected]>
Co-authored-by: ajaybgunjal1 <[email protected]>
Co-authored-by: Hossein Golestani <[email protected]>
Co-authored-by: rahul2393 <[email protected]>
Co-authored-by: Chris <[email protected]>
Co-authored-by: bcreddy-gcp <[email protected]>
Co-authored-by: Matt Santa <[email protected]>
Co-authored-by: Sam Levenick <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>
Co-authored-by: wj-chen <[email protected]>
Co-authored-by: Thomas Rodgers <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
Co-authored-by: yiyinglovecoding <[email protected]>
Co-authored-by: Cameron Thornton <[email protected]>
Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Francis (Feng) Liu <[email protected]>
Co-authored-by: makuing <[email protected]>
Co-authored-by: Lingkai Shen <[email protected]>
Co-authored-by: Jared <[email protected]>
Co-authored-by: Joakim Tangnes <[email protected]>
Co-authored-by: Laurens Knoll <[email protected]>
Co-authored-by: Rustem Bekmukhametov <[email protected]>
Co-authored-by: xuchenma <[email protected]>
Co-authored-by: Esha Goel <[email protected]>
Co-authored-by: Filip 'Cherit' Szóstak <[email protected]>
Co-authored-by: Benjamin Kaplan <[email protected]>
Co-authored-by: jinyangtang <[email protected]>
Co-authored-by: Naitian Liu <[email protected]>
Co-authored-by: Sneha Prasad <[email protected]>
Co-authored-by: Sneha Prasad <[email protected]>
Co-authored-by: rishamchokshi <[email protected]>
Co-authored-by: kautikdk <[email protected]>
Co-authored-by: kangy-google <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Pavel Salnikov <[email protected]>

* Skip provider-defined functions' acc tests in VCR mode (#10156)

* Add `region_from_zone` provider function (#10073)

* add regionFromZone function logic

* add region_from_zone_test.go

* add region_from_zone_internal_test.go

* add region_from_zone.html.markdown

* get region_from_zone_test.go passing

* fix region_from_zone_internal_test.go tests

* shorten test config setup time

* have more explicit error for invalid input

* Update mmv1/third_party/terraform/functions/region_from_zone.go

Co-authored-by: Sarah French <[email protected]>

* use compute_disk for better test time

* Update mmv1/third_party/terraform/website/docs/functions/region_from_zone.html.markdown

Co-authored-by: Sarah French <[email protected]>

* Update mmv1/third_party/terraform/functions/region_from_zone_test.go

* Add `location_from_id` provider-defined function (#10061)

* Add `location_from_id` function, tests, docs

* Fix whitespace in acc test HCL config

* fix runtime error and update internal test output

* Update mmv1/third_party/terraform/functions/region_from_zone_test.go

Co-authored-by: Sarah French <[email protected]>

* update to use projectRegionRegex

* Update mmv1/third_party/terraform/functions/region_from_zone.go

---------

Co-authored-by: Sarah French <[email protected]>

* Refactor `location_from_id` acc test to provide its own location value instead of using ENVs (#10166)

* Refactor `project_from_id` acc test to use non-networking resources (#10165)

* Add `region_from_id` and `zone_from_id` provider-defined functions (#10060)

* Add `region_from_id` function, tests, docs

* Add `zone_from_id` function, tests, docs

* Fix whitespace in acc tests' HCL config

* Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go

* Update mmv1/third_party/terraform/functions/zone_from_id_internal_test.go

* Stop non-acceptance test from using ENVs, use handcoded values instead

* Skip all provider-defined function acceptance tests in VCR

* Update `zone_from_id` acc tests to not use networking-related resources

* Fix error in comment

* Update `region_from_id` acc tests to not use networking-related resources

* Update `region_from_id` and `zone_from_id` documentation to use provisioned resources in examples

* Update mmv1/third_party/terraform/website/docs/functions/zone_from_id.html.markdown

* Fix node type in region_from_id acc test (#10177)

* Sync main into FEATURE-BRANCH-provider-functions (#10198)

* fixes: permadiff issue if event trigger region is not specified (#9989)

* fixes: permadiff issue if event trigger region is not specified

- fixes https://github.com/hashicorp/terraform-provider-google/issues/17161

* chore: Updated cloudfunction2 test with no explicit trigger_region

* Fix GKE front matter, bad field docs (#10018)

* Fix GKE front matter, bad field docs

* Update container_cluster.html.markdown

* added default value for minPortsPerVM field at "google_compute_router_nat" (#9712)

* added default value for statis allocation

* setting the default from API

* add newline removal bash command in guidelines (#9734)

* Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009)

* Adding Data Source Forwarding Rules (#10004)

* Fix broken terraform datasource google_compute_machine_types example (#10020)

* Add check for Environment proto field before accessing in Dataflow provider (#10016)

* Add comments to skipped sweeper (#10023)

* Add comments for skipped sweepers (#10024)

* C3 version schema (#9986)

* add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress

* make build number optional

* regroup regex, cleaner comparison of versions

* correction

* Add Resource Manager Tags support to 'google_container_cluster' (#9531)

* resourceManagerTags added to Cluster Node Config schema

* update beta tag

* add cluster and node proto tests

* add expand and flatten proto

* removed beta tag

* added to documentation

* added resource manager tags to auto pilot

* migrating resourceManagerTags tests

* migrating node_pools test

* migrating additional tests

* minor fixes

* fixing tests

* add in-place update support

* fixed tests

* fixed annotations

* validated clusters and node pools tests. Isolated node pool auto config

* isolated resource manager tags from docs

* fixed permission issue

* fixed spaces

* fixed non determinism on tag keys

* removed auto_pilot rmts

* fixed time_sleep

* add depends_on to IAM policies

* Add volume replication support for Google Cloud NetApp Volumes (#9816)

* Initial replication commit

* Cleanup work

- Renamed a lot of files to make clear which resource the belong to
- Updated documentation for resource fields
- Renamed a few resource fields and changed some types
- Disabled the custom code for now. Needs to be discussed first

* Update example file

* Updated example file

* Major updates

- Reorganisation of block
- Reorganisation of fields to match API documentation
- Updated example parameters
- Added missing API fields
- Improved descriptions
-

* For replication deletion, stop replication first

* Add support for deleting destination volume on replication delete

* Make volumes deletable in presence of snapshots.

This change will be PRed for volume resource independently. Adding it here while it is not in main.

* Improving debug error message

* yaml check and format fix

* Add wait for mirror to initialize.

Required to run destroy shortly after create.

* Wait on destroy, not on create

* Make deleting a replication more robust

- doc improvements
- started to implement stop/resume. More work required.
- renamed a few files to better reflect what they are good for

* adding support for stop/resume

* yamlformat and lint

* Add force delete to delete volumes with nested snapshots

* resource test first version

* More changes to make tests solid

- Introduced new parameter to wait for mirror_status==MIRRORED
- more mirror state reconciliation

* Test updates

* few cleanups

* Make virtual field verifies happy

* Minor test improvements

* More fine tuning

- Remove merge conflict in volume.yaml
- make generated test work
- make output field work
- ignore_read for virtual fields

* Resource name change as suggested by @slevenick

* Remove snapshot code block and fix typo

* Detect manual stop/resume actions

* Remove ignore_read for deletion_policy

* - Made destinationVolumeParameters immutable. It still requires ignore_read.
- removed ignore_read from virtual_fields

* destinationVolumeParameters are only evaluated at create. Make the immutable.

* Name cleanups and comment improvements

* removed comment

Co-authored-by: Shuya Ma <[email protected]>

* tabs to spaces in resource block

Co-authored-by: Shuya Ma <[email protected]>

* Updates to address review comments

- make wait_for_mirror also work for stop/resume, additionally to create
- convert tabs in test resource blocks to spaces
- fix typos

* Rewording of comments

Co-authored-by: Shuya Ma <[email protected]>

---------

Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>

* Ensured that beta runs in TeamCity use only beta paths (#10025)

* Ensured that beta runs in TeamCity use only beta paths

* Added tests for sweeper package path

* Adds description for default value of `cpuidle` field (#10005)

* add description

* remove tail space

* Add location field in DNS authorization resource. (#9968)

* Add location field in DNS authorization resource

* Fixed the provider version of the updated config

---------

Co-authored-by: Hamza Hassan <[email protected]>

* Add discoveryengine service (#10017)

* Add validations for Composer 2/3 only fields (#9917)

* block upgrade to composer 3

* make isComposer3 more generic, correct imageVersionChangeValidationFunc

* added validation for Composer 2/3 specific fields

* add tests for validation

* add checks in flattenComposerEnvironmentConfig

* Update attributes of fields not used in Composer 3

* make customizeDiff functions beta only

* remove Computed from gke_cluster

* remove Optional instead of Computed

* add envCfg.PrivateEnvironmentConfig is nil check

* modify isComposer3 to take string

* minor correction to avoid merge conflicts

* Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034)

Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319

* Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011)

Co-authored-by: Hamza Hassan <[email protected]>

* Added missing quote marks to TEST_PREFIX (#10035)

Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied

* Promote enable_confidential_storage from beta to GA (#9993)

* Add IAM resources for Cloud Deploy Target (#9927)

* Add IAM resources for Cloud Deploy Target

* update

* add back import_format

* Add comments to skipped sweepers (#10028)

* Add sweepers for gkebackupPlan and restorePlan (#10026)

* add `remote_function_options` to bigquery_routine (#9893)

* Add dataGovernanceType and remoteFunctionOptions to bigquery_routine

* add function-sources.zip to biguquery fixtures

* fix resource names in TestAccBigQueryRoutine

* add bigquery routine remote function example

* Adding documentaiton build_worker_pool (#10022)

* docs: fix import example in google_project_iam (#10007)

Fix the example in the `google_project_iam` documentation.

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import

* ScottSuarez not on vacation anymore (#10052)

* I'm on vaction :)

* ScottSuarez not vaction anymore

* ScottSuarez not vaction anymore

* Promote (R)IGM.AllInstancesConfig to V1 (#9964)

* Promote metric settings in compute region autoscaler to GA (#10045)

* Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040)

* Retry delete network step while creating a google project. (#10046)

* Adding the namespace_labels field to the GKE Hub Scope resource (#9972)

* initial commit for scope-level namespace labels

* Add validation exceptions for the  field of the GKEHub Scope and Namespace resources to be of type

* Undoing unnecessary changes

* Fixing the type of Scope namespace_labels field

* Support connectivity fields for Composer 3 (#9889)

* add composer_network_attachment

* indicate conflicting configs

* commas

* no need for bidirectional conflict definition (generates double errors)

* protect nit PrivateClusterConfig

* for optimizing error messages about conflicts

* add 2 step update for composer_network_attachment

* make composer_network_attachment available in beta only

* add two step update for network and subnetwork

* corrections in 2 phase update for network/subnetwork

* remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working)

* added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains

* add docs for composer_network_attachment

* add test for network attachment

* ignore non empty plan in network attachment test

* add networkAttachment update and conflicting fields tests

* add ComputedIf for network, change isComposer3

* minor corrections

* remove computedIf

* filter equivalent values of network/subnetwork in ForceNewIf

* simplify ResourceConditionFunc, add beta/ga version conditions

* typo

* more general comparison of network references

* use tpgresource.CompareSelfLinkRelativePaths instead of custom function

* modify isComposer3 to avoid merge conflicts later.

* removing this since documentation is handled in other PR and to avoid conflicts while merging.

* replace ExpectNonEmptyPlan with lifecycle.ignore_changes

* add testcase for changing network attachment to network and subnetwork

* add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork

* modify tests to use different network for attachment

* remove unused constant

* remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes)

* Fix resource name to have sweepable prefix (#9984)

* doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049)

* Promote Cloud Deploy Automation to GA (#10043)

* Add Cloud Deploy Automation

* Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384

* Add update test for the new automation resource

* fix lint errors

* fix lint errors

* fix errors in GA provider test

* add the missing test

* add a full test

* Attempt to fix the acceptance test errors

* fix a lint error

* mark labels as default_from_api:true

* fix advance rollout rule definition

* Mark service_account as 'ignore_read: true'

* Fix test errors

* Resolve review comments

* Format test file

* promote google_clouddeploy_automation to GA

* promote google_clouddeploy_automation to GA

* fix lint errors

* Add cloudquota service (#10063)

* TeamCity: Add validating GHAs that check Kotlin files listing services (#9999)

* services diff gha

* add needs

* use setup-go@v3

* add artifacts

* remove checkout

* update directory

* update common-copy.yaml

* update common-copy.yaml

* remove data print

* add pull-request check on teamcitydiff

* update paths in GHA

* add exit code in diff_check

* update exit output

* add exit code diff_check.go

* services_beta diff check support

* beta services support args

* add arg for services kt file

* services_ga.kt in gha

* move diff_check to tools folder

* remove go command

* provide path for artifacts download

* add cd in Build Provider

* update diff_check_beta.yml

* fix paths

* yml typo beta

* directory testing

* directory testing

* fix directory issues with commands

* refactor tools/teamcity-diff-check/main.go

* directory check

* typo

* remove common copy

* generate both providers into one gha

* type artifacts name

* add -o flag into artifacts download

* use merge multiple artifacts

* use artifact@v4

* use mmv1 directory for services file

* remove .zip

* output missing services from diff

* find artifacts folder

* use beta provider as only artifact

* include google ga and beta in gha

* artifact name

* -o flag in artifacts

* output stdout from go list command

* diff test

* output beta main.go

* remove err check

* remove services print / refactor for final review

* output cleanup

* uncomment go list err

* regex assert

* add check for new services in PR / set googleServices as groundtruth in serviceDifference function

* add actions/checkout

* logic fix

* add needs: check-pr

* add mmv1/products for pull-request event

* set force depth 0

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* use text file for services, add if statement in gha

* directory typo

* typo

* typo in grep

* quotations on if statement

* unquote GITHUB_OUTPUT

* add quotes on output.services

* if test

* echo output

* github_output error fix

* services.outputs

* proper if statement syntax gpush

* invalid 0 format

* add missing assignment

* fix openfile error

* remove types in pull_request trigger

* send txt files to correct directory

* check services directory for ls command

* add weekly workflow for teamcity diff check

* change name for weekly check

---------

Co-authored-by: Sarah French <[email protected]>

* Fixed `runtime` in  app engine tests (#10047)

* Fix google_notebooks_instance labels not being able to be updated (#9933)

* add labels to runtime update test

* Add labels to ImportStateVerifyIgnore

* fix indentation

* Add IAM resources for Cloud Deploy Custom Target Type (#10051)

* Add IAM resources for Cloud Deploy Custom Target Type

* Add base url override for Cloud Deploy Custom Target Type IAM policy

* Update import format for IAM policy and primary resource name to examples for IAM generated tests

* Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075)

* Remove unneeded GHA steps - do we need to cache here?

* Whitespace

* Test change with fake new service

* Resolve build error in fake service

* Re-order GHA steps and add name

* Add name to checkout step

* Add comments, remove extra whitespace

* Remove fake new service

* Add output to identifiers, check for project as well as projectsId (#10081)

* Add output to identifiers, check for project as well as projectsId

* Correct identifier behavior

* Rubocop

* remove nickelliot/zli82016 from vacation reviewers (#10080)

* Update membership.go

* Update membership.go

* Update enrolled_teams.yml (#10082)

* Fix resource_bigquery_dataset ID validation (#10027)

* fix resource_bigquery_dataset ID validation

* skip the new validation tests for VCR

* update dataset IDs to be valid in terraform-google-conversion example files

* update more dataset IDs in the CAI files

* Prevent duplicate entries in vcr results (#10068)

* Prevent duplicate entries in vcr results

* Create empty map

* Only collect result from current test

* Split github tokens (#9988)

* Split github tokens

* Update .ci/gcb-generate-diffs-new.yml

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Remove redundant downstreams token

* Make diff processor use new token

* Update path to markdown file

* Replace GITHUB_TOKEN

* Make github tokens optional for generate downstream

* Allow either github token to be used

* Replace GITHUB_TOKEN

* Move environment variable lookup out of constructor

* Update .ci/magician/vcr/tester.go

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Add downstream token

* Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC

* Apply suggestions from code review

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

---------

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Instance lifecycle policy default action on failure implementation (#10050)

* Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076)

* Add wait in acc test to avoid 'API has not been used in project' error

* Add external provider `time`

* Fix defect when referencing google_project resource

* Set GITHUB_TOKEN environment variable for hub (#10091)

* fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)

* make go-converted YAML compatible with go compiler (#10033)

Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>

* Revert "Promote metric settings in compute region autoscaler to GA" (#10095)

This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d.

* Add type field to DNS authorization reosurce (#10030)

* Add type field to DNS authorization reosurce

* Add an example for regional DNS authorization

* Add an example for regional certs using regional DNS auth

* Fix lint errors

* Fix typo in the enum values

* Add type field in regional dns auth example

---------

Co-authored-by: Hamza Hassan <[email protected]>

* TeamCity: Refactor config tests (#9956)

* Add test util for locating a subproject 2 layers deep

* Refactor tests to use new helper function

* Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value

* Refactor new sweeper tests

* Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098)

* TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099)

* Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096)

* subnetwork and service_account_email params described (17211) (#10102)

* subnetwork and service_account_email params described (17211)

* Doc adjustment

* Update provider name in various spots (#10092)

* Go compiler skeleton and early terraform.rb provider code (#10104)

* Remove provider-related caches to address cache issues (#10097)

We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out.

* Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093)

* Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111)

This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174.

* Add support for string --> object map for DCL resources (#10039)

* Add support for string --> object map for DCL resources

* Fix whitespace in template

* Add test, need updated DCL to work

* Add override for key name

* Both overrides

* Update docs, prepend key for maps

* Update bundle descriptions

* Comment update

* fix permadiff by reading empty docker_config field (#10113)

* Import package golang.org/x/exp/slices in MMv1 go compiler (#10108)

* fix(kms): certificate chain type to array of strings (#9582)

* Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110)

* Update all uses of `actions/cache` to v3

* Update all uses of `actions/checkout` to v4

* Update all uses of `actions/upload-artifact` to v3.1.0

* Update all uses of `actions/setup-go` to v4.0.0

* Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2

[v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20

The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2

* Address dependency issues in TestAccFirestoreField_* tests (#9957)

* Add additional wait in TestAccFirestoreField_* tests

* Boost wait in test to 6 minutes

* Add dependency between database and service to control delete order

* Update dependency to explicitly include project

* Make firestore fields be removed from state when they're 'deleted'

* Add `destroy_duration`

* Remove from state after log line that uses id value

* Update destory check to accept a 403 as valid

* Remove unneeded changes in PR

* Remove call to SetId

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895)

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource

* Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes

* Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058)

* Handwrite sweepers for networkConnectivity hubs and spokes (#10069)

* Fix wrong variable in override logic (#10125)

* Make missing test detector reader into a separate module (#10115)

* Use old github token if new tokens are not available (#10114)

* Use old github token if new tokens are not available

* Add lookup function for github token or fallback

* Add fall back

* Add fallback

* Add test for PATCH environment type (#10131)

* Add new resource for Application for Apphub (#10079)

* Add new resource for Application for Apphub

* Add new resource for Application for Apphub

* Enable Apphub API in test cases

* Fix precheck error

* Fix precheck error

* Resolve comments

* Fix lint error

* Fix examples

* Remove apphub from teamcity config

This causes PR build failures, I'll add it later

---------

Co-authored-by: Sam Levenick <[email protected]>

* Add apphub (#10133)

Adds AppHub to APIs to activate

* Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031)

* Support service-level min instances in Cloud Run v2 services. (#10083)

* feat: Add CMEK support for Firestore database in Beta provider (#10044)

* Modify database.yaml to add cmek related fields

* Add two examples for firestore CMEK databases for testing

* Resolve trailing space

* Update documentation for kmsKeyName field

* Resolve trailing space

* Make field immutable

* Update field documentation

* Update field description

* Allow vcenter_network to be set (#9946)

* Allow vcenter_network to be set

* set default from api

* Sort resources and set in product in go compiler (#10135)

* fix faili…
  • Loading branch information
Show file tree
Hide file tree
Showing 31 changed files with 2,200 additions and 81 deletions.
6 changes: 6 additions & 0 deletions mmv1/provider/terraform/common~compile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@
-%>
'<%= dir -%>/<%= fname.delete_suffix(".erb") -%>': 'third_party/terraform/framework_models/<%= fname -%>'
<% end -%>
<%
Dir["third_party/terraform/functions/*.go.erb"].each do |file_path|
fname = file_path.split('/')[-1]
-%>
'<%= dir -%>/functions/<%= fname.delete_suffix(".erb") -%>': 'third_party/terraform/functions/<%= fname -%>'
<% end -%>
<%
Dir["third_party/terraform/scripts/**/*.erb"].each do |file_path|
fname = file_path.delete_prefix("third_party/terraform/")
Expand Down
7 changes: 7 additions & 0 deletions mmv1/provider/terraform/common~copy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,13 @@
'<%= dir -%>/envvar/<%= fname -%>': 'third_party/terraform/envvar/<%= fname -%>'
<% end -%>

<%
Dir["third_party/terraform/functions/*.go"].each do |file_path|
fname = file_path.split('/')[-1]
-%>
'<%= dir -%>/functions/<%= fname -%>': 'third_party/terraform/functions/<%= fname -%>'
<% end -%>

<%
Dir["third_party/terraform/scripts/**/*.*"].each do |file_path|
next if file_path.end_with?('.erb')
Expand Down
35 changes: 35 additions & 0 deletions mmv1/third_party/terraform/functions/element_from_id.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package functions

import (
"context"
"fmt"
"log"
"regexp"

"github.com/hashicorp/terraform-plugin-framework/function"
)

// ValidateElementFromIdArguments is reusable validation logic used in provider-defined functions that use the GetElementFromId function
func ValidateElementFromIdArguments(ctx context.Context, input string, regex *regexp.Regexp, pattern string, functionName string) *function.FuncError {
submatches := regex.FindAllStringSubmatchIndex(input, -1)

// Zero matches means unusable input; error returned
if len(submatches) == 0 {
return function.NewArgumentFuncError(0, fmt.Sprintf("The input string \"%s\" doesn't contain the expected pattern \"%s\".", input, pattern))
}

// >1 matches means input usable but not ideal; debug log
if len(submatches) > 1 {
log.Printf("[DEBUG] Provider-defined function %s was called with input string: %s. This contains more than one match for the pattern %s. Terraform will use the first found match.", functionName, input, pattern)
}

return nil
}

// GetElementFromId is reusable logic that is used in multiple provider-defined functions for pulling elements out of self links and ids of resources and data sources
func GetElementFromId(input string, regex *regexp.Regexp, template string) string {
submatches := regex.FindAllStringSubmatchIndex(input, -1)
submatch := submatches[0] // Take the only / left-most submatch
dst := []byte{}
return string(regex.ExpandString(dst, template, input, submatch))
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
package functions_test

import (
"context"
"regexp"
"testing"

tpg_functions "github.com/hashicorp/terraform-provider-google/google/functions"
)

func TestFunctionInternals_ValidateElementFromIdArguments(t *testing.T) {

// Values here are matched to test case values below
regex := regexp.MustCompile("two/(?P<Element>[^/]+)/")
pattern := "two/{two}/"

cases := map[string]struct {
Input string
ExpectedElement string
ExpectError bool
}{
"it sets an error if no match is found": {
Input: "one/element-1/three/element-3",
ExpectError: true,
},
"it doesn't set an error if more than one match is found": {
Input: "two/element-2/two/element-2/two/element-2",
ExpectedElement: "element-2",
},
}

for tn, tc := range cases {
t.Run(tn, func(t *testing.T) {

// Arrange
ctx := context.Background()

// Act
err := tpg_functions.ValidateElementFromIdArguments(ctx, tc.Input, regex, pattern, "function-name-here") // last arg value is inconsequential for this test

// Assert
if err != nil && !tc.ExpectError {
t.Fatalf("Unexpected error(s) were set in response diags: %s", err.Text)
}
if err == nil && tc.ExpectError {
t.Fatal("Expected error(s) to be set in response diags, but there were none.")
}
})
}
}

func TestFunctionInternals_GetElementFromId(t *testing.T) {

// Values here are matched to test case values below
regex := regexp.MustCompile("two/(?P<Element>[^/]+)/")
template := "$Element"

cases := map[string]struct {
Input string
ExpectedElement string
}{
"it can pull out a value from a string using a regex with a submatch": {
Input: "one/element-1/two/element-2/three/element-3",
ExpectedElement: "element-2",
},
"it will pull out the first value from a string with more than one submatch": {
Input: "one/element-1/two/element-2/two/not-this-one/three/element-3",
ExpectedElement: "element-2",
},
}

for tn, tc := range cases {
t.Run(tn, func(t *testing.T) {

// Act
result := tpg_functions.GetElementFromId(tc.Input, regex, template)

// Assert
if result != tc.ExpectedElement {
t.Fatalf("Expected function logic to retrieve %s from input %s, got %s", tc.ExpectedElement, tc.Input, result)
}
})
}
}
62 changes: 62 additions & 0 deletions mmv1/third_party/terraform/functions/location_from_id.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
package functions

import (
"context"
"regexp"

"github.com/hashicorp/terraform-plugin-framework/function"
)

var _ function.Function = LocationFromIdFunction{}

func NewLocationFromIdFunction() function.Function {
return &LocationFromIdFunction{
name: "location_from_id",
}
}

type LocationFromIdFunction struct {
name string // Makes function name available in Run logic for logging purposes
}

func (f LocationFromIdFunction) Metadata(ctx context.Context, req function.MetadataRequest, resp *function.MetadataResponse) {
resp.Name = f.name
}

func (f LocationFromIdFunction) Definition(ctx context.Context, req function.DefinitionRequest, resp *function.DefinitionResponse) {
resp.Definition = function.Definition{
Summary: "Returns the location name within a provided resource id, self link, or OP style resource name.",
Description: "Takes a single string argument, which should be a resource id, self link, or OP style resource name. This function will either return the location name from the input string or raise an error due to no location being present in the string. The function uses the presence of \"locations/{{location}}/\" in the input string to identify the location name, e.g. when the function is passed the id \"projects/my-project/locations/us-central1/services/my-service\" as an argument it will return \"us-central1\".",
Parameters: []function.Parameter{
function.StringParameter{
Name: "id",
Description: "A string of a resource's id, a resource's self link, or an OP style resource name. For example, \"projects/my-project/locations/us-central1/services/my-service\" and \"https://run.googleapis.com/v2/projects/my-project/locations/us-central1/services/my-service\" are valid values containing locations",
},
},
Return: function.StringReturn{},
}
}

func (f LocationFromIdFunction) Run(ctx context.Context, req function.RunRequest, resp *function.RunResponse) {
// Load arguments from function call
var arg0 string
resp.Error = function.ConcatFuncErrors(req.Arguments.GetArgument(ctx, 0, &arg0))
if resp.Error != nil {
return
}

// Prepare how we'll identify location name from input string
regex := regexp.MustCompile("locations/(?P<LocationName>[^/]+)/") // Should match the pattern below
template := "$LocationName" // Should match the submatch identifier in the regex
pattern := "locations/{location}/" // Human-readable pseudo-regex pattern used in errors and warnings

// Validate input
resp.Error = function.ConcatFuncErrors(ValidateElementFromIdArguments(ctx, arg0, regex, pattern, f.name))
if resp.Error != nil {
return
}

// Get and return element from input string
location := GetElementFromId(arg0, regex, template)
resp.Error = function.ConcatFuncErrors(resp.Result.Set(ctx, location))
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
package functions

import (
"context"
"fmt"
"testing"

"github.com/google/go-cmp/cmp"
"github.com/hashicorp/terraform-plugin-framework/attr"
"github.com/hashicorp/terraform-plugin-framework/function"
"github.com/hashicorp/terraform-plugin-framework/types"
"github.com/hashicorp/terraform-plugin-framework/types/basetypes"
)

func TestFunctionRun_location_from_id(t *testing.T) {
t.Parallel()

location := "us-central1"

// Happy path inputs
validId := fmt.Sprintf("projects/my-project/locations/%s/services/my-service", location)
validSelfLink := fmt.Sprintf("https://run.googleapis.com/v2/%s", validId)
validOpStyleResourceName := fmt.Sprintf("//run.googleapis.com/v2/%s", validId)

// Unhappy path inputs
repetitiveInput := fmt.Sprintf("https://run.googleapis.com/v2/projects/my-project/locations/%s/locations/not-this-one/services/my-service", location) // Multiple /locations/{{location}}/
invalidInput := "zones/us-central1-c/instances/my-instance"

testCases := map[string]struct {
request function.RunRequest
expected function.RunResponse
}{
"it returns the expected output value when given a valid resource id input": {
request: function.RunRequest{
Arguments: function.NewArgumentsData([]attr.Value{types.StringValue(validId)}),
},
expected: function.RunResponse{
Result: function.NewResultData(types.StringValue(location)),
},
},
"it returns the expected output value when given a valid resource self_link input": {
request: function.RunRequest{
Arguments: function.NewArgumentsData([]attr.Value{types.StringValue(validSelfLink)}),
},
expected: function.RunResponse{
Result: function.NewResultData(types.StringValue(location)),
},
},
"it returns the expected output value when given a valid OP style resource name input": {
request: function.RunRequest{
Arguments: function.NewArgumentsData([]attr.Value{types.StringValue(validOpStyleResourceName)}),
},
expected: function.RunResponse{
Result: function.NewResultData(types.StringValue(location)),
},
},
"it returns the first submatch (with no error) when given repetitive input": {
request: function.RunRequest{
Arguments: function.NewArgumentsData([]attr.Value{types.StringValue(repetitiveInput)}),
},
expected: function.RunResponse{
Result: function.NewResultData(types.StringValue(location)),
},
},
"it returns an error when given input with no submatches": {
request: function.RunRequest{
Arguments: function.NewArgumentsData([]attr.Value{types.StringValue(invalidInput)}),
},
expected: function.RunResponse{
Result: function.NewResultData(types.StringNull()),
Error: function.NewArgumentFuncError(0, fmt.Sprintf("The input string \"%s\" doesn't contain the expected pattern \"locations/{location}/\".", invalidInput)),
},
},
}

for name, testCase := range testCases {
tn, tc := name, testCase

t.Run(tn, func(t *testing.T) {
t.Parallel()

// Arrange
got := function.RunResponse{
Result: function.NewResultData(basetypes.StringValue{}),
}

// Act
NewLocationFromIdFunction().Run(context.Background(), tc.request, &got)

// Assert
if diff := cmp.Diff(got.Result, tc.expected.Result); diff != "" {
t.Errorf("unexpected diff between expected and received result: %s", diff)
}
if diff := cmp.Diff(got.Error, tc.expected.Error); diff != "" {
t.Errorf("unexpected diff between expected and received errors: %s", diff)
}
})
}
}
75 changes: 75 additions & 0 deletions mmv1/third_party/terraform/functions/location_from_id_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
package functions_test

import (
"fmt"
"regexp"
"testing"

"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
"github.com/hashicorp/terraform-provider-google/google/acctest"
)

func TestAccProviderFunction_location_from_id(t *testing.T) {
t.Parallel()
// Skipping due to requiring TF 1.8.0 in VCR systems : https://github.com/hashicorp/terraform-provider-google/issues/17451
acctest.SkipIfVcr(t)

location := "us-central1"
locationRegex := regexp.MustCompile(fmt.Sprintf("^%s$", location))

context := map[string]interface{}{
"function_name": "location_from_id",
"output_name": "location",
"resource_name": fmt.Sprintf("tf-test-location-id-func-%s", acctest.RandString(t, 10)),
"resource_location": location,
}

acctest.VcrTest(t, resource.TestCase{
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories(t),
Steps: []resource.TestStep{
{
// Can get the location from a resource's id in one step
// Uses google_cloud_run_service resource's id attribute with format projects/{project}/locations/{location}/services/{service}.
Config: testProviderFunction_get_location_from_resource_id(context),
Check: resource.ComposeTestCheckFunc(
resource.TestMatchOutput(context["output_name"].(string), locationRegex),
),
},
},
})
}

func testProviderFunction_get_location_from_resource_id(context map[string]interface{}) string {
return acctest.Nprintf(`
# terraform block required for provider function to be found
terraform {
required_providers {
google = {
source = "hashicorp/google"
}
}
}
resource "google_cloud_run_service" "default" {
name = "%{resource_name}"
location = "%{resource_location}"
template {
spec {
containers {
image = "us-docker.pkg.dev/cloudrun/container/hello"
}
}
}
traffic {
percent = 100
latest_revision = true
}
}
output "%{output_name}" {
value = provider::google::%{function_name}(google_cloud_run_service.default.id)
}
`, context)
}
Loading

0 comments on commit a424b54

Please sign in to comment.