Skip to content

Commit

Permalink
Sync main feature branch provider functions (#10273)
Browse files Browse the repository at this point in the history
* fixes: permadiff issue if event trigger region is not specified (#9989)

* fixes: permadiff issue if event trigger region is not specified

- fixes https://github.com/hashicorp/terraform-provider-google/issues/17161

* chore: Updated cloudfunction2 test with no explicit trigger_region

* Fix GKE front matter, bad field docs (#10018)

* Fix GKE front matter, bad field docs

* Update container_cluster.html.markdown

* added default value for minPortsPerVM field at "google_compute_router_nat" (#9712)

* added default value for statis allocation

* setting the default from API

* add newline removal bash command in guidelines (#9734)

* Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009)

* Adding Data Source Forwarding Rules (#10004)

* Fix broken terraform datasource google_compute_machine_types example (#10020)

* Add check for Environment proto field before accessing in Dataflow provider (#10016)

* Add comments to skipped sweeper (#10023)

* Add comments for skipped sweepers (#10024)

* C3 version schema (#9986)

* add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress

* make build number optional

* regroup regex, cleaner comparison of versions

* correction

* Add Resource Manager Tags support to 'google_container_cluster' (#9531)

* resourceManagerTags added to Cluster Node Config schema

* update beta tag

* add cluster and node proto tests

* add expand and flatten proto

* removed beta tag

* added to documentation

* added resource manager tags to auto pilot

* migrating resourceManagerTags tests

* migrating node_pools test

* migrating additional tests

* minor fixes

* fixing tests

* add in-place update support

* fixed tests

* fixed annotations

* validated clusters and node pools tests. Isolated node pool auto config

* isolated resource manager tags from docs

* fixed permission issue

* fixed spaces

* fixed non determinism on tag keys

* removed auto_pilot rmts

* fixed time_sleep

* add depends_on to IAM policies

* Add volume replication support for Google Cloud NetApp Volumes (#9816)

* Initial replication commit

* Cleanup work

- Renamed a lot of files to make clear which resource the belong to
- Updated documentation for resource fields
- Renamed a few resource fields and changed some types
- Disabled the custom code for now. Needs to be discussed first

* Update example file

* Updated example file

* Major updates

- Reorganisation of block
- Reorganisation of fields to match API documentation
- Updated example parameters
- Added missing API fields
- Improved descriptions
-

* For replication deletion, stop replication first

* Add support for deleting destination volume on replication delete

* Make volumes deletable in presence of snapshots.

This change will be PRed for volume resource independently. Adding it here while it is not in main.

* Improving debug error message

* yaml check and format fix

* Add wait for mirror to initialize.

Required to run destroy shortly after create.

* Wait on destroy, not on create

* Make deleting a replication more robust

- doc improvements
- started to implement stop/resume. More work required.
- renamed a few files to better reflect what they are good for

* adding support for stop/resume

* yamlformat and lint

* Add force delete to delete volumes with nested snapshots

* resource test first version

* More changes to make tests solid

- Introduced new parameter to wait for mirror_status==MIRRORED
- more mirror state reconciliation

* Test updates

* few cleanups

* Make virtual field verifies happy

* Minor test improvements

* More fine tuning

- Remove merge conflict in volume.yaml
- make generated test work
- make output field work
- ignore_read for virtual fields

* Resource name change as suggested by @slevenick

* Remove snapshot code block and fix typo

* Detect manual stop/resume actions

* Remove ignore_read for deletion_policy

* - Made destinationVolumeParameters immutable. It still requires ignore_read.
- removed ignore_read from virtual_fields

* destinationVolumeParameters are only evaluated at create. Make the immutable.

* Name cleanups and comment improvements

* removed comment

Co-authored-by: Shuya Ma <[email protected]>

* tabs to spaces in resource block

Co-authored-by: Shuya Ma <[email protected]>

* Updates to address review comments

- make wait_for_mirror also work for stop/resume, additionally to create
- convert tabs in test resource blocks to spaces
- fix typos

* Rewording of comments

Co-authored-by: Shuya Ma <[email protected]>

---------

Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>

* Ensured that beta runs in TeamCity use only beta paths (#10025)

* Ensured that beta runs in TeamCity use only beta paths

* Added tests for sweeper package path

* Adds description for default value of `cpuidle` field (#10005)

* add description

* remove tail space

* Add location field in DNS authorization resource. (#9968)

* Add location field in DNS authorization resource

* Fixed the provider version of the updated config

---------

Co-authored-by: Hamza Hassan <[email protected]>

* Add discoveryengine service (#10017)

* Add validations for Composer 2/3 only fields (#9917)

* block upgrade to composer 3

* make isComposer3 more generic, correct imageVersionChangeValidationFunc

* added validation for Composer 2/3 specific fields

* add tests for validation

* add checks in flattenComposerEnvironmentConfig

* Update attributes of fields not used in Composer 3

* make customizeDiff functions beta only

* remove Computed from gke_cluster

* remove Optional instead of Computed

* add envCfg.PrivateEnvironmentConfig is nil check

* modify isComposer3 to take string

* minor correction to avoid merge conflicts

* Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034)

Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319

* Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011)

Co-authored-by: Hamza Hassan <[email protected]>

* Added missing quote marks to TEST_PREFIX (#10035)

Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied

* Promote enable_confidential_storage from beta to GA (#9993)

* Add IAM resources for Cloud Deploy Target (#9927)

* Add IAM resources for Cloud Deploy Target

* update

* add back import_format

* Add comments to skipped sweepers (#10028)

* Add sweepers for gkebackupPlan and restorePlan (#10026)

* add `remote_function_options` to bigquery_routine (#9893)

* Add dataGovernanceType and remoteFunctionOptions to bigquery_routine

* add function-sources.zip to biguquery fixtures

* fix resource names in TestAccBigQueryRoutine

* add bigquery routine remote function example

* Adding documentaiton build_worker_pool (#10022)

* docs: fix import example in google_project_iam (#10007)

Fix the example in the `google_project_iam` documentation.

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import

* ScottSuarez not on vacation anymore (#10052)

* I'm on vaction :)

* ScottSuarez not vaction anymore

* ScottSuarez not vaction anymore

* Promote (R)IGM.AllInstancesConfig to V1 (#9964)

* Promote metric settings in compute region autoscaler to GA (#10045)

* Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040)

* Retry delete network step while creating a google project. (#10046)

* Adding the namespace_labels field to the GKE Hub Scope resource (#9972)

* initial commit for scope-level namespace labels

* Add validation exceptions for the  field of the GKEHub Scope and Namespace resources to be of type

* Undoing unnecessary changes

* Fixing the type of Scope namespace_labels field

* Support connectivity fields for Composer 3 (#9889)

* add composer_network_attachment

* indicate conflicting configs

* commas

* no need for bidirectional conflict definition (generates double errors)

* protect nit PrivateClusterConfig

* for optimizing error messages about conflicts

* add 2 step update for composer_network_attachment

* make composer_network_attachment available in beta only

* add two step update for network and subnetwork

* corrections in 2 phase update for network/subnetwork

* remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working)

* added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains

* add docs for composer_network_attachment

* add test for network attachment

* ignore non empty plan in network attachment test

* add networkAttachment update and conflicting fields tests

* add ComputedIf for network, change isComposer3

* minor corrections

* remove computedIf

* filter equivalent values of network/subnetwork in ForceNewIf

* simplify ResourceConditionFunc, add beta/ga version conditions

* typo

* more general comparison of network references

* use tpgresource.CompareSelfLinkRelativePaths instead of custom function

* modify isComposer3 to avoid merge conflicts later.

* removing this since documentation is handled in other PR and to avoid conflicts while merging.

* replace ExpectNonEmptyPlan with lifecycle.ignore_changes

* add testcase for changing network attachment to network and subnetwork

* add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork

* modify tests to use different network for attachment

* remove unused constant

* remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes)

* Fix resource name to have sweepable prefix (#9984)

* doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049)

* Promote Cloud Deploy Automation to GA (#10043)

* Add Cloud Deploy Automation

* Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384

* Add update test for the new automation resource

* fix lint errors

* fix lint errors

* fix errors in GA provider test

* add the missing test

* add a full test

* Attempt to fix the acceptance test errors

* fix a lint error

* mark labels as default_from_api:true

* fix advance rollout rule definition

* Mark service_account as 'ignore_read: true'

* Fix test errors

* Resolve review comments

* Format test file

* promote google_clouddeploy_automation to GA

* promote google_clouddeploy_automation to GA

* fix lint errors

* Add cloudquota service (#10063)

* TeamCity: Add validating GHAs that check Kotlin files listing services (#9999)

* services diff gha

* add needs

* use setup-go@v3

* add artifacts

* remove checkout

* update directory

* update common-copy.yaml

* update common-copy.yaml

* remove data print

* add pull-request check on teamcitydiff

* update paths in GHA

* add exit code in diff_check

* update exit output

* add exit code diff_check.go

* services_beta diff check support

* beta services support args

* add arg for services kt file

* services_ga.kt in gha

* move diff_check to tools folder

* remove go command

* provide path for artifacts download

* add cd in Build Provider

* update diff_check_beta.yml

* fix paths

* yml typo beta

* directory testing

* directory testing

* fix directory issues with commands

* refactor tools/teamcity-diff-check/main.go

* directory check

* typo

* remove common copy

* generate both providers into one gha

* type artifacts name

* add -o flag into artifacts download

* use merge multiple artifacts

* use artifact@v4

* use mmv1 directory for services file

* remove .zip

* output missing services from diff

* find artifacts folder

* use beta provider as only artifact

* include google ga and beta in gha

* artifact name

* -o flag in artifacts

* output stdout from go list command

* diff test

* output beta main.go

* remove err check

* remove services print / refactor for final review

* output cleanup

* uncomment go list err

* regex assert

* add check for new services in PR / set googleServices as groundtruth in serviceDifference function

* add actions/checkout

* logic fix

* add needs: check-pr

* add mmv1/products for pull-request event

* set force depth 0

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* use text file for services, add if statement in gha

* directory typo

* typo

* typo in grep

* quotations on if statement

* unquote GITHUB_OUTPUT

* add quotes on output.services

* if test

* echo output

* github_output error fix

* services.outputs

* proper if statement syntax gpush

* invalid 0 format

* add missing assignment

* fix openfile error

* remove types in pull_request trigger

* send txt files to correct directory

* check services directory for ls command

* add weekly workflow for teamcity diff check

* change name for weekly check

---------

Co-authored-by: Sarah French <[email protected]>

* Fixed `runtime` in  app engine tests (#10047)

* Fix google_notebooks_instance labels not being able to be updated (#9933)

* add labels to runtime update test

* Add labels to ImportStateVerifyIgnore

* fix indentation

* Add IAM resources for Cloud Deploy Custom Target Type (#10051)

* Add IAM resources for Cloud Deploy Custom Target Type

* Add base url override for Cloud Deploy Custom Target Type IAM policy

* Update import format for IAM policy and primary resource name to examples for IAM generated tests

* Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075)

* Remove unneeded GHA steps - do we need to cache here?

* Whitespace

* Test change with fake new service

* Resolve build error in fake service

* Re-order GHA steps and add name

* Add name to checkout step

* Add comments, remove extra whitespace

* Remove fake new service

* Add output to identifiers, check for project as well as projectsId (#10081)

* Add output to identifiers, check for project as well as projectsId

* Correct identifier behavior

* Rubocop

* remove nickelliot/zli82016 from vacation reviewers (#10080)

* Update membership.go

* Update membership.go

* Update enrolled_teams.yml (#10082)

* Fix resource_bigquery_dataset ID validation (#10027)

* fix resource_bigquery_dataset ID validation

* skip the new validation tests for VCR

* update dataset IDs to be valid in terraform-google-conversion example files

* update more dataset IDs in the CAI files

* Prevent duplicate entries in vcr results (#10068)

* Prevent duplicate entries in vcr results

* Create empty map

* Only collect result from current test

* Split github tokens (#9988)

* Split github tokens

* Update .ci/gcb-generate-diffs-new.yml

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Remove redundant downstreams token

* Make diff processor use new token

* Update path to markdown file

* Replace GITHUB_TOKEN

* Make github tokens optional for generate downstream

* Allow either github token to be used

* Replace GITHUB_TOKEN

* Move environment variable lookup out of constructor

* Update .ci/magician/vcr/tester.go

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Add downstream token

* Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC

* Apply suggestions from code review

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

---------

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Instance lifecycle policy default action on failure implementation (#10050)

* Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076)

* Add wait in acc test to avoid 'API has not been used in project' error

* Add external provider `time`

* Fix defect when referencing google_project resource

* Set GITHUB_TOKEN environment variable for hub (#10091)

* fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)

* make go-converted YAML compatible with go compiler (#10033)

Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>

* Revert "Promote metric settings in compute region autoscaler to GA" (#10095)

This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d.

* Add type field to DNS authorization reosurce (#10030)

* Add type field to DNS authorization reosurce

* Add an example for regional DNS authorization

* Add an example for regional certs using regional DNS auth

* Fix lint errors

* Fix typo in the enum values

* Add type field in regional dns auth example

---------

Co-authored-by: Hamza Hassan <[email protected]>

* TeamCity: Refactor config tests (#9956)

* Add test util for locating a subproject 2 layers deep

* Refactor tests to use new helper function

* Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value

* Refactor new sweeper tests

* Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098)

* TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099)

* Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096)

* subnetwork and service_account_email params described (17211) (#10102)

* subnetwork and service_account_email params described (17211)

* Doc adjustment

* Update provider name in various spots (#10092)

* Go compiler skeleton and early terraform.rb provider code (#10104)

* Remove provider-related caches to address cache issues (#10097)

We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out.

* Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093)

* Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111)

This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174.

* Add support for string --> object map for DCL resources (#10039)

* Add support for string --> object map for DCL resources

* Fix whitespace in template

* Add test, need updated DCL to work

* Add override for key name

* Both overrides

* Update docs, prepend key for maps

* Update bundle descriptions

* Comment update

* fix permadiff by reading empty docker_config field (#10113)

* Import package golang.org/x/exp/slices in MMv1 go compiler (#10108)

* fix(kms): certificate chain type to array of strings (#9582)

* Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110)

* Update all uses of `actions/cache` to v3

* Update all uses of `actions/checkout` to v4

* Update all uses of `actions/upload-artifact` to v3.1.0

* Update all uses of `actions/setup-go` to v4.0.0

* Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2

[v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20

The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2

* Address dependency issues in TestAccFirestoreField_* tests (#9957)

* Add additional wait in TestAccFirestoreField_* tests

* Boost wait in test to 6 minutes

* Add dependency between database and service to control delete order

* Update dependency to explicitly include project

* Make firestore fields be removed from state when they're 'deleted'

* Add `destroy_duration`

* Remove from state after log line that uses id value

* Update destory check to accept a 403 as valid

* Remove unneeded changes in PR

* Remove call to SetId

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895)

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource

* Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes

* Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058)

* Handwrite sweepers for networkConnectivity hubs and spokes (#10069)

* Fix wrong variable in override logic (#10125)

* Make missing test detector reader into a separate module (#10115)

* Use old github token if new tokens are not available (#10114)

* Use old github token if new tokens are not available

* Add lookup function for github token or fallback

* Add fall back

* Add fallback

* Add test for PATCH environment type (#10131)

* Add new resource for Application for Apphub (#10079)

* Add new resource for Application for Apphub

* Add new resource for Application for Apphub

* Enable Apphub API in test cases

* Fix precheck error

* Fix precheck error

* Resolve comments

* Fix lint error

* Fix examples

* Remove apphub from teamcity config

This causes PR build failures, I'll add it later

---------

Co-authored-by: Sam Levenick <[email protected]>

* Add apphub (#10133)

Adds AppHub to APIs to activate

* Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031)

* Support service-level min instances in Cloud Run v2 services. (#10083)

* feat: Add CMEK support for Firestore database in Beta provider (#10044)

* Modify database.yaml to add cmek related fields

* Add two examples for firestore CMEK databases for testing

* Resolve trailing space

* Update documentation for kmsKeyName field

* Resolve trailing space

* Make field immutable

* Update field documentation

* Update field description

* Allow vcenter_network to be set (#9946)

* Allow vcenter_network to be set

* set default from api

* Sort resources and set in product in go compiler (#10135)

* fix failing posture test (#10086)

* fix failing posture test

* modify posture name

---------

Co-authored-by: Sneha Prasad <[email protected]>

* Create support for KMS Ekmconnection resource (#10094)

* Fixes two lifecycle rules with different no_age value always generates change. (#10137)

* TeamCity: Add project for testing the provider functions feature branch (#10088)

* Add ability to use non-default versions of Terraform in TeamCity builds

* Add function to enable making build configs for single packages at a time

* Add new sub project that contains 2 builds for testing provider functions

the 2 builds:
1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo
2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo

These builds both use an alpha release of TF 1.8.0

* Add builds for testing auto generated branches in the MM upstream repos

These re-use existing VCR Roots.

* Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time

* Fix defect in 'Download Terraform' build step definition

* Update build step to solve bug

* Update build_configuration_per_package.kt

* Add `ephemeral_directories` to google_workstations_workstation_config (#10042)

* Add `ephemeral_directories` to google_workstations_workstation_config

* Remove trailing spaces

* Add a test for ephemeral_directories

* Fix test

* Add a test for source_image

* Fix typo in test

* Remove unnecessary immutable field

* App Check DeviceCheck provider (#9978)

* App Check DeviceCheck provider

* Remove minimal example & pattern field

* Add real private keys that are not useful anywhere

* Limit tests to beta

* Documentation (#10012)

* duplicate Composer 2 argument reference

* remove fields that are not supported in Composer 3

* move fields that are new in Composer 3 to Composer 3 section only

* make suggested changes, add description if new versioning schema

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* add notice that composer 3 is not yet released.

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* specify composer version in section links.

* specify composer version in composer 1 documentation links

* add section links in composer 2 argument reference

* add section links in composer 3 argument reference

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

---------

Co-authored-by: Pavel Salnikov <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Promote metric settings in compute region autoscaler to GA (#10103)

* Clarified permissions for a few workflows (#10124)

* Clarified permissions for a few workflows

* fixed conditions for running repository-documentation workflows

* Bumped actions/checkout to v4

* force downstream generation

* Revert "force downstream generation"

This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a.

* Post statuses earlier (#10128)

* Post initial statuses earlier so that we are resilient issues during cloning

* force generation

* Revert "force generation"

This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140.

* Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (#10140)

* Simplified / clarified approval flow for presubmits (#10142)

* Simplified / clarified approval flow for presubmits

* Force generation

* Removed unused command args and cleaned up documentation for membership-checker

* Revert "Force generation"

This reverts commit db32066e198daf930006f72bae43602f22d8e3d9.

* Clean up delete template (#10144)

* added missing web documentation about create_timestamp for compute_(region_)instance_group_manager (#10148)

* Move pre_delete code closer to the request (#10143)

* Create resource definition for IAP Tunnel DestGroup (#10064)

* Add new resource for Service Project for Apphub (#10048)

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* [#15779] Add google_network_security_firewall_endpoint_association resource (#9815)

* [#15779] Add google_network_security_firewall_endpoint_association resource

* Fixing tests

* Fixes

* Add test for update

---------

Co-authored-by: Luca Prete <[email protected]>

* update teamcity diff check to run every monday morning also (#10159)

* Add DisplayName to Product struct (#10161)

* Add data source for apphub discovered service (#10105)

* Add data source for apphub discovered service

* Add data source for apphub discovered service'

* Add data source for apphub discovered service

* resolved comments

* retry logic added

* add tests and documentation

* Corrected tests and added project field in the data source

* changed id field

* Added random_suffix for the resources created and enabled iam policy and compute api

* modified retry logic

* Modified schema for the data source, and tests

* Removed IAM permission blocks

* Add dependency for compute api

* Resolve merge conflict

* Add time sleep

* Modified test function name

* Shorten service project name

* Add billing account

* corrected get env variable function call

* Modified project id

* Combined time delay

* Resolving comments

* Remove retry logic and add time sleep for resource ingestion

* Graduate support of GKE Queued Provisioning to GA (#10053)

* Update index page of documentation to link to issue templates for Bugs and FRs (#10160)

* Update index page of documentation to link to issue templates for Bugs and FRs

* Remove unused param from URL

* Remove other unused parameter from URL

* Add data source for AppHub discovered workload (#10107)

* Add data source for Apphub discovered workload

* Add data source for Apphub discovered workload

* Add data source for Apphub Discovered Workload

* Resolved comments

* Resolved comments

* Resolved comments

* Resolved comments

* Add tests and documentation for data source discovered workload

* Resolved comments and added tests

* Resolved comments

* Added modifications in tests, data source and documentation

* Added modifications in tests, data source and documentation

* Added modifications in tests, data source and documentation

* Verified tests

* Tests Verification

* Tests Verification

* Updated logic to obtain workload_uri

* Updated logic to obtain workload_uri

* Resolved comments

* Resolved comments

* Add billing account and shorten service project name

* Add header

* Change the project name to start with tf-test

* Resolved comments

* Lint changes

* Removing two sleeps

* Removing two sleeps

* Modifying documentation

* Resolved tests

* Resolved comments

* Resolved comments

* logic modification

* Update Retry logic

* Update Retry logic

* Resolved tests

* Adding workload_uri description

* Adding workload_uri description

* Update Documentation for ACM Service Perimeter resources to reflect Granular Controls group support (#10087)

groups

Co-authored-by: Charles Leon <[email protected]>

* fix tags cause notebooks instances to recreate (#10179)

* add labels to runtime update test

* Add labels to ImportStateVerifyIgnore

* fix indentation

* fix tags cause notebooks instances to create

* Documentation Fix compute_security_policy (#9732)

Fix mixed up descriptions between request_uri and request_query_param.

* Add new resource Workload for Apphub (#10155)

* Add new resource Workload for Apphub

* Fix lint error

* Add billing account

* Remove extra delay from test

* Resolve comments

* Add handwritten update test

* Fix lint error

* Add data source changes

* Remove merged changes

* Remove merged changes

* Batch all update tests into one step to make test fast

* Add a delay between discovered resource fetch and managed instance group manager creation

* Add back org_id

* Update region from us-east1 to us-central1

* Use standard diff suppress function

---------

Co-authored-by: praseedhaPK <[email protected]>
Co-authored-by: Krishnan Gopal <[email protected]>

* Fix for PrivateNat test failures (#10174)

* Add new resource Service for Apphub (#10132)

* Add new resource for Service Project for Apphub

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* Add new resource for Service Project for Apphub

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* Add new resource for Service Project Attachment for Apphub

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* Add new resource for Service Project Attachment for Apphub

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* Add new resource for Service Project Attachment for Apphub

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* Add new resource for Application for Apphub

* Add new resource for Application for Apphub

* Enable Apphub API in test cases

* Add new resource for Service Project Attachment for Apphub

FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405

* Enable apphub API in test cases

* Fix precheck error

* Fix precheck error

* Fix precheck error

* Fix precheck error

* Resolve comments

* Fix lint error

* Fix errors

* Add new resource for Service for Apphub

* Fix examples

* Add test

* Fix tests

* Add IAM permissions

* Add IAM permission

* Fix IAM issues

* Remove merged changes

* Add billing account

* Remove extra delay from tests

* Add handwritten update test

* Remove merged changes

* Make changes to yaml file incorporating comments from https://github.com/GoogleCloudPlatform/magic-modules/pull/10155

* Fix lint errors in Service YAML definition

* Add a delay between discovered resource fetch and forwarding rule creation

* Add a delay between discovered resource fetch and forwarding rule creation

* Add a delay between discovered resource fetch and forwarding rule creation

* Batch update tests to improve speed

* Use standard diff suppress function

* Update region from us-east1 to us-central1

---------

Co-authored-by: Krishnan Gopal <[email protected]>
Co-authored-by: praseedhaPK <[email protected]>

* feat: add credentials_secret field in azure blob storage block for google storage transfer job resource (#9278)

* feat: add `credentials_secret` in `azure_blob_storage_data` source config

* feat: add version guard for `credentials_secret` as in preview

* docs: add documentation on `credentials_secret`

* Added CloudQuotas service and Create QuotaInfo Datasource (#10071)

* add quota info data source

* add test and documention for quota_info data source

* fix lint error

* clean up

* update test

* manually import cloudquotas to terraform provider

* Update mmv1/third_party/terraform/provider/provider_mmv1_resources.go.erb

Co-authored-by: Riley Karson <[email protected]>

* Update mmv1/third_party/terraform/provider/provider_mmv1_resources.go.erb

Co-authored-by: Riley Karson <[email protected]>

---------

Co-authored-by: Riley Karson <[email protected]>

* Update comments in .teamcity/ files for clarity (#10149)

* Make warning about files being controlled by MM more explicit

* Add guidance to the input files about how to update them, and when

* Add fields pathTemplateMatch and pathTemplateRewrite to resource google_compute_region_url_map (#10157)

* Refactor TeamCity Services Diff Check to run as one job (#10158)

* Add explicity dependency on IAM in PubSub acc test config (#9921)

* Improve the error messages when mmv1 compilation fails (#10191)

* Update enrolled teams (#10192)

* Fix generate comment diff processor build failure (#10164)

* Refactored generate_comment to continue in the face of errors

Also switched to go template for comment formatting

* Moved to structured data for diffs

Also added error formatting into comment

* Added basic tests for comment formatting

* Skipped missing tests and breaking changes if repo failed to clone

* Force generation

* Breaking change

* Fixed error display in diff comment

* Marked diff-processor targets as not being real files

* Added PATH to diffProcessorEnv

* Fixed formatting for error sections

* Cleaned up passthrough env var usage

* Made exec.sh exit 1 if there are any errors running the magician

* Don't error on initial build of the magician binary

* Added missing  env var for diff processor build

* Force generation of tf-oics

* Revert "Force generation of tf-oics"

This reverts commit fcb65f121cbb0f7040a6b40c635eae3b318d936d.

* Revert "Force generation"

This reverts commit 596d1eb5afb10d36ab934411e0f1c47aa8b363e9.

* Revert "Breaking change"

This reverts commit fb04ad401e1d4ee93ef767297a2a59af7d2acffa.

* Force missing test run

* force missing tests to run - take 2

* Revert "force missing tests to run - take 2"

This reverts commit 34f7d16a8442f09aa4a52e0ceafb8dae73f89b08.

* Revert "Force missing test run"

This reverts commit 90e664a935c4645314c3aabf1f48836e4547fc68.

* Rewrite versions related functions (#10181)

* Update Dockerfiles to Go 1.21 (#10175)

* Add an override for generating long form tests (#10162)

* Update to go 1.21 (#10169)

* Update .go-version file in TPG and TPGB repos

* Update go.mod used to generate TPG/TPGB and run `go mod tidy`

* Update go.mod in mmv1/third_party/terraform/scripts

* Update go.mod in mmv1

* Update go.mod in docs

* Update go.mod in tools/go-changelog and run `go mod tidy`

* Update go.mod in tools/missing-test-detector and run `go mod tidy`

* Update go.mod in tools/diff-processor

* Update go.mod in tools/issue-labeler

* Update go.mod in tools/teamcity-generator

* Update go.mod in tpgtools and run `go mod tidy`

* Update doctor script to ensure developers use Go 1.21 locally

* Update GHA workflows to use Go 1.21

* Add missing update to GHA

* Add note about permissions (#10195)

* Fix the bug for computed labels and annotations (#10182)

* AndroidApp package_name should be immutable & required (#10194)

* Fix service team reviews (#10200)

* Corrected unknown command messge to clean up stderr

* Made tests throw error if team name is empty

* Fixed handling of unset github_team name

* Add gcs volume type to Cloud Run v2 beta provider (#9746)

* Add gcs volume type to Cloud Run v2 beta provider

* update gcs v2 docs to specify required launch stage and execution environment

* merge changes with upstream

* cloudrunv2: Validate number of ports specified restricting to 1. (#10173)

* cloudrunv2: Validate number of ports specified restricting to 1.

* Update mmv1/products/cloudrunv2/Service.yaml

* Bump google.golang.org/protobuf from 1.28.1 to 1.33.0 in /.ci/magician (#10196)

Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* attempt at fixing the (R)IGM.status output message (#10188)

* attempt at fixing the (R)IGM.status output message

* Fix plural of instances

* Fix plural of instances

* Fix plural of instances

* Fix plural of instances

* Access Context Manager - make ingress and egress rules immutable (#10147)

* Access Context Manager - make ingress and egress rules immutable

* Move immutable state to top level resource

---------

Co-authored-by: Charles Leon <[email protected]>

* Bump golang.org/x/net from 0.10.0 to 0.17.0 in /.ci/magician (#9243)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.10.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix datastream private connection sweeper (#10201)

* fix datastream private connection sweeper

* update

* run gofmt

* Add SarahFrench to on vacation list (#10189)

* Modify issue labeler to skip linked issues (#10199)

* Modify issue labeler to skip linked issues

* Update tests

* Remove misleading examples from docs for `google_dataform_repository`, add new example (#10187)

* Remove misleading examples from docs for `google_dataform_repository`, add new example

* Remove unusued var from new example

* Fix typo

* Fix other instances of typo

* Bump google.golang.org/grpc from 1.53.0 to 1.56.3 in /.ci/magician (#9366)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.53.0 to 1.56.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.53.0...v1.56.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Moving the netapp kmsconfig resource update test to a different region to avoid conflict with create test (#10185)

* Only run mmv1 rake tests and yaml linting if relevant files have changed (#10207)

* Only run mmv1 rake tests and yaml linting if relevant files have changed

* fix path for rb files

Co-authored-by: Sarah French <[email protected]>

---------

Co-authored-by: Sarah French <[email protected]>

* Clean up functions and add slice utility functions (#10210)

* Clean up function all_resourcerefs

* Add slice utility functions

* Change sensitive_props as resource method

* Removed github-differ bash scripts (#10208)

* Add ingestion settings to `google_pubsub_topic` (#9985)

* Add ingestion settings to `google_pubsub_topic`

* Add ingestion settings to `google_pubsub_topic`

* Add create/update tests for `google_pubsub_topic` ingestion settings

* Update topic ingestion settings test

* Run Firestore index tests in fresh databases. (#10168)

* Run Firestore index tests in fresh databases.

This removes the index dependency on creating a document, as creating indexes on a non-existent collection is a valid operation in Firestore.

This removes the dependency on a fixed (default) database, which should deflake tests.

Fixes #17393.

* Move project_id to test_env_vars.

* Move project_id to test_env_vars for datastore_mode example.

* AccessContextManager - Add dry run service perimeter resource (#10145)

Co-authored-by: Charlesleonius <[email protected]>

* Create quota infos datasource (#10074)

* add quota info data source

* add test and documention for quota_info data source

* fix lint error

* add quota infos datasource

* update read test

* remove duplicate import

* update formatting and documentation

* remove pagination parameters, concat and return all qualified entries

* remove specific check for number of entries

* fix(spanner): remove validation on version retention period on spanner database (#10184)

* fix(spanner): remove validation on version retention period on spanner database

* fix build

* initial mmv1 go templating and flags (#10219)

* Adds support for Soft Delete feature, which allows setting soft delete policy on 'google_storage_bucket' resource. (#10171)

* Fixes and enhancements after the bug bash. (#10223)

* remove float from documentation

* change storage-gb type to int

* Revert "change storage-gb type to int"

This reverts commit 89795a80889287fb287497fbcbbfdbb365d40884.

* add support for dag_processor count

* fix int64 unpacking

* add validation for composer internal ip range size

* add upper limit for dag_prcessor.count

* google_monitoring_alert_policy periods doc update & dataproc doc update (#10226)

* enable_http_port_access is a boolean not a string

* add doc update to alert policy notification rate limit period

* Add support for boost configs in workstations configs (#10176)

* add boostConfig workstationConfig

* add boostConfig workstationConfig

* link example

* remove

* update descriptions and add update test

* update machine type

* fix update test

* Add a rule for removing diff suppress functions (#10167)

* Add a rule for removing diff suppress functions

* Add rule link to docs

* Rewrite Resource functions (#10228)

* Add missing guest_os_features enumeration value (#10229)

* add billing project id support to firewall endpoint resource (#10122)

* add billing project support

* removed description field due to API issue

* test updated and fixed for ADC support

* added ADC warning

* removing ADC

* Upgraded version of actions related to scorecard (#10234)

* Don't re-request review if initial PR comment is edited (#10232)

* Pinned versions for GitHub Actions (#10239)

* Switched from diff-processor adding labels to just computing and returning them as JSON (#10211)

* Switched diff-processor from add-labels command to changed-schema-labels command

* Force update to redis cluster

* Revert "Force update to redis cluster"

This reverts commit 750c39ddf7f30d8a448fb775b86711ecbf3830f0.

* change the dataflow zone from us-central1-f to us-east5-b (#10205)

* Added `CloudQuotas` service and Create `QuotaPreference` Resource (#10019)

* setup resoruce config and add create tests

* add create and update test

* fix lint error

* fix lint error

* update yaml config

* remove trailing spaces

* modify update test

* update acceptance test and cleanup

* add upsert scenario for testing

* update acceptance test syntax

* remove allow-missing flag & fix update test failure

* add import and id format

* teamcity test failure - manually add cloudquotas service

* teamcity test failure - manually add cloudquotas service for beta

* add field 'validate_only' to tests

* address comments & fix test failure

* remove unused import

* remove `validate_only` field, add ignore read feature to `annotations` field

* add input only fields to ImportStateVerifyIgnore

* update field name

* VCR test failure: remove batch service creation

* fix formatting

* Magician tgc integration (#9907)

* Add command for tgc integration tests

* Use magician for TGC integration tests

* Fix variable name

* Remove owner

* Fix go mod tidy command

* Use new github token

* Fix https://github.com/hashicorp/terraform-provider-google/issues/17389 (#10206)

* generated_id added to compute_region_backend_service (#10243)

* Clarify CI README a bit (#10245)

* Add integrations product. Create Client resource. (#10186)

Co-authored-by: Cameron Thornton <[email protected]>

* Add apphub.admin role for the organization in the test environment (#10152)

* add apphub.admin role for the organizations that the CI runs the tests

* Change to use org instead of billing id

* Change to use org instead of billing id

---------

Co-authored-by: Krishnan Gopal <[email protected]>

* Change ItemType to object (#10249)

* Clarify usage of min/max throughput and instances (#10130)

* Clarify usage of min/max throughput and instances

* fix lint

* fix snake casing

* Fixed issues with diff-processor caused by GA-only compilation errors (#10250)

* Added file to trigger google-only failure

* Check if cleaning the diff processor up properly solves the issue

* Revert "Check if cleaning the diff processor up properly solves the issue"

This reverts commit 85c358e4191d5e6115bfef0e02c819f92b45dfe0.

* Clean up diff processor before building instead of after

* Revert "Added file to trigger google-only failure"

This reverts commit 0e4486080477902820ad282666d0c8f51e890076.

* Consolidate Documentation for App Hub resources and data sources into one subcategory (#10238)

* add apphub.admin role for the organizations that the CI runs the tests

* Change to use org instead of billing id

* Change to use org instead of billing id

* Fix subcategory of data sources consistent with rest of AppHub product resources

* This change is supposed to be a part of another feature

---------

Co-authored-by: Krishnan Gopal <[email protected]>

* Corrected names of iam_workload resource and data source files (#10253)

beta is not part of the resource name. This is important for https://github.com/GoogleCloudPlatform/magic-modules/pull/10248

* Pin workflow dependency versions (#10256)

* Pinned python yaml library versions

* Pinned goimports version in build-downstreams

* Force generation

* Revert "Force generation"

This reverts commit b2d876045277156ae2fd3a44194fe58164297186.

* Rewrite type functions (#10259)

* Removed double quotes and typo in google_project_iam_member documentation (#9750)

* Iniital setup for codeql scanning (#10254)

* Added permissions: read-all to codeql action (#10264)

* Remove java-kotlin codeql scanning (#10263)

* Initial resource documentation templating (#10266)

* Add note about modifying yaml files (#10202)

* bq table - add geojson support (#10215)

* bq table - add geojson support

* bq table - add geojson support

* CR comments

* Add logsBucket (#10267)

* Remove SarahFrench from vacation list (#10272)

* ci: add missing pathspec dependency for yamllint pipeline (#10269)

pipelines for YAML lints are failing due to:
```
Traceback (most recent call last):
  File "/home/runner/.local/bin/yamllint", line 5, in <module>
    from yamllint.cli import run
  File "/home/runner/.local/lib/python3.10/site-packages/yamllint/cli.py", line 24, in <module>
    from yamllint.config import YamlLintConfig, YamlLintConfigError
  File "/home/runner/.local/lib/python3.10/site-packages/yamllint/config.py", line 19, in <module>
    import pathspec
ModuleNotFoundError: No module named 'pathspec'
```

A fresh install in a virtual environment using the following command:
```bash
$ pip install yamllint==1.32.0 pyyaml==6.0.1
$ pip freeze
pathspec==0.12.1
PyYAML==6.0.1
yamllint==1.32.0
```

Regression introduced by 60917c85e2f6645e21846fcf0197ff18244c747d

* remove `pattern` field from `.yaml` files (#10227)

* remove pattern field

* fix lint yaml files

* metastore - add support for scheduled backups (#10213)

* add support for dataproc metastore scheduled backups

* CR comments

* CR comments

* CR comments

* CR comments

* CR comments

* Update mmv1/third_party/terraform/services/dataprocmetastore/resource_dataproc_metastore_service_test.go.erb

Co-authored-by: Zhenhua Li <[email protected]>

* Update mmv1/third_party/terraform/services/dataprocmetastore/resource_dataproc_metastore_service_test.go.erb

Co-authored-by: Zhenhua Li <[email protected]>

* CR comments

---------

Co-authored-by: Zhenhua Li <[email protected]>

* Update default TF version used in TeamCity to be `"1.8.0-rc1"` (#10276)

* Update default TF version used in TeamCity to be 1.8.0-rc1

This will be updated to 1.8.0 after the GA release

* Update pom.xml to pull in a hardcoded server-api dependency version : 2024.07-SNAPSHOT

* Fix issue where `cloudquotas` was imported twice in GA provider

When I generate the Beta provider locally I don't see the import being added

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Deepak Kumar <[email protected]>
Co-authored-by: Riley Karson <[email protected]>
Co-authored-by: Max Portocarrero CI&T <[email protected]>
Co-authored-by: Mauricio Alvarez Leon <[email protected]>
Co-authored-by: Jesse Liddle <[email protected]>
Co-authored-by: Wessel Blokzijl <[email protected]>
Co-authored-by: Jack McCluskey <[email protected]>
Co-authored-by: hao-nan-li <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Oliver Krause <[email protected]>
Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
Co-authored-by: Yanwei Guo <[email protected]>
Co-authored-by: Hamzawy63 <[email protected]>
Co-authored-by: Hamza Hassan <[email protected]>
Co-authored-by: Ryan Oaks <[email protected]>
Co-authored-by: Sneha-at <[email protected]>
Co-authored-by: Shogo Watanabe <[email protected]>
Co-authored-by: Obada Alabbadi <[email protected]>
Co-authored-by: NA2047 <[email protected]>
Co-authored-by: Yuki Watanabe <[email protected]>
Co-authored-by: Scott Suarez <[email protected]>
Co-authored-by: askubis <[email protected]>
Co-authored-by: Pavan Kumar Sunkara <[email protected]>
Co-authored-by: ajaybgunjal1 <[email protected]>
Co-authored-by: Hossein Golestani <[email protected]>
Co-authored-by: rahul2393 <[email protected]>
Co-authored-by: Chris <[email protected]>
Co-authored-by: bcreddy-gcp <[email protected]>
Co-authored-by: Matt Santa <[email protected]>
Co-authored-by: Sam Levenick <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>
Co-authored-by: wj-chen <[email protected]>
Co-authored-by: Thomas Rodgers <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
Co-authored-by: yiyinglovecoding <[email protected]>
Co-authored-by: Cameron Thornton <[email protected]>
Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Francis (Feng) Liu <[email protected]>
Co-authored-by: makuing <[email protected]>
Co-authored-by: Lingkai Shen <[email protected]>
Co-authored-by: Jared <[email protected]>
Co-authored-by: Joakim Tangnes <[email protected]>
Co-authored-by: Laurens Knoll <[email protected]>
Co-authored-by: Rustem Bekmukhametov <[email protected]>
Co-authored-by: xuchenma <[email protected]>
Co-authored-by: Esha Goel <[email protected]>
Co-authored-by: Filip 'Cherit' Szóstak <[email protected]>
Co-authored-by: Benjamin Kaplan <[email protected]>
Co-authored-by: jinyangtang <[email protected]>
Co-authored-by: Naitian Liu <[email protected]>
Co-authored-by: Sneha Prasad <[email protected]>
Co-authored-by: Sneha Prasad <[email protected]>
Co-authored-by: rishamchokshi <[email protected]>
Co-authored-by: kautikdk <[email protected]>
Co-authored-by: kangy-google <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Pavel Salnikov <[email protected]>
Co-authored-by: Michael R. Torres <[email protected]>
Co-authored-by: Esha Goel <[email protected]>
Co-authored-by: Luca Prete <[email protected]>
Co-authored-by: Luca Prete <[email protected]>
Co-authored-by: gurankitt <[email protected]>
Co-authored-by: Antoni Zawodny <[email protected]>
Co-authored-by: praseedhaPK <[email protected]>
Co-authored-by: Charles Leon <[email protected]>
Co-authored-by: Charles Leon <[email protected]>
Co-authored-by: Andrew Grasso <[email protected]>
Co-authored-by: Krishnan Gopal <[email protected]>
Co-authored-by: shijeesh-ns <[email protected]>
Co-authored-by: kumailkermalli-datatonic <[email protected]>
Co-authored-by: Xuran Liu <[email protected]>
Co-authored-by: Rostislav Bobrovsky <[email protected]>
Co-authored-by: Ali Ebrahim <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: roop2 <[email protected]>
Co-authored-by: fayssalmartanigcp <[email protected]>
Co-authored-by: Patrick Costello <[email protected]>
Co-authored-by: Daniel Randell <[email protected]>
Co-authored-by: joelkattapuram <[email protected]>
Co-authored-by: Harrison Powers <[email protected]>
Co-authored-by: liferoad <[email protected]>
Co-authored-by: Pawel Jasinski <[email protected]>
Co-authored-by: Rohit Jangid <[email protected]>
Co-authored-by: krishnangopal1810 <[email protected]>
Co-authored-by: Jashan Sudan <[email protected]>
Co-authored-by: Aleks Vujić <[email protected]>
Co-authored-by: Ilia Lazebnik <[email protected]>
Co-authored-by: Srevin Saju <[email protected]>

[upstream:8821b6ce8241ddfd68707af219c52493dcce5d34]

Signed-off-by: Modular Magician <[email protected]>
  • Loading branch information
modular-magician committed Mar 26, 2024
1 parent d522723 commit 15c1b21
Show file tree
Hide file tree
Showing 16 changed files with 485 additions and 0 deletions.
15 changes: 15 additions & 0 deletions dataproc_metastore_service_scheduled_backup/backing_file.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
31 changes: 31 additions & 0 deletions dataproc_metastore_service_scheduled_backup/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
resource "google_dataproc_metastore_service" "backup" {
service_id = "backup-${local.name_suffix}"
location = "us-central1"
port = 9080
tier = "DEVELOPER"

maintenance_window {
hour_of_day = 2
day_of_week = "SUNDAY"
}

hive_metastore_config {
version = "2.3.6"
}

scheduled_backup {
enabled = true
cron_schedule = "0 0 * * *"
time_zone = "UTC"
backup_location = "gs://${google_storage_bucket.bucket.name}"
}

labels = {
env = "test"
}
}

resource "google_storage_bucket" "bucket" {
name = "backup-${local.name_suffix}"
location = "us-central1"
}
7 changes: 7 additions & 0 deletions dataproc_metastore_service_scheduled_backup/motd
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
79 changes: 79 additions & 0 deletions dataproc_metastore_service_scheduled_backup/tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Dataproc Metastore Service Scheduled Backup - Terraform

## Setup

<walkthrough-author name="[email protected]" analyticsId="UA-125550242-1" tutorialName="dataproc_metastore_service_scheduled_backup" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
15 changes: 15 additions & 0 deletions integrations_client_advance/backing_file.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
34 changes: 34 additions & 0 deletions integrations_client_advance/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
data "google_project" "test_project" {
}

resource "google_kms_key_ring" "keyring" {
name = "my-keyring-${local.name_suffix}"
location = "us-central1"
}

resource "google_kms_crypto_key" "cryptokey" {
name = "crypto-key-example"
key_ring = google_kms_key_ring.keyring.id
rotation_period = "7776000s"
depends_on = [google_kms_key_ring.keyring]
}

resource "google_kms_crypto_key_version" "test_key" {
crypto_key = google_kms_crypto_key.cryptokey.id
depends_on = [google_kms_crypto_key.cryptokey]
}

resource "google_integrations_client" "example" {
location = "us-central1"
create_sample_workflows = true
provision_gmek = true
run_as_service_account = "radndom-service-account"
cloud_kms_config {
kms_location = "us-central1"
kms_ring = google_kms_key_ring.keyring.id
key = google_kms_crypto_key.cryptokey.id
key_version = google_kms_crypto_key_version.test_key.id
kms_project_id = data.google_project.test_project.id
}
depends_on = [google_kms_crypto_key_version.test_key]
}
7 changes: 7 additions & 0 deletions integrations_client_advance/motd
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
79 changes: 79 additions & 0 deletions integrations_client_advance/tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Integrations Client Advance - Terraform

## Setup

<walkthrough-author name="[email protected]" analyticsId="UA-125550242-1" tutorialName="integrations_client_advance" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
15 changes: 15 additions & 0 deletions integrations_client_basic/backing_file.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
3 changes: 3 additions & 0 deletions integrations_client_basic/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
resource "google_integrations_client" "example" {
location = "us-central1"
}
7 changes: 7 additions & 0 deletions integrations_client_basic/motd
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
===

These examples use real resources that will be billed to the
Google Cloud Platform project you use - so make sure that you
run "terraform destroy" before quitting!

===
79 changes: 79 additions & 0 deletions integrations_client_basic/tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Integrations Client Basic - Terraform

## Setup

<walkthrough-author name="[email protected]" analyticsId="UA-125550242-1" tutorialName="integrations_client_basic" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>

Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.

<walkthrough-project-billing-setup></walkthrough-project-billing-setup>

Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.

## Terraforming!

Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
the project name from the environment variable.

```bash
export GOOGLE_CLOUD_PROJECT={{project-id}}
```

After that, let's get Terraform started. Run the following to pull in the providers.

```bash
terraform init
```

With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!

```bash
terraform apply
```

Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.

```bash
yes
```


## Post-Apply

### Editing your config

Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.

```bash
terraform plan
```

So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
run a 'plan' again.

```bash
terraform plan
```

Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
at the 'yes' prompt.

```bash
terraform apply
```

```bash
yes
```

## Cleanup

Run the following to remove the resources Terraform provisioned:

```bash
terraform destroy
```
```bash
yes
```
15 changes: 15 additions & 0 deletions pubsub_topic_ingestion_kinesis/backing_file.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file has some scaffolding to make sure that names are unique and that
# a region and zone are selected when you try to create your Terraform resources.

locals {
name_suffix = "${random_pet.suffix.id}"
}

resource "random_pet" "suffix" {
length = 2
}

provider "google" {
region = "us-central1"
zone = "us-central1-c"
}
Loading

0 comments on commit 15c1b21

Please sign in to comment.