From 15c1b21411fb81cd9dda6c71bbd90527dfcbffd2 Mon Sep 17 00:00:00 2001 From: Modular Magician Date: Tue, 26 Mar 2024 16:30:17 +0000 Subject: [PATCH] Sync main feature branch provider functions (#10273) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes https://github.com/hashicorp/terraform-provider-google/issues/17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com> * tabs to spaces in resource block Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com> --------- Co-authored-by: G-NamanGupta Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path * Adds description for default value of `cpuidle` field (#10005) * add description * remove tail space * Add location field in DNS authorization resource. (#9968) * Add location field in DNS authorization resource * Fixed the provider version of the updated config --------- Co-authored-by: Hamza Hassan * Add discoveryengine service (#10017) * Add validations for Composer 2/3 only fields (#9917) * block upgrade to composer 3 * make isComposer3 more generic, correct imageVersionChangeValidationFunc * added validation for Composer 2/3 specific fields * add tests for validation * add checks in flattenComposerEnvironmentConfig * Update attributes of fields not used in Composer 3 * make customizeDiff functions beta only * remove Computed from gke_cluster * remove Optional instead of Computed * add envCfg.PrivateEnvironmentConfig is nil check * modify isComposer3 to take string * minor correction to avoid merge conflicts * Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (#10034) Fixed https://github.com/hashicorp/terraform-provider-google/issues/17319 * Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (#10011) Co-authored-by: Hamza Hassan * Added missing quote marks to TEST_PREFIX (#10035) Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied * Promote enable_confidential_storage from beta to GA (#9993) * Add IAM resources for Cloud Deploy Target (#9927) * Add IAM resources for Cloud Deploy Target * update * add back import_format * Add comments to skipped sweepers (#10028) * Add sweepers for gkebackupPlan and restorePlan (#10026) * add `remote_function_options` to bigquery_routine (#9893) * Add dataGovernanceType and remoteFunctionOptions to bigquery_routine * add function-sources.zip to biguquery fixtures * fix resource names in TestAccBigQueryRoutine * add bigquery routine remote function example * Adding documentaiton build_worker_pool (#10022) * docs: fix import example in google_project_iam (#10007) Fix the example in the `google_project_iam` documentation. https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import * ScottSuarez not on vacation anymore (#10052) * I'm on vaction :) * ScottSuarez not vaction anymore * ScottSuarez not vaction anymore * Promote (R)IGM.AllInstancesConfig to V1 (#9964) * Promote metric settings in compute region autoscaler to GA (#10045) * Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (#10040) * Retry delete network step while creating a google project. (#10046) * Adding the namespace_labels field to the GKE Hub Scope resource (#9972) * initial commit for scope-level namespace labels * Add validation exceptions for the field of the GKEHub Scope and Namespace resources to be of type * Undoing unnecessary changes * Fixing the type of Scope namespace_labels field * Support connectivity fields for Composer 3 (#9889) * add composer_network_attachment * indicate conflicting configs * commas * no need for bidirectional conflict definition (generates double errors) * protect nit PrivateClusterConfig * for optimizing error messages about conflicts * add 2 step update for composer_network_attachment * make composer_network_attachment available in beta only * add two step update for network and subnetwork * corrections in 2 phase update for network/subnetwork * remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working) * added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains * add docs for composer_network_attachment * add test for network attachment * ignore non empty plan in network attachment test * add networkAttachment update and conflicting fields tests * add ComputedIf for network, change isComposer3 * minor corrections * remove computedIf * filter equivalent values of network/subnetwork in ForceNewIf * simplify ResourceConditionFunc, add beta/ga version conditions * typo * more general comparison of network references * use tpgresource.CompareSelfLinkRelativePaths instead of custom function * modify isComposer3 to avoid merge conflicts later. * removing this since documentation is handled in other PR and to avoid conflicts while merging. * replace ExpectNonEmptyPlan with lifecycle.ignore_changes * add testcase for changing network attachment to network and subnetwork * add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork * modify tests to use different network for attachment * remove unused constant * remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes) * Fix resource name to have sweepable prefix (#9984) * doc(spanner): update documents to have example for IAM conditions use with google spanner database (#10049) * Promote Cloud Deploy Automation to GA (#10043) * Add Cloud Deploy Automation * Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384 * Add update test for the new automation resource * fix lint errors * fix lint errors * fix errors in GA provider test * add the missing test * add a full test * Attempt to fix the acceptance test errors * fix a lint error * mark labels as default_from_api:true * fix advance rollout rule definition * Mark service_account as 'ignore_read: true' * Fix test errors * Resolve review comments * Format test file * promote google_clouddeploy_automation to GA * promote google_clouddeploy_automation to GA * fix lint errors * Add cloudquota service (#10063) * TeamCity: Add validating GHAs that check Kotlin files listing services (#9999) * services diff gha * add needs * use setup-go@v3 * add artifacts * remove checkout * update directory * update common-copy.yaml * update common-copy.yaml * remove data print * add pull-request check on teamcitydiff * update paths in GHA * add exit code in diff_check * update exit output * add exit code diff_check.go * services_beta diff check support * beta services support args * add arg for services kt file * services_ga.kt in gha * move diff_check to tools folder * remove go command * provide path for artifacts download * add cd in Build Provider * update diff_check_beta.yml * fix paths * yml typo beta * directory testing * directory testing * fix directory issues with commands * refactor tools/teamcity-diff-check/main.go * directory check * typo * remove common copy * generate both providers into one gha * type artifacts name * add -o flag into artifacts download * use merge multiple artifacts * use artifact@v4 * use mmv1 directory for services file * remove .zip * output missing services from diff * find artifacts folder * use beta provider as only artifact * include google ga and beta in gha * artifact name * -o flag in artifacts * output stdout from go list command * diff test * output beta main.go * remove err check * remove services print / refactor for final review * output cleanup * uncomment go list err * regex assert * add check for new services in PR / set googleServices as groundtruth in serviceDifference function * add actions/checkout * logic fix * add needs: check-pr * add mmv1/products for pull-request event * set force depth 0 * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> * Update .github/workflows/teamcity-services-diff-check.yml Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> * use text file for services, add if statement in gha * directory typo * typo * typo in grep * quotations on if statement * unquote GITHUB_OUTPUT * add quotes on output.services * if test * echo output * github_output error fix * services.outputs * proper if statement syntax gpush * invalid 0 format * add missing assignment * fix openfile error * remove types in pull_request trigger * send txt files to correct directory * check services directory for ls command * add weekly workflow for teamcity diff check * change name for weekly check --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> * Fixed `runtime` in app engine tests (#10047) * Fix google_notebooks_instance labels not being able to be updated (#9933) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * Add IAM resources for Cloud Deploy Custom Target Type (#10051) * Add IAM resources for Cloud Deploy Custom Target Type * Add base url override for Cloud Deploy Custom Target Type IAM policy * Update import format for IAM policy and primary resource name to examples for IAM generated tests * Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (#10075) * Remove unneeded GHA steps - do we need to cache here? * Whitespace * Test change with fake new service * Resolve build error in fake service * Re-order GHA steps and add name * Add name to checkout step * Add comments, remove extra whitespace * Remove fake new service * Add output to identifiers, check for project as well as projectsId (#10081) * Add output to identifiers, check for project as well as projectsId * Correct identifier behavior * Rubocop * remove nickelliot/zli82016 from vacation reviewers (#10080) * Update membership.go * Update membership.go * Update enrolled_teams.yml (#10082) * Fix resource_bigquery_dataset ID validation (#10027) * fix resource_bigquery_dataset ID validation * skip the new validation tests for VCR * update dataset IDs to be valid in terraform-google-conversion example files * update more dataset IDs in the CAI files * Prevent duplicate entries in vcr results (#10068) * Prevent duplicate entries in vcr results * Create empty map * Only collect result from current test * Split github tokens (#9988) * Split github tokens * Update .ci/gcb-generate-diffs-new.yml Co-authored-by: Stephen Lewis (Burrows) * Remove redundant downstreams token * Make diff processor use new token * Update path to markdown file * Replace GITHUB_TOKEN * Make github tokens optional for generate downstream * Allow either github token to be used * Replace GITHUB_TOKEN * Move environment variable lookup out of constructor * Update .ci/magician/vcr/tester.go Co-authored-by: Stephen Lewis (Burrows) * Add downstream token * Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC * Apply suggestions from code review Co-authored-by: Stephen Lewis (Burrows) --------- Co-authored-by: Stephen Lewis (Burrows) * Instance lifecycle policy default action on failure implementation (#10050) * Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (#10076) * Add wait in acc test to avoid 'API has not been used in project' error * Add external provider `time` * Fix defect when referencing google_project resource * Set GITHUB_TOKEN environment variable for hub (#10091) * fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089) * make go-converted YAML compatible with go compiler (#10033) Co-authored-by: Zhenhua Li Co-authored-by: Nick Elliot * Revert "Promote metric settings in compute region autoscaler to GA" (#10095) This reverts commit c3ce700765fbc4b3dc9896da767194b8eda9a13d. * Add type field to DNS authorization reosurce (#10030) * Add type field to DNS authorization reosurce * Add an example for regional DNS authorization * Add an example for regional certs using regional DNS auth * Fix lint errors * Fix typo in the enum values * Add type field in regional dns auth example --------- Co-authored-by: Hamza Hassan * TeamCity: Refactor config tests (#9956) * Add test util for locating a subproject 2 layers deep * Refactor tests to use new helper function * Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value * Refactor new sweeper tests * Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (#10098) * TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (#10099) * Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (#10089)" (#10096) * subnetwork and service_account_email params described (17211) (#10102) * subnetwork and service_account_email params described (17211) * Doc adjustment * Update provider name in various spots (#10092) * Go compiler skeleton and early terraform.rb provider code (#10104) * Remove provider-related caches to address cache issues (#10097) We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out. * Fix hashicorp/terraform-provider-google#17388 (only run tests in beta) (#10093) * Revert "Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)" (#10111) This reverts commit ae819d0619edd36414af1c5dc5f9e68a6b999174. * Add support for string --> object map for DCL resources (#10039) * Add support for string --> object map for DCL resources * Fix whitespace in template * Add test, need updated DCL to work * Add override for key name * Both overrides * Update docs, prepend key for maps * Update bundle descriptions * Comment update * fix permadiff by reading empty docker_config field (#10113) * Import package golang.org/x/exp/slices in MMv1 go compiler (#10108) * fix(kms): certificate chain type to array of strings (#9582) * Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (#10110) * Update all uses of `actions/cache` to v3 * Update all uses of `actions/checkout` to v4 * Update all uses of `actions/upload-artifact` to v3.1.0 * Update all uses of `actions/setup-go` to v4.0.0 * Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2 [v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20 The previous version, ruby/setup-ruby@ec02537da5712d66d4d50a0f33b7eb52773b5ed1, is v1.144.2 * Address dependency issues in TestAccFirestoreField_* tests (#9957) * Add additional wait in TestAccFirestoreField_* tests * Boost wait in test to 6 minutes * Add dependency between database and service to control delete order * Update dependency to explicitly include project * Make firestore fields be removed from state when they're 'deleted' * Add `destroy_duration` * Remove from state after log line that uses id value * Update destory check to accept a 403 as valid * Remove unneeded changes in PR * Remove call to SetId * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (#9895) * add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource * Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes * Update the GCF resource to reflect transition from Container Registry to Artifact Registry (#10058) * Handwrite sweepers for networkConnectivity hubs and spokes (#10069) * Fix wrong variable in override logic (#10125) * Make missing test detector reader into a separate module (#10115) * Use old github token if new tokens are not available (#10114) * Use old github token if new tokens are not available * Add lookup function for github token or fallback * Add fall back * Add fallback * Add test for PATCH environment type (#10131) * Add new resource for Application for Apphub (#10079) * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix examples * Remove apphub from teamcity config This causes PR build failures, I'll add it later --------- Co-authored-by: Sam Levenick * Add apphub (#10133) Adds AppHub to APIs to activate * Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (#10031) * Support service-level min instances in Cloud Run v2 services. (#10083) * feat: Add CMEK support for Firestore database in Beta provider (#10044) * Modify database.yaml to add cmek related fields * Add two examples for firestore CMEK databases for testing * Resolve trailing space * Update documentation for kmsKeyName field * Resolve trailing space * Make field immutable * Update field documentation * Update field description * Allow vcenter_network to be set (#9946) * Allow vcenter_network to be set * set default from api * Sort resources and set in product in go compiler (#10135) * fix failing posture test (#10086) * fix failing posture test * modify posture name --------- Co-authored-by: Sneha Prasad * Create support for KMS Ekmconnection resource (#10094) * Fixes two lifecycle rules with different no_age value always generates change. (#10137) * TeamCity: Add project for testing the provider functions feature branch (#10088) * Add ability to use non-default versions of Terraform in TeamCity builds * Add function to enable making build configs for single packages at a time * Add new sub project that contains 2 builds for testing provider functions the 2 builds: 1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo 2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo These builds both use an alpha release of TF 1.8.0 * Add builds for testing auto generated branches in the MM upstream repos These re-use existing VCR Roots. * Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time * Fix defect in 'Download Terraform' build step definition * Update build step to solve bug * Update build_configuration_per_package.kt * Add `ephemeral_directories` to google_workstations_workstation_config (#10042) * Add `ephemeral_directories` to google_workstations_workstation_config * Remove trailing spaces * Add a test for ephemeral_directories * Fix test * Add a test for source_image * Fix typo in test * Remove unnecessary immutable field * App Check DeviceCheck provider (#9978) * App Check DeviceCheck provider * Remove minimal example & pattern field * Add real private keys that are not useful anywhere * Limit tests to beta * Documentation (#10012) * duplicate Composer 2 argument reference * remove fields that are not supported in Composer 3 * move fields that are new in Composer 3 to Composer 3 section only * make suggested changes, add description if new versioning schema * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * add notice that composer 3 is not yet released. * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Stephen Lewis (Burrows) * specify composer version in section links. * specify composer version in composer 1 documentation links * add section links in composer 2 argument reference * add section links in composer 3 argument reference * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> * Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> --------- Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> Co-authored-by: Stephen Lewis (Burrows) * Promote metric settings in compute region autoscaler to GA (#10103) * Clarified permissions for a few workflows (#10124) * Clarified permissions for a few workflows * fixed conditions for running repository-documentation workflows * Bumped actions/checkout to v4 * force downstream generation * Revert "force downstream generation" This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a. * Post statuses earlier (#10128) * Post initial statuses earlier so that we are resilient issues during cloning * force generation * Revert "force generation" This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140. * Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (#10140) * Simplified / clarified approval flow for presubmits (#10142) * Simplified / clarified approval flow for presubmits * Force generation * Removed unused command args and cleaned up documentation for membership-checker * Revert "Force generation" This reverts commit db32066e198daf930006f72bae43602f22d8e3d9. * Clean up delete template (#10144) * added missing web documentation about create_timestamp for compute_(region_)instance_group_manager (#10148) * Move pre_delete code closer to the request (#10143) * Create resource definition for IAP Tunnel DestGroup (#10064) * Add new resource for Service Project for Apphub (#10048) FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * [#15779] Add google_network_security_firewall_endpoint_association resource (#9815) * [#15779] Add google_network_security_firewall_endpoint_association resource * Fixing tests * Fixes * Add test for update --------- Co-authored-by: Luca Prete * update teamcity diff check to run every monday morning also (#10159) * Add DisplayName to Product struct (#10161) * Add data source for apphub discovered service (#10105) * Add data source for apphub discovered service * Add data source for apphub discovered service' * Add data source for apphub discovered service * resolved comments * retry logic added * add tests and documentation * Corrected tests and added project field in the data source * changed id field * Added random_suffix for the resources created and enabled iam policy and compute api * modified retry logic * Modified schema for the data source, and tests * Removed IAM permission blocks * Add dependency for compute api * Resolve merge conflict * Add time sleep * Modified test function name * Shorten service project name * Add billing account * corrected get env variable function call * Modified project id * Combined time delay * Resolving comments * Remove retry logic and add time sleep for resource ingestion * Graduate support of GKE Queued Provisioning to GA (#10053) * Update index page of documentation to link to issue templates for Bugs and FRs (#10160) * Update index page of documentation to link to issue templates for Bugs and FRs * Remove unused param from URL * Remove other unused parameter from URL * Add data source for AppHub discovered workload (#10107) * Add data source for Apphub discovered workload * Add data source for Apphub discovered workload * Add data source for Apphub Discovered Workload * Resolved comments * Resolved comments * Resolved comments * Resolved comments * Add tests and documentation for data source discovered workload * Resolved comments and added tests * Resolved comments * Added modifications in tests, data source and documentation * Added modifications in tests, data source and documentation * Added modifications in tests, data source and documentation * Verified tests * Tests Verification * Tests Verification * Updated logic to obtain workload_uri * Updated logic to obtain workload_uri * Resolved comments * Resolved comments * Add billing account and shorten service project name * Add header * Change the project name to start with tf-test * Resolved comments * Lint changes * Removing two sleeps * Removing two sleeps * Modifying documentation * Resolved tests * Resolved comments * Resolved comments * logic modification * Update Retry logic * Update Retry logic * Resolved tests * Adding workload_uri description * Adding workload_uri description * Update Documentation for ACM Service Perimeter resources to reflect Granular Controls group support (#10087) groups Co-authored-by: Charles Leon * fix tags cause notebooks instances to recreate (#10179) * add labels to runtime update test * Add labels to ImportStateVerifyIgnore * fix indentation * fix tags cause notebooks instances to create * Documentation Fix compute_security_policy (#9732) Fix mixed up descriptions between request_uri and request_query_param. * Add new resource Workload for Apphub (#10155) * Add new resource Workload for Apphub * Fix lint error * Add billing account * Remove extra delay from test * Resolve comments * Add handwritten update test * Fix lint error * Add data source changes * Remove merged changes * Remove merged changes * Batch all update tests into one step to make test fast * Add a delay between discovered resource fetch and managed instance group manager creation * Add back org_id * Update region from us-east1 to us-central1 * Use standard diff suppress function --------- Co-authored-by: praseedhaPK <161299686+praseedhaPK@users.noreply.github.com> Co-authored-by: Krishnan Gopal * Fix for PrivateNat test failures (#10174) * Add new resource Service for Apphub (#10132) * Add new resource for Service Project for Apphub FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * Add new resource for Service Project for Apphub FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * Add new resource for Service Project Attachment for Apphub FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * Add new resource for Service Project Attachment for Apphub FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * Add new resource for Service Project Attachment for Apphub FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * Add new resource for Application for Apphub * Add new resource for Application for Apphub * Enable Apphub API in test cases * Add new resource for Service Project Attachment for Apphub FIXES https://github.com/hashicorp/terraform-provider-google/issues/17405 * Enable apphub API in test cases * Fix precheck error * Fix precheck error * Fix precheck error * Fix precheck error * Resolve comments * Fix lint error * Fix errors * Add new resource for Service for Apphub * Fix examples * Add test * Fix tests * Add IAM permissions * Add IAM permission * Fix IAM issues * Remove merged changes * Add billing account * Remove extra delay from tests * Add handwritten update test * Remove merged changes * Make changes to yaml file incorporating comments from https://github.com/GoogleCloudPlatform/magic-modules/pull/10155 * Fix lint errors in Service YAML definition * Add a delay between discovered resource fetch and forwarding rule creation * Add a delay between discovered resource fetch and forwarding rule creation * Add a delay between discovered resource fetch and forwarding rule creation * Batch update tests to improve speed * Use standard diff suppress function * Update region from us-east1 to us-central1 --------- Co-authored-by: Krishnan Gopal Co-authored-by: praseedhaPK <161299686+praseedhaPK@users.noreply.github.com> * feat: add credentials_secret field in azure blob storage block for google storage transfer job resource (#9278) * feat: add `credentials_secret` in `azure_blob_storage_data` source config * feat: add version guard for `credentials_secret` as in preview * docs: add documentation on `credentials_secret` * Added CloudQuotas service and Create QuotaInfo Datasource (#10071) * add quota info data source * add test and documention for quota_info data source * fix lint error * clean up * update test * manually import cloudquotas to terraform provider * Update mmv1/third_party/terraform/provider/provider_mmv1_resources.go.erb Co-authored-by: Riley Karson * Update mmv1/third_party/terraform/provider/provider_mmv1_resources.go.erb Co-authored-by: Riley Karson --------- Co-authored-by: Riley Karson * Update comments in .teamcity/ files for clarity (#10149) * Make warning about files being controlled by MM more explicit * Add guidance to the input files about how to update them, and when * Add fields pathTemplateMatch and pathTemplateRewrite to resource google_compute_region_url_map (#10157) * Refactor TeamCity Services Diff Check to run as one job (#10158) * Add explicity dependency on IAM in PubSub acc test config (#9921) * Improve the error messages when mmv1 compilation fails (#10191) * Update enrolled teams (#10192) * Fix generate comment diff processor build failure (#10164) * Refactored generate_comment to continue in the face of errors Also switched to go template for comment formatting * Moved to structured data for diffs Also added error formatting into comment * Added basic tests for comment formatting * Skipped missing tests and breaking changes if repo failed to clone * Force generation * Breaking change * Fixed error display in diff comment * Marked diff-processor targets as not being real files * Added PATH to diffProcessorEnv * Fixed formatting for error sections * Cleaned up passthrough env var usage * Made exec.sh exit 1 if there are any errors running the magician * Don't error on initial build of the magician binary * Added missing env var for diff processor build * Force generation of tf-oics * Revert "Force generation of tf-oics" This reverts commit fcb65f121cbb0f7040a6b40c635eae3b318d936d. * Revert "Force generation" This reverts commit 596d1eb5afb10d36ab934411e0f1c47aa8b363e9. * Revert "Breaking change" This reverts commit fb04ad401e1d4ee93ef767297a2a59af7d2acffa. * Force missing test run * force missing tests to run - take 2 * Revert "force missing tests to run - take 2" This reverts commit 34f7d16a8442f09aa4a52e0ceafb8dae73f89b08. * Revert "Force missing test run" This reverts commit 90e664a935c4645314c3aabf1f48836e4547fc68. * Rewrite versions related functions (#10181) * Update Dockerfiles to Go 1.21 (#10175) * Add an override for generating long form tests (#10162) * Update to go 1.21 (#10169) * Update .go-version file in TPG and TPGB repos * Update go.mod used to generate TPG/TPGB and run `go mod tidy` * Update go.mod in mmv1/third_party/terraform/scripts * Update go.mod in mmv1 * Update go.mod in docs * Update go.mod in tools/go-changelog and run `go mod tidy` * Update go.mod in tools/missing-test-detector and run `go mod tidy` * Update go.mod in tools/diff-processor * Update go.mod in tools/issue-labeler * Update go.mod in tools/teamcity-generator * Update go.mod in tpgtools and run `go mod tidy` * Update doctor script to ensure developers use Go 1.21 locally * Update GHA workflows to use Go 1.21 * Add missing update to GHA * Add note about permissions (#10195) * Fix the bug for computed labels and annotations (#10182) * AndroidApp package_name should be immutable & required (#10194) * Fix service team reviews (#10200) * Corrected unknown command messge to clean up stderr * Made tests throw error if team name is empty * Fixed handling of unset github_team name * Add gcs volume type to Cloud Run v2 beta provider (#9746) * Add gcs volume type to Cloud Run v2 beta provider * update gcs v2 docs to specify required launch stage and execution environment * merge changes with upstream * cloudrunv2: Validate number of ports specified restricting to 1. (#10173) * cloudrunv2: Validate number of ports specified restricting to 1. * Update mmv1/products/cloudrunv2/Service.yaml * Bump google.golang.org/protobuf from 1.28.1 to 1.33.0 in /.ci/magician (#10196) Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * attempt at fixing the (R)IGM.status output message (#10188) * attempt at fixing the (R)IGM.status output message * Fix plural of instances * Fix plural of instances * Fix plural of instances * Fix plural of instances * Access Context Manager - make ingress and egress rules immutable (#10147) * Access Context Manager - make ingress and egress rules immutable * Move immutable state to top level resource --------- Co-authored-by: Charles Leon * Bump golang.org/x/net from 0.10.0 to 0.17.0 in /.ci/magician (#9243) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.17.0. - [Commits](https://github.com/golang/net/compare/v0.10.0...v0.17.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix datastream private connection sweeper (#10201) * fix datastream private connection sweeper * update * run gofmt * Add SarahFrench to on vacation list (#10189) * Modify issue labeler to skip linked issues (#10199) * Modify issue labeler to skip linked issues * Update tests * Remove misleading examples from docs for `google_dataform_repository`, add new example (#10187) * Remove misleading examples from docs for `google_dataform_repository`, add new example * Remove unusued var from new example * Fix typo * Fix other instances of typo * Bump google.golang.org/grpc from 1.53.0 to 1.56.3 in /.ci/magician (#9366) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.53.0 to 1.56.3. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.53.0...v1.56.3) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Moving the netapp kmsconfig resource update test to a different region to avoid conflict with create test (#10185) * Only run mmv1 rake tests and yaml linting if relevant files have changed (#10207) * Only run mmv1 rake tests and yaml linting if relevant files have changed * fix path for rb files Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> * Clean up functions and add slice utility functions (#10210) * Clean up function all_resourcerefs * Add slice utility functions * Change sensitive_props as resource method * Removed github-differ bash scripts (#10208) * Add ingestion settings to `google_pubsub_topic` (#9985) * Add ingestion settings to `google_pubsub_topic` * Add ingestion settings to `google_pubsub_topic` * Add create/update tests for `google_pubsub_topic` ingestion settings * Update topic ingestion settings test * Run Firestore index tests in fresh databases. (#10168) * Run Firestore index tests in fresh databases. This removes the index dependency on creating a document, as creating indexes on a non-existent collection is a valid operation in Firestore. This removes the dependency on a fixed (default) database, which should deflake tests. Fixes #17393. * Move project_id to test_env_vars. * Move project_id to test_env_vars for datastore_mode example. * AccessContextManager - Add dry run service perimeter resource (#10145) Co-authored-by: Charlesleonius * Create quota infos datasource (#10074) * add quota info data source * add test and documention for quota_info data source * fix lint error * add quota infos datasource * update read test * remove duplicate import * update formatting and documentation * remove pagination parameters, concat and return all qualified entries * remove specific check for number of entries * fix(spanner): remove validation on version retention period on spanner database (#10184) * fix(spanner): remove validation on version retention period on spanner database * fix build * initial mmv1 go templating and flags (#10219) * Adds support for Soft Delete feature, which allows setting soft delete policy on 'google_storage_bucket' resource. (#10171) * Fixes and enhancements after the bug bash. (#10223) * remove float from documentation * change storage-gb type to int * Revert "change storage-gb type to int" This reverts commit 89795a80889287fb287497fbcbbfdbb365d40884. * add support for dag_processor count * fix int64 unpacking * add validation for composer internal ip range size * add upper limit for dag_prcessor.count * google_monitoring_alert_policy periods doc update & dataproc doc update (#10226) * enable_http_port_access is a boolean not a string * add doc update to alert policy notification rate limit period * Add support for boost configs in workstations configs (#10176) * add boostConfig workstationConfig * add boostConfig workstationConfig * link example * remove * update descriptions and add update test * update machine type * fix update test * Add a rule for removing diff suppress functions (#10167) * Add a rule for removing diff suppress functions * Add rule link to docs * Rewrite Resource functions (#10228) * Add missing guest_os_features enumeration value (#10229) * add billing project id support to firewall endpoint resource (#10122) * add billing project support * removed description field due to API issue * test updated and fixed for ADC support * added ADC warning * removing ADC * Upgraded version of actions related to scorecard (#10234) * Don't re-request review if initial PR comment is edited (#10232) * Pinned versions for GitHub Actions (#10239) * Switched from diff-processor adding labels to just computing and returning them as JSON (#10211) * Switched diff-processor from add-labels command to changed-schema-labels command * Force update to redis cluster * Revert "Force update to redis cluster" This reverts commit 750c39ddf7f30d8a448fb775b86711ecbf3830f0. * change the dataflow zone from us-central1-f to us-east5-b (#10205) * Added `CloudQuotas` service and Create `QuotaPreference` Resource (#10019) * setup resoruce config and add create tests * add create and update test * fix lint error * fix lint error * update yaml config * remove trailing spaces * modify update test * update acceptance test and cleanup * add upsert scenario for testing * update acceptance test syntax * remove allow-missing flag & fix update test failure * add import and id format * teamcity test failure - manually add cloudquotas service * teamcity test failure - manually add cloudquotas service for beta * add field 'validate_only' to tests * address comments & fix test failure * remove unused import * remove `validate_only` field, add ignore read feature to `annotations` field * add input only fields to ImportStateVerifyIgnore * update field name * VCR test failure: remove batch service creation * fix formatting * Magician tgc integration (#9907) * Add command for tgc integration tests * Use magician for TGC integration tests * Fix variable name * Remove owner * Fix go mod tidy command * Use new github token * Fix https://github.com/hashicorp/terraform-provider-google/issues/17389 (#10206) * generated_id added to compute_region_backend_service (#10243) * Clarify CI README a bit (#10245) * Add integrations product. Create Client resource. (#10186) Co-authored-by: Cameron Thornton * Add apphub.admin role for the organization in the test environment (#10152) * add apphub.admin role for the organizations that the CI runs the tests * Change to use org instead of billing id * Change to use org instead of billing id --------- Co-authored-by: Krishnan Gopal * Change ItemType to object (#10249) * Clarify usage of min/max throughput and instances (#10130) * Clarify usage of min/max throughput and instances * fix lint * fix snake casing * Fixed issues with diff-processor caused by GA-only compilation errors (#10250) * Added file to trigger google-only failure * Check if cleaning the diff processor up properly solves the issue * Revert "Check if cleaning the diff processor up properly solves the issue" This reverts commit 85c358e4191d5e6115bfef0e02c819f92b45dfe0. * Clean up diff processor before building instead of after * Revert "Added file to trigger google-only failure" This reverts commit 0e4486080477902820ad282666d0c8f51e890076. * Consolidate Documentation for App Hub resources and data sources into one subcategory (#10238) * add apphub.admin role for the organizations that the CI runs the tests * Change to use org instead of billing id * Change to use org instead of billing id * Fix subcategory of data sources consistent with rest of AppHub product resources * This change is supposed to be a part of another feature --------- Co-authored-by: Krishnan Gopal * Corrected names of iam_workload resource and data source files (#10253) beta is not part of the resource name. This is important for https://github.com/GoogleCloudPlatform/magic-modules/pull/10248 * Pin workflow dependency versions (#10256) * Pinned python yaml library versions * Pinned goimports version in build-downstreams * Force generation * Revert "Force generation" This reverts commit b2d876045277156ae2fd3a44194fe58164297186. * Rewrite type functions (#10259) * Removed double quotes and typo in google_project_iam_member documentation (#9750) * Iniital setup for codeql scanning (#10254) * Added permissions: read-all to codeql action (#10264) * Remove java-kotlin codeql scanning (#10263) * Initial resource documentation templating (#10266) * Add note about modifying yaml files (#10202) * bq table - add geojson support (#10215) * bq table - add geojson support * bq table - add geojson support * CR comments * Add logsBucket (#10267) * Remove SarahFrench from vacation list (#10272) * ci: add missing pathspec dependency for yamllint pipeline (#10269) pipelines for YAML lints are failing due to: ``` Traceback (most recent call last): File "/home/runner/.local/bin/yamllint", line 5, in from yamllint.cli import run File "/home/runner/.local/lib/python3.10/site-packages/yamllint/cli.py", line 24, in from yamllint.config import YamlLintConfig, YamlLintConfigError File "/home/runner/.local/lib/python3.10/site-packages/yamllint/config.py", line 19, in import pathspec ModuleNotFoundError: No module named 'pathspec' ``` A fresh install in a virtual environment using the following command: ```bash $ pip install yamllint==1.32.0 pyyaml==6.0.1 $ pip freeze pathspec==0.12.1 PyYAML==6.0.1 yamllint==1.32.0 ``` Regression introduced by 60917c85e2f6645e21846fcf0197ff18244c747d * remove `pattern` field from `.yaml` files (#10227) * remove pattern field * fix lint yaml files * metastore - add support for scheduled backups (#10213) * add support for dataproc metastore scheduled backups * CR comments * CR comments * CR comments * CR comments * CR comments * Update mmv1/third_party/terraform/services/dataprocmetastore/resource_dataproc_metastore_service_test.go.erb Co-authored-by: Zhenhua Li * Update mmv1/third_party/terraform/services/dataprocmetastore/resource_dataproc_metastore_service_test.go.erb Co-authored-by: Zhenhua Li * CR comments --------- Co-authored-by: Zhenhua Li * Update default TF version used in TeamCity to be `"1.8.0-rc1"` (#10276) * Update default TF version used in TeamCity to be 1.8.0-rc1 This will be updated to 1.8.0 after the GA release * Update pom.xml to pull in a hardcoded server-api dependency version : 2024.07-SNAPSHOT * Fix issue where `cloudquotas` was imported twice in GA provider When I generate the Beta provider locally I don't see the import being added --------- Signed-off-by: dependabot[bot] Co-authored-by: Deepak Kumar <21131061+kumadee@users.noreply.github.com> Co-authored-by: Riley Karson Co-authored-by: Max Portocarrero CI&T <105444618+maxi-cit@users.noreply.github.com> Co-authored-by: Mauricio Alvarez Leon <65101411+BBBmau@users.noreply.github.com> Co-authored-by: Jesse Liddle Co-authored-by: Wessel Blokzijl Co-authored-by: Jack McCluskey <34928439+jrmccluskey@users.noreply.github.com> Co-authored-by: hao-nan-li <100219545+hao-nan-li@users.noreply.github.com> Co-authored-by: Salome Papiashvili Co-authored-by: Oliver Krause <3621164+okrause@users.noreply.github.com> Co-authored-by: G-NamanGupta Co-authored-by: Shuya Ma <87669292+shuyama1@users.noreply.github.com> Co-authored-by: Stephen Lewis (Burrows) Co-authored-by: Yanwei Guo Co-authored-by: Hamzawy63 <43001514+Hamzawy63@users.noreply.github.com> Co-authored-by: Hamza Hassan Co-authored-by: Ryan Oaks Co-authored-by: Sneha-at Co-authored-by: Shogo Watanabe <1286807+nownabe@users.noreply.github.com> Co-authored-by: Obada Alabbadi <76101898+obada-ab@users.noreply.github.com> Co-authored-by: NA2047 <12290725+NA2047@users.noreply.github.com> Co-authored-by: Yuki Watanabe <47182350+yuki0920@users.noreply.github.com> Co-authored-by: Scott Suarez Co-authored-by: askubis Co-authored-by: Pavan Kumar Sunkara Co-authored-by: ajaybgunjal1 <161062655+ajaybgunjal1@users.noreply.github.com> Co-authored-by: Hossein Golestani Co-authored-by: rahul2393 Co-authored-by: Chris Co-authored-by: bcreddy-gcp <123543489+bcreddy-gcp@users.noreply.github.com> Co-authored-by: Matt Santa Co-authored-by: Sam Levenick Co-authored-by: Nick Elliot Co-authored-by: wj-chen Co-authored-by: Thomas Rodgers Co-authored-by: Stephen Lewis (Burrows) Co-authored-by: yiyinglovecoding <113492735+yiyinglovecoding@users.noreply.github.com> Co-authored-by: Cameron Thornton Co-authored-by: Zhenhua Li Co-authored-by: Francis (Feng) Liu Co-authored-by: makuing <71718183+makuing@users.noreply.github.com> Co-authored-by: Lingkai Shen Co-authored-by: Jared Co-authored-by: Joakim Tangnes <10198932+Zarux@users.noreply.github.com> Co-authored-by: Laurens Knoll <3205006+laurensknoll@users.noreply.github.com> Co-authored-by: Rustem Bekmukhametov Co-authored-by: xuchenma <67921399+xuchenma@users.noreply.github.com> Co-authored-by: Esha Goel Co-authored-by: Filip 'Cherit' Szóstak Co-authored-by: Benjamin Kaplan <58792807+bskaplan@users.noreply.github.com> Co-authored-by: jinyangtang <147192056+jinyangtang@users.noreply.github.com> Co-authored-by: Naitian Liu <83430653+naitianliu-google@users.noreply.github.com> Co-authored-by: Sneha Prasad <32434989+snpd25@users.noreply.github.com> Co-authored-by: Sneha Prasad Co-authored-by: rishamchokshi <44479344+rishamchokshi@users.noreply.github.com> Co-authored-by: kautikdk <144651627+kautikdk@users.noreply.github.com> Co-authored-by: kangy-google Co-authored-by: Salome Papiashvili Co-authored-by: Pavel Salnikov <90701144+pavel-salnikov@users.noreply.github.com> Co-authored-by: Michael R. Torres <6692889+micrictor@users.noreply.github.com> Co-authored-by: Esha Goel Co-authored-by: Luca Prete Co-authored-by: Luca Prete Co-authored-by: gurankitt <161278508+gurankitt@users.noreply.github.com> Co-authored-by: Antoni Zawodny Co-authored-by: praseedhaPK <161299686+praseedhaPK@users.noreply.github.com> Co-authored-by: Charles Leon Co-authored-by: Charles Leon Co-authored-by: Andrew Grasso Co-authored-by: Krishnan Gopal Co-authored-by: shijeesh-ns <92889532+shijeesh-ns@users.noreply.github.com> Co-authored-by: kumailkermalli-datatonic <108349674+kumailkermalli-datatonic@users.noreply.github.com> Co-authored-by: Xuran Liu Co-authored-by: Rostislav Bobrovsky Co-authored-by: Ali Ebrahim Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: roop2 <161707562+roop2@users.noreply.github.com> Co-authored-by: fayssalmartanigcp <73672393+fayssalmartanigcp@users.noreply.github.com> Co-authored-by: Patrick Costello Co-authored-by: Daniel Randell Co-authored-by: joelkattapuram <46967875+joelkattapuram@users.noreply.github.com> Co-authored-by: Harrison Powers Co-authored-by: liferoad Co-authored-by: Pawel Jasinski <56267784+pawelJas@users.noreply.github.com> Co-authored-by: Rohit Jangid Co-authored-by: krishnangopal1810 <156300588+krishnangopal1810@users.noreply.github.com> Co-authored-by: Jashan Sudan Co-authored-by: Aleks Vujić Co-authored-by: Ilia Lazebnik Co-authored-by: Srevin Saju [upstream:8821b6ce8241ddfd68707af219c52493dcce5d34] Signed-off-by: Modular Magician --- .../backing_file.tf | 15 ++++ .../main.tf | 31 ++++++++ .../motd | 7 ++ .../tutorial.md | 79 +++++++++++++++++++ integrations_client_advance/backing_file.tf | 15 ++++ integrations_client_advance/main.tf | 34 ++++++++ integrations_client_advance/motd | 7 ++ integrations_client_advance/tutorial.md | 79 +++++++++++++++++++ integrations_client_basic/backing_file.tf | 15 ++++ integrations_client_basic/main.tf | 3 + integrations_client_basic/motd | 7 ++ integrations_client_basic/tutorial.md | 79 +++++++++++++++++++ .../backing_file.tf | 15 ++++ pubsub_topic_ingestion_kinesis/main.tf | 13 +++ pubsub_topic_ingestion_kinesis/motd | 7 ++ pubsub_topic_ingestion_kinesis/tutorial.md | 79 +++++++++++++++++++ 16 files changed, 485 insertions(+) create mode 100644 dataproc_metastore_service_scheduled_backup/backing_file.tf create mode 100644 dataproc_metastore_service_scheduled_backup/main.tf create mode 100644 dataproc_metastore_service_scheduled_backup/motd create mode 100644 dataproc_metastore_service_scheduled_backup/tutorial.md create mode 100644 integrations_client_advance/backing_file.tf create mode 100644 integrations_client_advance/main.tf create mode 100644 integrations_client_advance/motd create mode 100644 integrations_client_advance/tutorial.md create mode 100644 integrations_client_basic/backing_file.tf create mode 100644 integrations_client_basic/main.tf create mode 100644 integrations_client_basic/motd create mode 100644 integrations_client_basic/tutorial.md create mode 100644 pubsub_topic_ingestion_kinesis/backing_file.tf create mode 100644 pubsub_topic_ingestion_kinesis/main.tf create mode 100644 pubsub_topic_ingestion_kinesis/motd create mode 100644 pubsub_topic_ingestion_kinesis/tutorial.md diff --git a/dataproc_metastore_service_scheduled_backup/backing_file.tf b/dataproc_metastore_service_scheduled_backup/backing_file.tf new file mode 100644 index 00000000..c60b1199 --- /dev/null +++ b/dataproc_metastore_service_scheduled_backup/backing_file.tf @@ -0,0 +1,15 @@ +# This file has some scaffolding to make sure that names are unique and that +# a region and zone are selected when you try to create your Terraform resources. + +locals { + name_suffix = "${random_pet.suffix.id}" +} + +resource "random_pet" "suffix" { + length = 2 +} + +provider "google" { + region = "us-central1" + zone = "us-central1-c" +} diff --git a/dataproc_metastore_service_scheduled_backup/main.tf b/dataproc_metastore_service_scheduled_backup/main.tf new file mode 100644 index 00000000..ebba1289 --- /dev/null +++ b/dataproc_metastore_service_scheduled_backup/main.tf @@ -0,0 +1,31 @@ +resource "google_dataproc_metastore_service" "backup" { + service_id = "backup-${local.name_suffix}" + location = "us-central1" + port = 9080 + tier = "DEVELOPER" + + maintenance_window { + hour_of_day = 2 + day_of_week = "SUNDAY" + } + + hive_metastore_config { + version = "2.3.6" + } + + scheduled_backup { + enabled = true + cron_schedule = "0 0 * * *" + time_zone = "UTC" + backup_location = "gs://${google_storage_bucket.bucket.name}" + } + + labels = { + env = "test" + } +} + +resource "google_storage_bucket" "bucket" { + name = "backup-${local.name_suffix}" + location = "us-central1" +} diff --git a/dataproc_metastore_service_scheduled_backup/motd b/dataproc_metastore_service_scheduled_backup/motd new file mode 100644 index 00000000..45a906e8 --- /dev/null +++ b/dataproc_metastore_service_scheduled_backup/motd @@ -0,0 +1,7 @@ +=== + +These examples use real resources that will be billed to the +Google Cloud Platform project you use - so make sure that you +run "terraform destroy" before quitting! + +=== diff --git a/dataproc_metastore_service_scheduled_backup/tutorial.md b/dataproc_metastore_service_scheduled_backup/tutorial.md new file mode 100644 index 00000000..9ac7d909 --- /dev/null +++ b/dataproc_metastore_service_scheduled_backup/tutorial.md @@ -0,0 +1,79 @@ +# Dataproc Metastore Service Scheduled Backup - Terraform + +## Setup + + + +Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform. + + + +Terraform provisions real GCP resources, so anything you create in this session will be billed against this project. + +## Terraforming! + +Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command +to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up +the project name from the environment variable. + +```bash +export GOOGLE_CLOUD_PROJECT={{project-id}} +``` + +After that, let's get Terraform started. Run the following to pull in the providers. + +```bash +terraform init +``` + +With the providers downloaded and a project set, you're ready to use Terraform. Go ahead! + +```bash +terraform apply +``` + +Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan. + +```bash +yes +``` + + +## Post-Apply + +### Editing your config + +Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed. + +```bash +terraform plan +``` + +So let's make a change! Try editing a number, or appending a value to the name in the editor. Then, +run a 'plan' again. + +```bash +terraform plan +``` + +Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes +at the 'yes' prompt. + +```bash +terraform apply +``` + +```bash +yes +``` + +## Cleanup + +Run the following to remove the resources Terraform provisioned: + +```bash +terraform destroy +``` +```bash +yes +``` diff --git a/integrations_client_advance/backing_file.tf b/integrations_client_advance/backing_file.tf new file mode 100644 index 00000000..c60b1199 --- /dev/null +++ b/integrations_client_advance/backing_file.tf @@ -0,0 +1,15 @@ +# This file has some scaffolding to make sure that names are unique and that +# a region and zone are selected when you try to create your Terraform resources. + +locals { + name_suffix = "${random_pet.suffix.id}" +} + +resource "random_pet" "suffix" { + length = 2 +} + +provider "google" { + region = "us-central1" + zone = "us-central1-c" +} diff --git a/integrations_client_advance/main.tf b/integrations_client_advance/main.tf new file mode 100644 index 00000000..e4b8e1e8 --- /dev/null +++ b/integrations_client_advance/main.tf @@ -0,0 +1,34 @@ +data "google_project" "test_project" { +} + +resource "google_kms_key_ring" "keyring" { + name = "my-keyring-${local.name_suffix}" + location = "us-central1" +} + +resource "google_kms_crypto_key" "cryptokey" { + name = "crypto-key-example" + key_ring = google_kms_key_ring.keyring.id + rotation_period = "7776000s" + depends_on = [google_kms_key_ring.keyring] +} + +resource "google_kms_crypto_key_version" "test_key" { + crypto_key = google_kms_crypto_key.cryptokey.id + depends_on = [google_kms_crypto_key.cryptokey] +} + +resource "google_integrations_client" "example" { + location = "us-central1" + create_sample_workflows = true + provision_gmek = true + run_as_service_account = "radndom-service-account" + cloud_kms_config { + kms_location = "us-central1" + kms_ring = google_kms_key_ring.keyring.id + key = google_kms_crypto_key.cryptokey.id + key_version = google_kms_crypto_key_version.test_key.id + kms_project_id = data.google_project.test_project.id + } + depends_on = [google_kms_crypto_key_version.test_key] +} diff --git a/integrations_client_advance/motd b/integrations_client_advance/motd new file mode 100644 index 00000000..45a906e8 --- /dev/null +++ b/integrations_client_advance/motd @@ -0,0 +1,7 @@ +=== + +These examples use real resources that will be billed to the +Google Cloud Platform project you use - so make sure that you +run "terraform destroy" before quitting! + +=== diff --git a/integrations_client_advance/tutorial.md b/integrations_client_advance/tutorial.md new file mode 100644 index 00000000..021b9f0d --- /dev/null +++ b/integrations_client_advance/tutorial.md @@ -0,0 +1,79 @@ +# Integrations Client Advance - Terraform + +## Setup + + + +Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform. + + + +Terraform provisions real GCP resources, so anything you create in this session will be billed against this project. + +## Terraforming! + +Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command +to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up +the project name from the environment variable. + +```bash +export GOOGLE_CLOUD_PROJECT={{project-id}} +``` + +After that, let's get Terraform started. Run the following to pull in the providers. + +```bash +terraform init +``` + +With the providers downloaded and a project set, you're ready to use Terraform. Go ahead! + +```bash +terraform apply +``` + +Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan. + +```bash +yes +``` + + +## Post-Apply + +### Editing your config + +Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed. + +```bash +terraform plan +``` + +So let's make a change! Try editing a number, or appending a value to the name in the editor. Then, +run a 'plan' again. + +```bash +terraform plan +``` + +Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes +at the 'yes' prompt. + +```bash +terraform apply +``` + +```bash +yes +``` + +## Cleanup + +Run the following to remove the resources Terraform provisioned: + +```bash +terraform destroy +``` +```bash +yes +``` diff --git a/integrations_client_basic/backing_file.tf b/integrations_client_basic/backing_file.tf new file mode 100644 index 00000000..c60b1199 --- /dev/null +++ b/integrations_client_basic/backing_file.tf @@ -0,0 +1,15 @@ +# This file has some scaffolding to make sure that names are unique and that +# a region and zone are selected when you try to create your Terraform resources. + +locals { + name_suffix = "${random_pet.suffix.id}" +} + +resource "random_pet" "suffix" { + length = 2 +} + +provider "google" { + region = "us-central1" + zone = "us-central1-c" +} diff --git a/integrations_client_basic/main.tf b/integrations_client_basic/main.tf new file mode 100644 index 00000000..3f71ec00 --- /dev/null +++ b/integrations_client_basic/main.tf @@ -0,0 +1,3 @@ +resource "google_integrations_client" "example" { + location = "us-central1" +} diff --git a/integrations_client_basic/motd b/integrations_client_basic/motd new file mode 100644 index 00000000..45a906e8 --- /dev/null +++ b/integrations_client_basic/motd @@ -0,0 +1,7 @@ +=== + +These examples use real resources that will be billed to the +Google Cloud Platform project you use - so make sure that you +run "terraform destroy" before quitting! + +=== diff --git a/integrations_client_basic/tutorial.md b/integrations_client_basic/tutorial.md new file mode 100644 index 00000000..2b049c34 --- /dev/null +++ b/integrations_client_basic/tutorial.md @@ -0,0 +1,79 @@ +# Integrations Client Basic - Terraform + +## Setup + + + +Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform. + + + +Terraform provisions real GCP resources, so anything you create in this session will be billed against this project. + +## Terraforming! + +Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command +to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up +the project name from the environment variable. + +```bash +export GOOGLE_CLOUD_PROJECT={{project-id}} +``` + +After that, let's get Terraform started. Run the following to pull in the providers. + +```bash +terraform init +``` + +With the providers downloaded and a project set, you're ready to use Terraform. Go ahead! + +```bash +terraform apply +``` + +Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan. + +```bash +yes +``` + + +## Post-Apply + +### Editing your config + +Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed. + +```bash +terraform plan +``` + +So let's make a change! Try editing a number, or appending a value to the name in the editor. Then, +run a 'plan' again. + +```bash +terraform plan +``` + +Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes +at the 'yes' prompt. + +```bash +terraform apply +``` + +```bash +yes +``` + +## Cleanup + +Run the following to remove the resources Terraform provisioned: + +```bash +terraform destroy +``` +```bash +yes +``` diff --git a/pubsub_topic_ingestion_kinesis/backing_file.tf b/pubsub_topic_ingestion_kinesis/backing_file.tf new file mode 100644 index 00000000..c60b1199 --- /dev/null +++ b/pubsub_topic_ingestion_kinesis/backing_file.tf @@ -0,0 +1,15 @@ +# This file has some scaffolding to make sure that names are unique and that +# a region and zone are selected when you try to create your Terraform resources. + +locals { + name_suffix = "${random_pet.suffix.id}" +} + +resource "random_pet" "suffix" { + length = 2 +} + +provider "google" { + region = "us-central1" + zone = "us-central1-c" +} diff --git a/pubsub_topic_ingestion_kinesis/main.tf b/pubsub_topic_ingestion_kinesis/main.tf new file mode 100644 index 00000000..5206dfe8 --- /dev/null +++ b/pubsub_topic_ingestion_kinesis/main.tf @@ -0,0 +1,13 @@ +resource "google_pubsub_topic" "example" { + name = "example-topic-${local.name_suffix}" + + # Outside of automated terraform-provider-google CI tests, these values must be of actual AWS resources for the test to pass. + ingestion_data_source_settings { + aws_kinesis { + stream_arn = "arn:aws:kinesis:us-west-2:111111111111:stream/fake-stream-name" + consumer_arn = "arn:aws:kinesis:us-west-2:111111111111:stream/fake-stream-name/consumer/consumer-1:1111111111" + aws_role_arn = "arn:aws:iam::111111111111:role/fake-role-name" + gcp_service_account = "fake-service-account@fake-gcp-project.iam.gserviceaccount.com" + } + } +} diff --git a/pubsub_topic_ingestion_kinesis/motd b/pubsub_topic_ingestion_kinesis/motd new file mode 100644 index 00000000..45a906e8 --- /dev/null +++ b/pubsub_topic_ingestion_kinesis/motd @@ -0,0 +1,7 @@ +=== + +These examples use real resources that will be billed to the +Google Cloud Platform project you use - so make sure that you +run "terraform destroy" before quitting! + +=== diff --git a/pubsub_topic_ingestion_kinesis/tutorial.md b/pubsub_topic_ingestion_kinesis/tutorial.md new file mode 100644 index 00000000..7368380a --- /dev/null +++ b/pubsub_topic_ingestion_kinesis/tutorial.md @@ -0,0 +1,79 @@ +# Pubsub Topic Ingestion Kinesis - Terraform + +## Setup + + + +Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform. + + + +Terraform provisions real GCP resources, so anything you create in this session will be billed against this project. + +## Terraforming! + +Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command +to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up +the project name from the environment variable. + +```bash +export GOOGLE_CLOUD_PROJECT={{project-id}} +``` + +After that, let's get Terraform started. Run the following to pull in the providers. + +```bash +terraform init +``` + +With the providers downloaded and a project set, you're ready to use Terraform. Go ahead! + +```bash +terraform apply +``` + +Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan. + +```bash +yes +``` + + +## Post-Apply + +### Editing your config + +Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed. + +```bash +terraform plan +``` + +So let's make a change! Try editing a number, or appending a value to the name in the editor. Then, +run a 'plan' again. + +```bash +terraform plan +``` + +Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes +at the 'yes' prompt. + +```bash +terraform apply +``` + +```bash +yes +``` + +## Cleanup + +Run the following to remove the resources Terraform provisioned: + +```bash +terraform destroy +``` +```bash +yes +```