Skip to content

Commit

Permalink
Sync main feature branch provider functions (hashicorp#10138)
Browse files Browse the repository at this point in the history
* fixes: permadiff issue if event trigger region is not specified (hashicorp#9989)

* fixes: permadiff issue if event trigger region is not specified

- fixes hashicorp#17161

* chore: Updated cloudfunction2 test with no explicit trigger_region

* Fix GKE front matter, bad field docs (hashicorp#10018)

* Fix GKE front matter, bad field docs

* Update container_cluster.html.markdown

* added default value for minPortsPerVM field at "google_compute_router_nat" (hashicorp#9712)

* added default value for statis allocation

* setting the default from API

* add newline removal bash command in guidelines (hashicorp#9734)

* Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (hashicorp#10009)

* Adding Data Source Forwarding Rules (hashicorp#10004)

* Fix broken terraform datasource google_compute_machine_types example (hashicorp#10020)

* Add check for Environment proto field before accessing in Dataflow provider (hashicorp#10016)

* Add comments to skipped sweeper (hashicorp#10023)

* Add comments for skipped sweepers (hashicorp#10024)

* C3 version schema (hashicorp#9986)

* add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress

* make build number optional

* regroup regex, cleaner comparison of versions

* correction

* Add Resource Manager Tags support to 'google_container_cluster' (hashicorp#9531)

* resourceManagerTags added to Cluster Node Config schema

* update beta tag

* add cluster and node proto tests

* add expand and flatten proto

* removed beta tag

* added to documentation

* added resource manager tags to auto pilot

* migrating resourceManagerTags tests

* migrating node_pools test

* migrating additional tests

* minor fixes

* fixing tests

* add in-place update support

* fixed tests

* fixed annotations

* validated clusters and node pools tests. Isolated node pool auto config

* isolated resource manager tags from docs

* fixed permission issue

* fixed spaces

* fixed non determinism on tag keys

* removed auto_pilot rmts

* fixed time_sleep

* add depends_on to IAM policies

* Add volume replication support for Google Cloud NetApp Volumes (hashicorp#9816)

* Initial replication commit

* Cleanup work

- Renamed a lot of files to make clear which resource the belong to
- Updated documentation for resource fields
- Renamed a few resource fields and changed some types
- Disabled the custom code for now. Needs to be discussed first

* Update example file

* Updated example file

* Major updates

- Reorganisation of block
- Reorganisation of fields to match API documentation
- Updated example parameters
- Added missing API fields
- Improved descriptions
-

* For replication deletion, stop replication first

* Add support for deleting destination volume on replication delete

* Make volumes deletable in presence of snapshots.

This change will be PRed for volume resource independently. Adding it here while it is not in main.

* Improving debug error message

* yaml check and format fix

* Add wait for mirror to initialize.

Required to run destroy shortly after create.

* Wait on destroy, not on create

* Make deleting a replication more robust

- doc improvements
- started to implement stop/resume. More work required.
- renamed a few files to better reflect what they are good for

* adding support for stop/resume

* yamlformat and lint

* Add force delete to delete volumes with nested snapshots

* resource test first version

* More changes to make tests solid

- Introduced new parameter to wait for mirror_status==MIRRORED
- more mirror state reconciliation

* Test updates

* few cleanups

* Make virtual field verifies happy

* Minor test improvements

* More fine tuning

- Remove merge conflict in volume.yaml
- make generated test work
- make output field work
- ignore_read for virtual fields

* Resource name change as suggested by @slevenick

* Remove snapshot code block and fix typo

* Detect manual stop/resume actions

* Remove ignore_read for deletion_policy

* - Made destinationVolumeParameters immutable. It still requires ignore_read.
- removed ignore_read from virtual_fields

* destinationVolumeParameters are only evaluated at create. Make the immutable.

* Name cleanups and comment improvements

* removed comment

Co-authored-by: Shuya Ma <[email protected]>

* tabs to spaces in resource block

Co-authored-by: Shuya Ma <[email protected]>

* Updates to address review comments

- make wait_for_mirror also work for stop/resume, additionally to create
- convert tabs in test resource blocks to spaces
- fix typos

* Rewording of comments

Co-authored-by: Shuya Ma <[email protected]>

---------

Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>

* Ensured that beta runs in TeamCity use only beta paths (hashicorp#10025)

* Ensured that beta runs in TeamCity use only beta paths

* Added tests for sweeper package path

* Adds description for default value of `cpuidle` field (hashicorp#10005)

* add description

* remove tail space

* Add location field in DNS authorization resource. (hashicorp#9968)

* Add location field in DNS authorization resource

* Fixed the provider version of the updated config

---------

Co-authored-by: Hamza Hassan <[email protected]>

* Add discoveryengine service (hashicorp#10017)

* Add validations for Composer 2/3 only fields (hashicorp#9917)

* block upgrade to composer 3

* make isComposer3 more generic, correct imageVersionChangeValidationFunc

* added validation for Composer 2/3 specific fields

* add tests for validation

* add checks in flattenComposerEnvironmentConfig

* Update attributes of fields not used in Composer 3

* make customizeDiff functions beta only

* remove Computed from gke_cluster

* remove Optional instead of Computed

* add envCfg.PrivateEnvironmentConfig is nil check

* modify isComposer3 to take string

* minor correction to avoid merge conflicts

* Made TestAccContainerCluster_withGatewayApiConfig error regex more forgiving (hashicorp#10034)

Fixed hashicorp#17319

* Add ceritificateManagerCertificates field to ComputeRegionTargetHttpsProxy resource (hashicorp#10011)

Co-authored-by: Hamza Hassan <[email protected]>

* Added missing quote marks to TEST_PREFIX (hashicorp#10035)

Without this, | gets interpreted as a pipe and the test count always becomes 0 if there are multiple prefixes supplied

* Promote enable_confidential_storage from beta to GA (hashicorp#9993)

* Add IAM resources for Cloud Deploy Target (hashicorp#9927)

* Add IAM resources for Cloud Deploy Target

* update

* add back import_format

* Add comments to skipped sweepers (hashicorp#10028)

* Add sweepers for gkebackupPlan and restorePlan (hashicorp#10026)

* add `remote_function_options` to bigquery_routine (hashicorp#9893)

* Add dataGovernanceType and remoteFunctionOptions to bigquery_routine

* add function-sources.zip to biguquery fixtures

* fix resource names in TestAccBigQueryRoutine

* add bigquery routine remote function example

* Adding documentaiton build_worker_pool (hashicorp#10022)

* docs: fix import example in google_project_iam (hashicorp#10007)

Fix the example in the `google_project_iam` documentation.

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_project_iam#import

* ScottSuarez not on vacation anymore (hashicorp#10052)

* I'm on vaction :)

* ScottSuarez not vaction anymore

* ScottSuarez not vaction anymore

* Promote (R)IGM.AllInstancesConfig to V1 (hashicorp#9964)

* Promote metric settings in compute region autoscaler to GA (hashicorp#10045)

* Fixed flakiness in TestAccDataPipelinePipeline_dataPipelinePipelineExample (hashicorp#10040)

* Retry delete network step while creating a google project. (hashicorp#10046)

* Adding the namespace_labels field to the GKE Hub Scope resource (hashicorp#9972)

* initial commit for scope-level namespace labels

* Add validation exceptions for the  field of the GKEHub Scope and Namespace resources to be of type

* Undoing unnecessary changes

* Fixing the type of Scope namespace_labels field

* Support connectivity fields for Composer 3 (hashicorp#9889)

* add composer_network_attachment

* indicate conflicting configs

* commas

* no need for bidirectional conflict definition (generates double errors)

* protect nit PrivateClusterConfig

* for optimizing error messages about conflicts

* add 2 step update for composer_network_attachment

* make composer_network_attachment available in beta only

* add two step update for network and subnetwork

* corrections in 2 phase update for network/subnetwork

* remove composer3 check(CustomizeDiff will solve this), filter api error, add tests (unsetting netwok/subnetwork not working)

* added ForceNewIf fot network/subnetwork, problem with unsetting these fields remains

* add docs for composer_network_attachment

* add test for network attachment

* ignore non empty plan in network attachment test

* add networkAttachment update and conflicting fields tests

* add ComputedIf for network, change isComposer3

* minor corrections

* remove computedIf

* filter equivalent values of network/subnetwork in ForceNewIf

* simplify ResourceConditionFunc, add beta/ga version conditions

* typo

* more general comparison of network references

* use tpgresource.CompareSelfLinkRelativePaths instead of custom function

* modify isComposer3 to avoid merge conflicts later.

* removing this since documentation is handled in other PR and to avoid conflicts while merging.

* replace ExpectNonEmptyPlan with lifecycle.ignore_changes

* add testcase for changing network attachment to network and subnetwork

* add third step to TestAccComposerEnvironmentComposer3_updateWithNetworkAndSubnetwork

* modify tests to use different network for attachment

* remove unused constant

* remove ExpectNonEmptyPlan (already replaced with lifecycle.ignore_changes)

* Fix resource name to have sweepable prefix (hashicorp#9984)

* doc(spanner): update documents to have example for IAM conditions use with google spanner database (hashicorp#10049)

* Promote Cloud Deploy Automation to GA (hashicorp#10043)

* Add Cloud Deploy Automation

* Attempt to solve the test errors based on https://yaqs.corp.google.com/eng/q/7753655943518224384

* Add update test for the new automation resource

* fix lint errors

* fix lint errors

* fix errors in GA provider test

* add the missing test

* add a full test

* Attempt to fix the acceptance test errors

* fix a lint error

* mark labels as default_from_api:true

* fix advance rollout rule definition

* Mark service_account as 'ignore_read: true'

* Fix test errors

* Resolve review comments

* Format test file

* promote google_clouddeploy_automation to GA

* promote google_clouddeploy_automation to GA

* fix lint errors

* Add cloudquota service (hashicorp#10063)

* TeamCity: Add validating GHAs that check Kotlin files listing services (hashicorp#9999)

* services diff gha

* add needs

* use setup-go@v3

* add artifacts

* remove checkout

* update directory

* update common-copy.yaml

* update common-copy.yaml

* remove data print

* add pull-request check on teamcitydiff

* update paths in GHA

* add exit code in diff_check

* update exit output

* add exit code diff_check.go

* services_beta diff check support

* beta services support args

* add arg for services kt file

* services_ga.kt in gha

* move diff_check to tools folder

* remove go command

* provide path for artifacts download

* add cd in Build Provider

* update diff_check_beta.yml

* fix paths

* yml typo beta

* directory testing

* directory testing

* fix directory issues with commands

* refactor tools/teamcity-diff-check/main.go

* directory check

* typo

* remove common copy

* generate both providers into one gha

* type artifacts name

* add -o flag into artifacts download

* use merge multiple artifacts

* use artifact@v4

* use mmv1 directory for services file

* remove .zip

* output missing services from diff

* find artifacts folder

* use beta provider as only artifact

* include google ga and beta in gha

* artifact name

* -o flag in artifacts

* output stdout from go list command

* diff test

* output beta main.go

* remove err check

* remove services print / refactor for final review

* output cleanup

* uncomment go list err

* regex assert

* add check for new services in PR / set googleServices as groundtruth in serviceDifference function

* add actions/checkout

* logic fix

* add needs: check-pr

* add mmv1/products for pull-request event

* set force depth 0

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* Update .github/workflows/teamcity-services-diff-check.yml

Co-authored-by: Sarah French <[email protected]>

* use text file for services, add if statement in gha

* directory typo

* typo

* typo in grep

* quotations on if statement

* unquote GITHUB_OUTPUT

* add quotes on output.services

* if test

* echo output

* github_output error fix

* services.outputs

* proper if statement syntax gpush

* invalid 0 format

* add missing assignment

* fix openfile error

* remove types in pull_request trigger

* send txt files to correct directory

* check services directory for ls command

* add weekly workflow for teamcity diff check

* change name for weekly check

---------

Co-authored-by: Sarah French <[email protected]>

* Fixed `runtime` in  app engine tests (hashicorp#10047)

* Fix google_notebooks_instance labels not being able to be updated (hashicorp#9933)

* add labels to runtime update test

* Add labels to ImportStateVerifyIgnore

* fix indentation

* Add IAM resources for Cloud Deploy Custom Target Type (hashicorp#10051)

* Add IAM resources for Cloud Deploy Custom Target Type

* Add base url override for Cloud Deploy Custom Target Type IAM policy

* Update import format for IAM policy and primary resource name to examples for IAM generated tests

* Remove unneeded cache step in TeamCity-related GHAs, lint GHA files (hashicorp#10075)

* Remove unneeded GHA steps - do we need to cache here?

* Whitespace

* Test change with fake new service

* Resolve build error in fake service

* Re-order GHA steps and add name

* Add name to checkout step

* Add comments, remove extra whitespace

* Remove fake new service

* Add output to identifiers, check for project as well as projectsId (hashicorp#10081)

* Add output to identifiers, check for project as well as projectsId

* Correct identifier behavior

* Rubocop

* remove nickelliot/zli82016 from vacation reviewers (hashicorp#10080)

* Update membership.go

* Update membership.go

* Update enrolled_teams.yml (hashicorp#10082)

* Fix resource_bigquery_dataset ID validation (hashicorp#10027)

* fix resource_bigquery_dataset ID validation

* skip the new validation tests for VCR

* update dataset IDs to be valid in terraform-google-conversion example files

* update more dataset IDs in the CAI files

* Prevent duplicate entries in vcr results (hashicorp#10068)

* Prevent duplicate entries in vcr results

* Create empty map

* Only collect result from current test

* Split github tokens (hashicorp#9988)

* Split github tokens

* Update .ci/gcb-generate-diffs-new.yml

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Remove redundant downstreams token

* Make diff processor use new token

* Update path to markdown file

* Replace GITHUB_TOKEN

* Make github tokens optional for generate downstream

* Allow either github token to be used

* Replace GITHUB_TOKEN

* Move environment variable lookup out of constructor

* Update .ci/magician/vcr/tester.go

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Add downstream token

* Make request reviewer use GITHUB_TOKEN and tgc integration use GITHUB_TOKEN_CLASSIC

* Apply suggestions from code review

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

---------

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Instance lifecycle policy default action on failure implementation (hashicorp#10050)

* Add wait in `TestAccAppEngineStandardAppVersion_update` test to avoid 'API has not been used in project' error (hashicorp#10076)

* Add wait in acc test to avoid 'API has not been used in project' error

* Add external provider `time`

* Fix defect when referencing google_project resource

* Set GITHUB_TOKEN environment variable for hub (hashicorp#10091)

* fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (hashicorp#10089)

* make go-converted YAML compatible with go compiler (hashicorp#10033)

Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>

* Revert "Promote metric settings in compute region autoscaler to GA" (hashicorp#10095)

This reverts commit c3ce700.

* Add type field to DNS authorization reosurce (hashicorp#10030)

* Add type field to DNS authorization reosurce

* Add an example for regional DNS authorization

* Add an example for regional certs using regional DNS auth

* Fix lint errors

* Fix typo in the enum values

* Add type field in regional dns auth example

---------

Co-authored-by: Hamza Hassan <[email protected]>

* TeamCity: Refactor config tests (hashicorp#9956)

* Add test util for locating a subproject 2 layers deep

* Refactor tests to use new helper function

* Consolidate testing of service sweeper builds, refactor `getSubProject` to return non-nullable value

* Refactor new sweeper tests

* Update `teamcity-diff-check` script and GHAs to have more explicit calls to action (hashicorp#10098)

* TeamCity: enable ad hoc triggers of teamcity-diff-check GHA (hashicorp#10099)

* Revert "fix forceNew on master_ipv4_cidr_block and private_endpoint_subnetwork (hashicorp#10089)" (hashicorp#10096)

* subnetwork and service_account_email params described (17211) (hashicorp#10102)

* subnetwork and service_account_email params described (17211)

* Doc adjustment

* Update provider name in various spots (hashicorp#10092)

* Go compiler skeleton and early terraform.rb provider code (hashicorp#10104)

* Remove provider-related caches to address cache issues (hashicorp#10097)

We keep exceeding the 10GB cache limit for the repo, resulting in workflows stalling for 30min on caching steps and then timing out.

* Fix hashicorp#17388 (only run tests in beta) (hashicorp#10093)

* Revert "Fix hashicorp#17388 (only run tests in beta)" (hashicorp#10111)

This reverts commit ae819d0.

* Add support for string --> object map for DCL resources (hashicorp#10039)

* Add support for string --> object map for DCL resources

* Fix whitespace in template

* Add test, need updated DCL to work

* Add override for key name

* Both overrides

* Update docs, prepend key for maps

* Update bundle descriptions

* Comment update

* fix permadiff by reading empty docker_config field (hashicorp#10113)

* Import package golang.org/x/exp/slices in MMv1 go compiler (hashicorp#10108)

* fix(kms): certificate chain type to array of strings (hashicorp#9582)

* Bump GHA actions to navigate deprecations of NodeJS 12 and 16 (hashicorp#10110)

* Update all uses of `actions/cache` to v3

* Update all uses of `actions/checkout` to v4

* Update all uses of `actions/upload-artifact` to v3.1.0

* Update all uses of `actions/setup-go` to v4.0.0

* Update all uses of `ruby/setup-ruby` to v1.160.0 from v1.144.2

[v1.160.0](https://github.com/ruby/setup-ruby/releases/tag/v1.160.0) is the version after they upgraded to NodeJS 20 where they fixed some issues from the upgrade from nodejs 16->20

The previous version, ruby/setup-ruby@ec02537, is v1.144.2

* Address dependency issues in TestAccFirestoreField_* tests (hashicorp#9957)

* Add additional wait in TestAccFirestoreField_* tests

* Boost wait in test to 6 minutes

* Add dependency between database and service to control delete order

* Update dependency to explicitly include project

* Make firestore fields be removed from state when they're 'deleted'

* Add `destroy_duration`

* Remove from state after log line that uses id value

* Update destory check to accept a 403 as valid

* Remove unneeded changes in PR

* Remove call to SetId

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource (hashicorp#9895)

* add network_url attribute in consumer_accept_list block of google_compute_service_attachment resource

* Bugfix: Use SelfLinkRelativePath check to prevent false positive resource changes

* Update the GCF resource to reflect transition from Container Registry to Artifact Registry (hashicorp#10058)

* Handwrite sweepers for networkConnectivity hubs and spokes (hashicorp#10069)

* Fix wrong variable in override logic (hashicorp#10125)

* Make missing test detector reader into a separate module (hashicorp#10115)

* Use old github token if new tokens are not available (hashicorp#10114)

* Use old github token if new tokens are not available

* Add lookup function for github token or fallback

* Add fall back

* Add fallback

* Add test for PATCH environment type (hashicorp#10131)

* Add new resource for Application for Apphub (hashicorp#10079)

* Add new resource for Application for Apphub

* Add new resource for Application for Apphub

* Enable Apphub API in test cases

* Fix precheck error

* Fix precheck error

* Resolve comments

* Fix lint error

* Fix examples

* Remove apphub from teamcity config

This causes PR build failures, I'll add it later

---------

Co-authored-by: Sam Levenick <[email protected]>

* Add apphub (hashicorp#10133)

Adds AppHub to APIs to activate

* Allow sending empty app_engine and serverless google_compute_region_network_endpoint_group (hashicorp#10031)

* Support service-level min instances in Cloud Run v2 services. (hashicorp#10083)

* feat: Add CMEK support for Firestore database in Beta provider (hashicorp#10044)

* Modify database.yaml to add cmek related fields

* Add two examples for firestore CMEK databases for testing

* Resolve trailing space

* Update documentation for kmsKeyName field

* Resolve trailing space

* Make field immutable

* Update field documentation

* Update field description

* Allow vcenter_network to be set (hashicorp#9946)

* Allow vcenter_network to be set

* set default from api

* Sort resources and set in product in go compiler (hashicorp#10135)

* fix failing posture test (hashicorp#10086)

* fix failing posture test

* modify posture name

---------

Co-authored-by: Sneha Prasad <[email protected]>

* Create support for KMS Ekmconnection resource (hashicorp#10094)

* Fixes two lifecycle rules with different no_age value always generates change. (hashicorp#10137)

* TeamCity: Add project for testing the provider functions feature branch (hashicorp#10088)

* Add ability to use non-default versions of Terraform in TeamCity builds

* Add function to enable making build configs for single packages at a time

* Add new sub project that contains 2 builds for testing provider functions

the 2 builds:
1) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google repo
2) only pulls code from the feature branch on the downstream hashicorp/terraform-provider-google-beta repo

These builds both use an alpha release of TF 1.8.0

* Add builds for testing auto generated branches in the MM upstream repos

These re-use existing VCR Roots.

* Make the builds that test the `FEATURE-BRANCH-provider-functions branches in the downstream repos run every night at the default time

* Fix defect in 'Download Terraform' build step definition

* Update build step to solve bug

* Update build_configuration_per_package.kt

* Add `ephemeral_directories` to google_workstations_workstation_config (hashicorp#10042)

* Add `ephemeral_directories` to google_workstations_workstation_config

* Remove trailing spaces

* Add a test for ephemeral_directories

* Fix test

* Add a test for source_image

* Fix typo in test

* Remove unnecessary immutable field

* App Check DeviceCheck provider (hashicorp#9978)

* App Check DeviceCheck provider

* Remove minimal example & pattern field

* Add real private keys that are not useful anywhere

* Limit tests to beta

* Documentation (hashicorp#10012)

* duplicate Composer 2 argument reference

* remove fields that are not supported in Composer 3

* move fields that are new in Composer 3 to Composer 3 section only

* make suggested changes, add description if new versioning schema

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown, remove outdated info

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* add notice that composer 3 is not yet released.

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* specify composer version in section links.

* specify composer version in composer 1 documentation links

* add section links in composer 2 argument reference

* add section links in composer 3 argument reference

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

* Update mmv1/third_party/terraform/website/docs/r/composer_environment.html.markdown

Co-authored-by: Pavel Salnikov <[email protected]>

---------

Co-authored-by: Pavel Salnikov <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>

* Promote metric settings in compute region autoscaler to GA (hashicorp#10103)

* Clarified permissions for a few workflows (hashicorp#10124)

* Clarified permissions for a few workflows

* fixed conditions for running repository-documentation workflows

* Bumped actions/checkout to v4

* force downstream generation

* Revert "force downstream generation"

This reverts commit d54857b4dc5767ab8f98543e6a84d25c8be1104a.

* Post statuses earlier (hashicorp#10128)

* Post initial statuses earlier so that we are resilient issues during cloning

* force generation

* Revert "force generation"

This reverts commit 864ec92ec03b75dea021cdb617d86d31112d9140.

* Make TestAccDefaultUniverseDomain_doesNotMatchExplicit independent from test credentials (hashicorp#10140)

---------

Co-authored-by: Deepak Kumar <[email protected]>
Co-authored-by: Riley Karson <[email protected]>
Co-authored-by: Max Portocarrero CI&T <[email protected]>
Co-authored-by: Mauricio Alvarez Leon <[email protected]>
Co-authored-by: Jesse Liddle <[email protected]>
Co-authored-by: Wessel Blokzijl <[email protected]>
Co-authored-by: Jack McCluskey <[email protected]>
Co-authored-by: hao-nan-li <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Oliver Krause <[email protected]>
Co-authored-by: G-NamanGupta <[email protected]>
Co-authored-by: Shuya Ma <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
Co-authored-by: Yanwei Guo <[email protected]>
Co-authored-by: Hamzawy63 <[email protected]>
Co-authored-by: Hamza Hassan <[email protected]>
Co-authored-by: Ryan Oaks <[email protected]>
Co-authored-by: Sneha-at <[email protected]>
Co-authored-by: Shogo Watanabe <[email protected]>
Co-authored-by: Obada Alabbadi <[email protected]>
Co-authored-by: NA2047 <[email protected]>
Co-authored-by: Yuki Watanabe <[email protected]>
Co-authored-by: Scott Suarez <[email protected]>
Co-authored-by: askubis <[email protected]>
Co-authored-by: Pavan Kumar Sunkara <[email protected]>
Co-authored-by: ajaybgunjal1 <[email protected]>
Co-authored-by: Hossein Golestani <[email protected]>
Co-authored-by: rahul2393 <[email protected]>
Co-authored-by: Chris <[email protected]>
Co-authored-by: bcreddy-gcp <[email protected]>
Co-authored-by: Matt Santa <[email protected]>
Co-authored-by: Sam Levenick <[email protected]>
Co-authored-by: Nick Elliot <[email protected]>
Co-authored-by: wj-chen <[email protected]>
Co-authored-by: Thomas Rodgers <[email protected]>
Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
Co-authored-by: yiyinglovecoding <[email protected]>
Co-authored-by: Cameron Thornton <[email protected]>
Co-authored-by: Zhenhua Li <[email protected]>
Co-authored-by: Francis (Feng) Liu <[email protected]>
Co-authored-by: makuing <[email protected]>
Co-authored-by: Lingkai Shen <[email protected]>
Co-authored-by: Jared <[email protected]>
Co-authored-by: Joakim Tangnes <[email protected]>
Co-authored-by: Laurens Knoll <[email protected]>
Co-authored-by: Rustem Bekmukhametov <[email protected]>
Co-authored-by: xuchenma <[email protected]>
Co-authored-by: Esha Goel <[email protected]>
Co-authored-by: Filip 'Cherit' Szóstak <[email protected]>
Co-authored-by: Benjamin Kaplan <[email protected]>
Co-authored-by: jinyangtang <[email protected]>
Co-authored-by: Naitian Liu <[email protected]>
Co-authored-by: Sneha Prasad <[email protected]>
Co-authored-by: Sneha Prasad <[email protected]>
Co-authored-by: rishamchokshi <[email protected]>
Co-authored-by: kautikdk <[email protected]>
Co-authored-by: kangy-google <[email protected]>
Co-authored-by: Salome Papiashvili <[email protected]>
Co-authored-by: Pavel Salnikov <[email protected]>

[upstream:cfe060aa9185bc04e202f2c885aaeadfacd8f24d]

Signed-off-by: Modular Magician <[email protected]>
  • Loading branch information
modular-magician committed Mar 8, 2024
1 parent 816402c commit f8e2462
Show file tree
Hide file tree
Showing 147 changed files with 11,004 additions and 471 deletions.
3 changes: 3 additions & 0 deletions .changelog/10138.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:none

```
11 changes: 10 additions & 1 deletion .teamcity/components/builds/build_configuration_per_package.kt
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ import jetbrains.buildServer.configs.kotlin.sharedResources
import jetbrains.buildServer.configs.kotlin.vcs.GitVcsRoot
import replaceCharsId

// BuildConfigurationsForPackages accepts a map containing details of multiple packages in a provider and returns a list of build configurations for them all.
// Intended to be used in projects where we're testing all packages, e.g. the nightly test projects
fun BuildConfigurationsForPackages(packages: Map<String, Map<String, String>>, providerName: String, parentProjectName: String, vcsRoot: GitVcsRoot, sharedResources: List<String>, environmentVariables: AccTestConfiguration): List<BuildType> {
val list = ArrayList<BuildType>()

Expand All @@ -31,6 +33,13 @@ fun BuildConfigurationsForPackages(packages: Map<String, Map<String, String>>, p
return list
}

// BuildConfigurationForSinglePackage accepts details of a single package in a provider and returns a build configuration for it
// Intended to be used in short-lived projects where we're testing specific packages, e.g. feature branch testing
fun BuildConfigurationForSinglePackage(packageName: String, packagePath: String, packageDisplayName: String, providerName: String, parentProjectName: String, vcsRoot: GitVcsRoot, sharedResources: List<String>, environmentVariables: AccTestConfiguration): BuildType{
val pkg = PackageDetails(packageName, packageDisplayName, providerName, parentProjectName)
return pkg.buildConfiguration(packagePath, vcsRoot, sharedResources, environmentVariables)
}

class PackageDetails(private val packageName: String, private val displayName: String, private val providerName: String, private val parentProjectName: String) {

// buildConfiguration returns a BuildType for a service package
Expand Down Expand Up @@ -102,4 +111,4 @@ class PackageDetails(private val packageName: String, private val displayName: S
var id = "%s_%s_PACKAGE_%s".format(this.parentProjectName, this.providerName, this.packageName)
return replaceCharsId(id)
}
}
}
14 changes: 11 additions & 3 deletions .teamcity/components/builds/build_parameters.kt
Original file line number Diff line number Diff line change
Expand Up @@ -252,12 +252,20 @@ fun ParametrizedWithType.readOnlySettings() {
}

// ParametrizedWithType.terraformCoreBinaryTesting sets environment variables that control what Terraform version is downloaded
// and ensures the testing framework uses that downloaded version
fun ParametrizedWithType.terraformCoreBinaryTesting() {
text("env.TERRAFORM_CORE_VERSION", DefaultTerraformCoreVersion, "The version of Terraform Core which should be used for testing")
// and ensures the testing framework uses that downloaded version. The default Terraform core version is used if no argument is supplied.
fun ParametrizedWithType.terraformCoreBinaryTesting(tfVersion: String = DefaultTerraformCoreVersion) {
text("env.TERRAFORM_CORE_VERSION", tfVersion, "The version of Terraform Core which should be used for testing")
hiddenVariable("env.TF_ACC_TERRAFORM_PATH", "%system.teamcity.build.checkoutDir%/tools/terraform", "The path where the Terraform Binary is located. Used by the testing framework.")
}

// BuildType.overrideTerraformCoreVersion is used to override the value of TERRAFORM_CORE_VERSION in special cases where we're testing new features
// that rely on a specific version of Terraform we might not want to be used for all our tests in TeamCity.
fun BuildType.overrideTerraformCoreVersion(tfVersion: String){
params {
terraformCoreBinaryTesting(tfVersion)
}
}

fun ParametrizedWithType.terraformShouldPanicForSchemaErrors() {
hiddenVariable("env.TF_SCHEMA_PANIC_ON_ERROR", "1", "Panic if unknown/unmatched fields are set into the state")
}
Expand Down
9 changes: 5 additions & 4 deletions .teamcity/components/builds/build_steps.kt
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,15 @@ fun BuildSteps.downloadTerraformBinary() {
// https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip
val terraformUrl = "https://releases.hashicorp.com/terraform/%env.TERRAFORM_CORE_VERSION%/terraform_%env.TERRAFORM_CORE_VERSION%_linux_amd64.zip"
step(ScriptBuildStep {
name = "Download Terraform version %s".format(DefaultTerraformCoreVersion)
name = "Download Terraform"
scriptContent = """
#!/bin/bash
echo "Downloading Terraform version %env.TERRAFORM_CORE_VERSION%"
mkdir -p tools
wget -O tf.zip %s
wget -O tf.zip $terraformUrl
unzip tf.zip
mv terraform tools/
""".format(terraformUrl).trimIndent()
""".trimIndent()
})
}

Expand Down Expand Up @@ -118,7 +119,7 @@ fun BuildSteps.runAcceptanceTests() {
exit 0
fi
export TEST_COUNT=${'$'}(./test-binary -test.list=%TEST_PREFIX% | wc -l)
export TEST_COUNT=${'$'}(./test-binary -test.list="%TEST_PREFIX%" | wc -l)
echo "Found ${'$'}{TEST_COUNT} tests that match the given test prefix %TEST_PREFIX%"
if test ${'$'}TEST_COUNT -le "0"; then
echo "Skipping test execution; no tests to run"
Expand Down
10 changes: 10 additions & 0 deletions .teamcity/components/inputs/packages.kt
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ var PackagesListGa = mapOf(
"displayName" to "Environment Variables",
"path" to "./google/envvar"
),
"functions" to mapOf(
"name" to "functions",
"displayName" to "Provider-Defined Functions",
"path" to "./google/functions"
),
"fwmodels" to mapOf(
"name" to "fwmodels",
"displayName" to "Framework Models",
Expand Down Expand Up @@ -64,6 +69,11 @@ var PackagesListBeta = mapOf(
"displayName" to "Environment Variables",
"path" to "./google-beta/envvar"
),
"functions" to mapOf(
"name" to "functions",
"displayName" to "Provider-Defined Functions",
"path" to "./google-beta/functions"
),
"fwmodels" to mapOf(
"name" to "fwmodels",
"displayName" to "Framework Models",
Expand Down
5 changes: 5 additions & 0 deletions .teamcity/components/inputs/services_beta.kt
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ var ServicesListBeta = mapOf(
"displayName" to "Appengine",
"path" to "./google-beta/services/appengine"
),
"apphub" to mapOf(
"name" to "apphub",
"displayName" to "Apphub",
"path" to "./google-beta/services/apphub"
),
"artifactregistry" to mapOf(
"name" to "artifactregistry",
"displayName" to "Artifactregistry",
Expand Down
5 changes: 5 additions & 0 deletions .teamcity/components/inputs/services_ga.kt
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ var ServicesListGa = mapOf(
"displayName" to "Appengine",
"path" to "./google/services/appengine"
),
"apphub" to mapOf(
"name" to "apphub",
"displayName" to "Apphub",
"path" to "./google/services/apphub"
),
"artifactregistry" to mapOf(
"name" to "artifactregistry",
"displayName" to "Artifactregistry",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
/*
* Copyright (c) HashiCorp, Inc.
* SPDX-License-Identifier: MPL-2.0
*/

// This file is controlled by MMv1, any changes made here will be overwritten

package projects.feature_branches

import ProviderNameBeta
import ProviderNameGa
import builds.*
import generated.PackagesListBeta
import generated.PackagesListGa
import jetbrains.buildServer.configs.kotlin.Project
import jetbrains.buildServer.configs.kotlin.vcs.GitVcsRoot
import replaceCharsId
import vcs_roots.ModularMagicianVCSRootBeta
import vcs_roots.ModularMagicianVCSRootGa

const val featureBranchProviderFunctionsName = "FEATURE-BRANCH-provider-functions"
const val providerFunctionsTfCoreVersion = "1.8.0-alpha20240228"

// VCS Roots specifically for pulling code from the feature branches in the downstream and upstream repos
object HashicorpVCSRootGa_featureBranchProviderFunctions: GitVcsRoot({
name = "VCS root for the hashicorp/terraform-provider-${ProviderNameGa} repo @ refs/heads/${featureBranchProviderFunctionsName}"
url = "https://github.com/hashicorp/terraform-provider-${ProviderNameGa}"
branch = "refs/heads/${featureBranchProviderFunctionsName}"
branchSpec = "" // empty as we'll access no other branches
})

object HashicorpVCSRootBeta_featureBranchProviderFunctions: GitVcsRoot({
name = "VCS root for the hashicorp/terraform-provider-${ProviderNameBeta} repo @ refs/heads/${featureBranchProviderFunctionsName}"
url = "https://github.com/hashicorp/terraform-provider-${ProviderNameBeta}"
branch = "refs/heads/${featureBranchProviderFunctionsName}"
branchSpec = "" // empty as we'll access no other branches
})

fun featureBranchProviderFunctionSubProject(allConfig: AllContextParameters): Project {

val projectId = replaceCharsId(featureBranchProviderFunctionsName)

val packageName = "functions" // This project will contain only builds to test this single package
val sharedResourcesEmpty: List<String> = listOf() // No locking when testing functions
val vcrConfig = getVcrAcceptanceTestConfig(allConfig) // Reused below for both MM testing build configs
val trigger = NightlyTriggerConfiguration() // Resued below for running tests against the downstream repos every night.

var parentId: String // To be overwritten when each build config is generated below.

// GA
val gaConfig = getGaAcceptanceTestConfig(allConfig)
// How to make only build configuration to the relevant package(s)
val functionPackageGa = PackagesListGa.getValue(packageName)

// Enable testing using hashicorp/terraform-provider-google
parentId = "${projectId}_HC_GA"
val buildConfigHashiCorpGa = BuildConfigurationForSinglePackage(packageName, functionPackageGa.getValue("path"), "Provider-Defined Functions (GA provider, HashiCorp downstream)", ProviderNameGa, parentId, HashicorpVCSRootGa_featureBranchProviderFunctions, sharedResourcesEmpty, gaConfig)
buildConfigHashiCorpGa.addTrigger(trigger)

// Enable testing using modular-magician/terraform-provider-google
parentId = "${projectId}_MM_GA"
val buildConfigModularMagicianGa = BuildConfigurationForSinglePackage(packageName, functionPackageGa.getValue("path"), "Provider-Defined Functions (GA provider, MM upstream)", ProviderNameGa, parentId, ModularMagicianVCSRootGa, sharedResourcesEmpty, vcrConfig)

// Beta
val betaConfig = getBetaAcceptanceTestConfig(allConfig)
val functionPackageBeta = PackagesListBeta.getValue("functions")

// Enable testing using hashicorp/terraform-provider-google-beta
parentId = "${projectId}_HC_BETA"
val buildConfigHashiCorpBeta = BuildConfigurationForSinglePackage(packageName, functionPackageBeta.getValue("path"), "Provider-Defined Functions (Beta provider, HashiCorp downstream)", ProviderNameBeta, parentId, HashicorpVCSRootBeta_featureBranchProviderFunctions, sharedResourcesEmpty, betaConfig)
buildConfigHashiCorpBeta.addTrigger(trigger)

// Enable testing using modular-magician/terraform-provider-google-beta
parentId = "${projectId}_MM_BETA"
val buildConfigModularMagicianBeta = BuildConfigurationForSinglePackage(packageName, functionPackageBeta.getValue("path"), "Provider-Defined Functions (Beta provider, MM upstream)", ProviderNameBeta, parentId, ModularMagicianVCSRootBeta, sharedResourcesEmpty, vcrConfig)

val allBuildConfigs = listOf(buildConfigHashiCorpGa, buildConfigModularMagicianGa, buildConfigHashiCorpBeta, buildConfigModularMagicianBeta)

// Make these builds use a 1.8.0-ish version of TF core
allBuildConfigs.forEach{ b ->
b.overrideTerraformCoreVersion(providerFunctionsTfCoreVersion)
}

return Project{
id(projectId)
name = featureBranchProviderFunctionsName
description = "Subproject for testing feature branch $featureBranchProviderFunctionsName"

// Register feature branch-specific VCS roots in the project
vcsRoot(HashicorpVCSRootGa_featureBranchProviderFunctions)
vcsRoot(HashicorpVCSRootBeta_featureBranchProviderFunctions)

// Register all build configs in the project
allBuildConfigs.forEach{ b ->
buildType(b)
}

params {
readOnlySettings()
}
}
}
5 changes: 5 additions & 0 deletions .teamcity/components/projects/root_project.kt
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import generated.ServicesListBeta
import generated.ServicesListGa
import jetbrains.buildServer.configs.kotlin.Project
import jetbrains.buildServer.configs.kotlin.sharedResource
import projects.feature_branches.featureBranchProviderFunctionSubProject

// googleCloudRootProject returns a root project that contains a subprojects for the GA and Beta version of the
// Google provider. There are also resources to help manage the test projects used for acceptance tests.
Expand Down Expand Up @@ -57,10 +58,14 @@ fun googleCloudRootProject(allConfig: AllContextParameters): Project {
}
}

// Projects required for nightly testing, testing MM upstreams, and sweepers
subProject(googleSubProjectGa(allConfig))
subProject(googleSubProjectBeta(allConfig))
subProject(projectSweeperSubProject(allConfig))

// Feature branch-testing projects - these will be added and removed as needed
subProject(featureBranchProviderFunctionSubProject(allConfig))

params {
readOnlySettings()
}
Expand Down
21 changes: 4 additions & 17 deletions .teamcity/tests/nightly_tests_project.kt
Original file line number Diff line number Diff line change
Expand Up @@ -19,26 +19,13 @@ class NightlyTestProjectsTests {
val project = googleCloudRootProject(testContextParameters())

// Find GA nightly test project
var gaProject: Project? = project.subProjects.find { p-> p.name == gaProjectName}
if (gaProject == null) {
Assert.fail("Could not find the Google (GA) project")
}
var gaNightlyTestProject: Project? = gaProject!!.subProjects.find { p-> p.name == nightlyTestsProjectName}
if (gaNightlyTestProject == null) {
Assert.fail("Could not find the Google (GA) Nightly Test project")
}
var gaNightlyTestProject = getSubProject(project, gaProjectName, nightlyTestsProjectName)

// Find Beta nightly test project
var betaProject: Project? = project.subProjects.find { p-> p.name == betaProjectName}
if (betaProject == null) {
Assert.fail("Could not find the Google (Beta) project")
}
var betaNightlyTestProject: Project? = betaProject!!.subProjects.find { p-> p.name == nightlyTestsProjectName}
if (betaNightlyTestProject == null) {
Assert.fail("Could not find the Google (GA) Nightly Test project")
}
var betaNightlyTestProject = getSubProject(project, betaProjectName, nightlyTestsProjectName)

(gaNightlyTestProject!!.buildTypes + betaNightlyTestProject!!.buildTypes).forEach{bt ->
// Make assertions about builds in both nightly test projects
(gaNightlyTestProject.buildTypes + betaNightlyTestProject.buildTypes).forEach{bt ->
assertTrue("Build configuration `${bt.name}` contains at least one trigger", bt.triggers.items.isNotEmpty())
// Look for at least one CRON trigger
var found: Boolean = false
Expand Down
Loading

0 comments on commit f8e2462

Please sign in to comment.