Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gke gcsfuse #7884

Merged
merged 8 commits into from
May 9, 2023
Merged

Gke gcsfuse #7884

merged 8 commits into from
May 9, 2023

Conversation

soumyapani
Copy link
Contributor

This PR is to add terraform support for running GCSFuse as CSI driver for GKE cluster.

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
  • Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

GKE: Add GCSFuse for GKE cluster.

@modular-magician modular-magician requested a review from megan07 May 4, 2023 22:03
@modular-magician
Copy link
Collaborator

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @megan07, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

Your assigned reviewer will help review your code by:

  • Ensuring it's backwards compatible, covers common error cases, etc.
  • Summarizing the change into a user-facing changelog note.
  • Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.


@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 44 insertions(+))
Terraform Beta: Diff ( 3 files changed, 44 insertions(+))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Affected tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

@megan07
Copy link
Contributor

megan07 commented May 5, 2023

I think if we want to do this we'll need to update the dependency here to v0.121.0

@soumyapani
Copy link
Contributor Author

I think if we want to do this we'll need to update the dependency here to v0.121.0

Thanks Megan.
I am looking into how to modify the go.mod.erb and go.sum files. Is it sufficient to just change in go.mod.erb to v0.121.0 or do I need to change go.sum as well ?

@DmitryKakurin
Copy link

Is it sufficient to just change in go.mod.erb to v0.121.0 or do I need to change go.sum as well ?

Run go get -u google.golang.org/api && go mod tidy to update go.sum

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 64 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 5 files changed, 64 insertions(+), 5 deletions(-))

@soumyapani
Copy link
Contributor Author

Is it sufficient to just change in go.mod.erb to v0.121.0 or do I need to change go.sum as well ?

Run go get -u google.golang.org/api && go mod tidy to update go.sum

even after updating the go.mod and go.sum files I am still getting this error.

Step #12 - "tgc-head": /go/pkg/mod/github.com/modular-magician/[email protected]/google/resource_container_cluster.go:3496:6: ac.GcsFuseCsiDriverConfig undefined (type *container.AddonsConfig has no field or method GcsFuseCsiDriverConfig)
Step #12 - "tgc-head": /go/pkg/mod/github.com/modular-magician/[email protected]/google/resource_container_cluster.go:3496:42: undefined: container.GcsFuseCsiDriverConfig
Step #12 - "tgc-head": /go/pkg/mod/github.com/modular-magician/[email protected]/google/resource_container_cluster.go:4392:7: c.GcsFuseCsiDriverConfig undefined (type *container.AddonsConfig has no field or method GcsFuseCsiDriverConfig)
Step #12 - "tgc-head": /go/pkg/mod/github.com/modular-magician/[email protected]/google/resource_container_cluster.go:4395:18: c.GcsFuseCsiDriverConfig undefined (type *container.AddonsConfig has no field or method GcsFuseCsiDriverConfig)
Step #12 - "tgc-head": make: *** [Makefile:4: build] Error 2

the new attribute is present in the go beta package https://pkg.go.dev/google.golang.org/api/container/v1beta1#GcsFuseCsiDriverConfig

Do I need to do anything else to use this attribute ?

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 2719
Passed tests 2426
Skipped tests: 279
Affected tests: 14

Action taken

Found 14 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeNetworkPeering_subnetRoutes|TestAccComputeNetworkPeering_basic|TestAccComputeNetworkPeeringRoutesConfig_networkPeeringRoutesConfigBasicExample|TestAccComputeRoute_routeIlbVipExample|TestAccComputeNetworkPeering_customRoutesUpdate|TestAccContainerCluster_withAddons|TestAccComputeFirewallPolicyRule_multipleRules|TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example|TestAccAlloydbCluster_missingLocation|TestAccAlloydbBackup_missingLocation|TestAccDataSourceAlloydbLocations_basic|TestAccDataSourceComputeNetworkPeering_basic|TestAccDataSourceGoogleFirebaseAndroidAppConfig|TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

Tests passed during RECORDING mode:
TestAccComputeNetworkPeering_subnetRoutes[Debug log]
TestAccComputeNetworkPeering_basic[Debug log]
TestAccComputeNetworkPeeringRoutesConfig_networkPeeringRoutesConfigBasicExample[Debug log]
TestAccComputeRoute_routeIlbVipExample[Debug log]
TestAccComputeNetworkPeering_customRoutesUpdate[Debug log]
TestAccContainerCluster_withAddons[Debug log]
TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example[Debug log]
TestAccAlloydbCluster_missingLocation[Debug log]
TestAccAlloydbBackup_missingLocation[Debug log]
TestAccDataSourceAlloydbLocations_basic[Debug log]
TestAccDataSourceComputeNetworkPeering_basic[Debug log]
TestAccDataSourceGoogleFirebaseAndroidAppConfig[Debug log]
TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample[Debug log]

Tests failed during RECORDING mode:
TestAccComputeFirewallPolicyRule_multipleRules[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

@megan07
Copy link
Contributor

megan07 commented May 8, 2023

/gcbrun

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 5 files changed, 64 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 5 files changed, 64 insertions(+), 5 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 2719
Passed tests 2433
Skipped tests: 279
Affected tests: 7

Action taken

Found 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeFirewallPolicyRule_multipleRules|TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example|TestAccAlloydbBackup_missingLocation|TestAccAlloydbCluster_missingLocation|TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample|TestAccDataSourceAlloydbLocations_basic|TestAccDataSourceGoogleFirebaseAndroidAppConfig

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

Tests passed during RECORDING mode:
TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example[Debug log]
TestAccAlloydbBackup_missingLocation[Debug log]
TestAccAlloydbCluster_missingLocation[Debug log]
TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample[Debug log]
TestAccDataSourceAlloydbLocations_basic[Debug log]

Tests failed during RECORDING mode:
TestAccComputeFirewallPolicyRule_multipleRules[Error message] [Debug log]
TestAccDataSourceGoogleFirebaseAndroidAppConfig[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

@soumyapani
Copy link
Contributor Author

Hi @megan07
Looks like the tests are failing because of quota issues. Can you please suggest how to fix this for this PR ?

@@ -75,6 +75,7 @@ var (
"addons_config.0.network_policy_config",
"addons_config.0.cloudrun_config",
"addons_config.0.gcp_filestore_csi_driver_config",
"addons_config.0.gcs_fuse_csi_driver_config",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to only be available in beta, can we move all of this into <% unless version == 'ga' -%> blocks? You can follow how istio_config and kalm_config are handled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we move only gcs_fuse_csi_driver_config to 'ga' block or should we move everything in addonsConfigKeys ?

I understand that istio_config and kalm_config are deprecated. What is the effect of moving gcs_fuse_csi_driver_config to 'ga' block ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just gcs_fuse_csi_driver_config should move into the block. Typically in cases like these, I just move them into the existing blocks that contain istio_config and kalm_config, thus they are all grouped together and its easier to read IMO. It shouldn't matter that the others are deprecated, they're still in the code and will be handled in the same manner wrt the provider. Does that make sense?

Essentially the effect is that the new config block won't make it to the google provider, but only the google-beta provider. You can look at this comment above and it will show what the code will look like once they're generated downstream.

For example, you'll see that istio_config is in the google-beta provider, but it doesn't exist in the google provider. (Except for docs, we standardize on beta docs)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the information.

I've updated it in the latest iteration.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! and sorry for the mis-understanding, but you'll want to move all the code below that references it into blocks as well. if you do a search in the code for istio_config you'll see what i mean, i think.

@modular-magician modular-magician requested a review from megan07 May 9, 2023 19:04
@@ -331,6 +332,23 @@ func ResourceContainerCluster() *schema.Resource {
},
},
},
"gcs_fuse_csi_driver_config": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, you'll want to move this block of code to line 423, and current line 5006 to 5056. Does that make sense? And even references in the tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for that.

I have updated them all in the latest iteration. Can you please check if it looks good.

@modular-magician modular-magician requested a review from megan07 May 9, 2023 19:13
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 25 insertions(+), 6 deletions(-))
Terraform Beta: Diff ( 5 files changed, 65 insertions(+), 6 deletions(-))
TF Conversion: Diff ( 2 files changed, 23 insertions(+), 14 deletions(-))

Copy link
Contributor

@megan07 megan07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll wait for tests to finish from here, but other than the two minor whitespace edits I've mentioned, I think it looks good! Thanks!

@@ -3920,6 +3920,9 @@ resource "google_container_cluster" "primary" {
kalm_config {
enabled = false
}
gcs_fuse_csi_driver_config {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we fix the spacing on this (and the line above it) please, to be lined up with kalm_config

@@ -3981,6 +3984,9 @@ resource "google_container_cluster" "primary" {
kalm_config {
enabled = true
}
gcs_fuse_csi_driver_config {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could we fix the spacing here too please? for both this and kalm_config

@modular-magician modular-magician requested a review from megan07 May 9, 2023 19:57
Copy link
Contributor

@megan07 megan07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much!

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 24 insertions(+), 5 deletions(-))
Terraform Beta: Diff ( 5 files changed, 69 insertions(+), 10 deletions(-))
TF Conversion: Diff ( 2 files changed, 23 insertions(+), 14 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 2722
Passed tests 2433
Skipped tests: 279
Affected tests: 10

Action taken

Found 10 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer|TestAccSqlDatabaseInstance_basicMSSQL|TestAccComputeFirewallPolicyRule_multipleRules|TestAccAlloydbBackup_missingLocation|TestAccAlloydbCluster_missingLocation|TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample|TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example|TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable|TestAccDataSourceGoogleFirebaseAndroidAppConfig|TestAccDataSourceAlloydbLocations_basic

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

Tests passed during RECORDING mode:
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer[Debug log]
TestAccSqlDatabaseInstance_basicMSSQL[Debug log]
TestAccAlloydbBackup_missingLocation[Debug log]
TestAccAlloydbCluster_missingLocation[Debug log]
TestAccApigeeKeystoresAliasesKeyCertFile_apigeeKeystoresAliasesKeyCertFileTestExample[Debug log]
TestAccApigeeKeystoresAliasesPkcs12_ApigeeKeystoresAliasesPkcs12Example[Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable[Debug log]
TestAccDataSourceGoogleFirebaseAndroidAppConfig[Debug log]
TestAccDataSourceAlloydbLocations_basic[Debug log]

Tests failed during RECORDING mode:
TestAccComputeFirewallPolicyRule_multipleRules[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

@soumyapani
Copy link
Contributor Author

Looks like this test is failing with quota error TestAccComputeFirewallPolicyRule_multipleRules. @megan07 can you please suggest how should we go about this ?

vcr_utils.go:145: Step 4/6 error: Error running apply: exit status 1
        
        Error: Error creating FirewallPolicyRule: operation received error: error code "QUOTA_EXCEEDED", message: Quota 'FIREWALL_SECURITY_POLICY_RULES' exceeded.  Limit: 250.0 globally.
        
        
          with google_compute_firewall_policy_rule.rule3,
          on terraform_plugin_test.tf line 67, in resource "google_compute_firewall_policy_rule" "rule3":
          67: resource "google_compute_firewall_policy_rule" "rule3" {

@megan07
Copy link
Contributor

megan07 commented May 9, 2023

That should be fine, it's unrelated to your change. Thanks!

@soumyapani
Copy link
Contributor Author

That should be fine, it's unrelated to your change. Thanks!

What is the process to start the merge ? can you please help me to merge this PR to magic module repo ?

@megan07 megan07 merged commit 5e53b53 into GoogleCloudPlatform:main May 9, 2023
@megan07
Copy link
Contributor

megan07 commented May 9, 2023

Done :) Thanks!

@soumyapani
Copy link
Contributor Author

Done :) Thanks!

Thank you so much @megan07. How long does it take for this to available in the terraform google_container_cluster resource for us to use this ?

@soumyapani soumyapani deleted the gke-gcsfuse branch May 10, 2023 00:12
ericayyliu pushed a commit to ericayyliu/magic-modules that referenced this pull request Jul 26, 2023
* Adding GCFuse addons attributes in GKE terraform resource.

* Fixing syntax error.

* Updating go.mud and go.sum files.

* Moving "gcp_filestore_csi_driver_config" to "ga" block to include it only in google-beta version.

* Update mmv1/third_party/terraform/website/docs/r/container_cluster.html.markdown

Co-authored-by: megan07 <[email protected]>

* Updating "gcs_fuse_csi_driver_config" location.

* Fixing the error and moving the right gcs_fuse_csi_driver_config attribute.

* Fixing spacing.

---------

Co-authored-by: megan07 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants