Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: run terraform validate on modules again #20230

Merged
merged 6 commits into from
Dec 16, 2023

Conversation

lilatomic
Copy link
Contributor

Only running terraform validate on deployments is unexpected for users. See #20222 . Running terraform validate on all modules might lead to errors with Terraform, but those are predictable and understandable by users.
This MR:

  • Runs terraform validate on terraform_modules again
  • Adds the skip_terraform_validate field for terraform_modules which cannot be validated by Terraform
  • Adds some documentation for the Terraform backend

It leaves open the possibility of validating both terraform_deployments and terraform_modules. Hopefully we can work out something like the Helm backend's handling of kubeconform.

We don't depend on the backend config, so we don't actually need to target a terraform_deployment
everything still worked because we can technically still use terraform_deployment targets, they just get filtered out
@lilatomic lilatomic added bug category:bugfix Bug fixes for released features backend: Terraform Terraform backend-related issues labels Nov 22, 2023
@lilatomic
Copy link
Contributor Author

We'll probably want to backport this to 2.18 and 2.19 too

@huonw huonw added this to the 2.18.x milestone Nov 22, 2023
@huonw
Copy link
Contributor

huonw commented Nov 22, 2023

I've marked it for 2.18. (And also, leaving a comment so I get notified when this merges, so I can cut 2.18.1rc2 immediately, to potentially get 2.18.1 out soon.)

@lilatomic lilatomic requested a review from tdyas November 23, 2023 04:21
---
> 🚧 Terraform support is in alpha stage
>
> Pants is currently building support for developing and deploying Terraform. Simple usecases might be supported, but many options are missing.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add space for "use cases"

@@ -0,0 +1,124 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding docs!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hoping that we get some feedback once people can see that it exists haha



@dataclass(frozen=True)
class TerraformValidateFieldSet(TerraformFieldSet):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this need to extend the parent's required_fields to include SkipTerraformValidateField?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know the answer, so I dug around. Seems the typical way does not add the skip field to the required fields, but also doesn't inherit from the parent FieldSet. Usually the fieldset has just the sources field as required. I think that's to let them cover all the target types that have those sources (for example, isort needs to cover python_source and python_test). The Helm backend, OTOH, subclasses. I think that allows us to avoid duplicating all the fields. I think it makes sense for us to follow the Helm backend, since Terraform files are mostly single-purpose. We might want to revisit that if we decide we want the check both modules and deployments (like helm.kubeconform).

I think using tgt.get is what allows the field to be optional and fill in the default.

docs/markdown/Terraform/terraform-overview.md Outdated Show resolved Hide resolved
@@ -0,0 +1,50 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add this documentation page in a separate PR in order to keep this PR on one topic?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benjyw: thoughts on above question?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm neutral on that question, since the docs are pertinent to the code change, even if wider.

@tdyas tdyas merged commit 3b90d9e into pantsbuild:main Dec 16, 2023
24 checks passed
WorkerPants pushed a commit that referenced this pull request Dec 16, 2023
Only running `terraform validate` on deployments is unexpected for
users. See #20222 . Running
`terraform validate` on all modules might lead to errors with Terraform,
but those are predictable and understandable by users.
This MR:
- Runs `terraform validate` on `terraform_modules` again
- Adds the `skip_terraform_validate` field for `terraform_modules` which
cannot be `validate`d by Terraform
- Adds some documentation for the Terraform backend

It leaves open the possibility of validating both
`terraform_deployment`s and `terraform_module`s. Hopefully we can work
out something like the Helm backend's handling of kubeconform.

---------

Co-authored-by: Benjy Weinberger <[email protected]>
WorkerPants pushed a commit that referenced this pull request Dec 16, 2023
Only running `terraform validate` on deployments is unexpected for
users. See #20222 . Running
`terraform validate` on all modules might lead to errors with Terraform,
but those are predictable and understandable by users.
This MR:
- Runs `terraform validate` on `terraform_modules` again
- Adds the `skip_terraform_validate` field for `terraform_modules` which
cannot be `validate`d by Terraform
- Adds some documentation for the Terraform backend

It leaves open the possibility of validating both
`terraform_deployment`s and `terraform_module`s. Hopefully we can work
out something like the Helm backend's handling of kubeconform.

---------

Co-authored-by: Benjy Weinberger <[email protected]>
@WorkerPants
Copy link
Member

I tried to automatically cherry-pick this change back to each relevant milestone, so that it is available in those older releases of Pants.

✔️ 2.18.x

Successfully opened #20299.

✔️ 2.19.x

Successfully opened #20298.


Thanks again for your contributions!

🤖 Beep Boop here's my run link

benjyw pushed a commit that referenced this pull request Dec 17, 2023
#20299)

Only running `terraform validate` on deployments is unexpected for
users. See #20222 . Running
`terraform validate` on all modules might lead to errors with Terraform,
but those are predictable and understandable by users.
This MR:
- Runs `terraform validate` on `terraform_modules` again
- Adds the `skip_terraform_validate` field for `terraform_modules` which
cannot be `validate`d by Terraform
- Adds some documentation for the Terraform backend

It leaves open the possibility of validating both
`terraform_deployment`s and `terraform_module`s. Hopefully we can work
out something like the Helm backend's handling of kubeconform.

Co-authored-by: Daniel Goldman <[email protected]>
benjyw pushed a commit that referenced this pull request Dec 17, 2023
#20298)

Only running `terraform validate` on deployments is unexpected for
users. See #20222 . Running
`terraform validate` on all modules might lead to errors with Terraform,
but those are predictable and understandable by users.
This MR:
- Runs `terraform validate` on `terraform_modules` again
- Adds the `skip_terraform_validate` field for `terraform_modules` which
cannot be `validate`d by Terraform
- Adds some documentation for the Terraform backend

It leaves open the possibility of validating both
`terraform_deployment`s and `terraform_module`s. Hopefully we can work
out something like the Helm backend's handling of kubeconform.

Co-authored-by: Daniel Goldman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend: Terraform Terraform backend-related issues bug category:bugfix Bug fixes for released features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants