Terraform state updated with new Chart values after apply errored

[jose-ledesma]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version:  v1.0.5
Provider version: v2.2.0
Kubernetes version: v1.21 (EKS)

Affected Resource(s)

• helm_release

Terraform Configuration Files

provider "helm" {
  kubernetes {
    host                   = try(module.eks_cluster["cluster1"].output.eks_cluster.endpoint, "")
    cluster_ca_certificate = try(base64decode(module.eks_cluster["cluster1"].output.eks_cluster.certificate_authority.0.data), "")
    token                  = try(data.aws_eks_cluster_auth.cluster["cluster1"].token, "")
  }
}

resource "helm_release" "filebeat" {
  name       = "filebeat"
  repository = "https://helm.elastic.co"
  chart      = "filebeat"
  version    = "~> 7.14.0"

  values = [templatefile("manifests/filebeat.yml.tpl", {
    "key1" : "value1",
    "key2" : "value2",
  })]
}

Debug Output

NOTE: In addition to Terraform debugging, please set HELM_DEBUG=1 to enable debugging info from helm.

Panic Output

Steps to Reproduce

1. Plan and apply the helm_release
2. make a modification in the values (Add/change/remove any of the Helm values)
3. Run terraform plan : with the provider configuration it will retrieve a new token for connecting to the EKS cluster. This token has a 15 minutes lifespan
4. wait 15 minutes
5. terraform apply

Terraform apply will fail with:

Error: Kubernetes cluster unreachable: the server has asked for the client to provide credentials
with helm_release.filebeat
on filebeat.tf line 15, in resource "helm_release" "filebeat":
resource "helm_release" "filebeat" {

Nonetheless the Terraform State will be updated with the new Values, and subsequent terraform plan will detect no changes

Expected Behavior

Terraform state should not be updated

Actual Behavior

Terraform state is updated after the failure

Important Factoids

Using Terraform Cloud

References

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[nfi-opsguru]

My workaround:

1. terraform state rm helm_release.$name
2. terraform import <addr> <namespace>/<release-name>

This is quite a nasty bug IMO.

[imriz]

This is a very nasty bug, and no one even reviewed it over two months later?

[imriz]

Trying to address it in PR #857

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.