TF not handling aws_autoscaling_group load_balancers correctly

[ghost]

This issue was originally opened by @darose as hashicorp/terraform#26125. It was migrated here as a result of the provider split. The original body of the issue is below.

---

TF is not handling aws_autoscaling_group load_balancers correctly. When I do a TF plan/apply, it does not recognize that the load balancers are attached to the ASG, and tries to delete them:

  ~ resource "aws_autoscaling_group" "master-us-east-1a-masters-use1-prod-k8s-local" {
...
      ~ load_balancers            = [
          - "api-use1-prod-k8s-local-dmvie0",
          - "api-use1-prod-k8s-local-internal",
        ]

If I allow it to do so, and then run TF plan/apply again, it then tries to perform the attachment again:

  # aws_autoscaling_attachment.master-us-east-1a-masters-use1-prod-k8s-local will be created
  + resource "aws_autoscaling_attachment" "master-us-east-1a-masters-use1-prod-k8s-local" {
      + autoscaling_group_name = "master-us-east-1a.masters.use1.prod.k8s.local"
      + elb                    = "api-use1-prod-k8s-local-dmvie0"
      + id                     = (known after apply)
    }

  # aws_autoscaling_attachment.master-us-east-1a-masters-use1-prod-k8s-local-internal will be created
  + resource "aws_autoscaling_attachment" "master-us-east-1a-masters-use1-prod-k8s-local-internal" {
      + autoscaling_group_name = "master-us-east-1a.masters.use1.prod.k8s.local"
      + elb                    = "api-use1-prod-k8s-local-internal"
      + id                     = (known after apply)
    }

Terraform Version

0.13.2

Terraform Configuration Files

Debug Output

Crash Output

Expected Behavior

Terraform should recognize that the load balancer attachments are valid and not try to remove them.

Actual Behavior

Terraform does not recognize that the load balancer attachments are valid, and so keeps alternating between adding and removing the attachments.

Steps to Reproduce

1. terraform apply - tries to remove the ASG/load balancer attachment.
2. terraform apply - tries to re-create the ASG/load balancer attachment.

Additional Context

References

[gdavison]

Hi @darose, can give us some more information so that we can look into this?

1. What version of the AWS Provider are you using? Running terraform version will include the provider version

2. Can you show us the Terraform configuration that you're using?

[EladDolev]

same thing happens to us.
these are the configurations Kops is generating.
it also happens for target_group_arns

Terraform v0.13.3

• provider registry.terraform.io/hashicorp/aws v3.7.0

resource "aws_autoscaling_group" "default-a-k8s-eu-west-1" {
  enabled_metrics      = ["GroupDesiredCapacity", "GroupInServiceInstances", "GroupMaxSize", "GroupMinSize", "GroupPendingInstances", "GroupStandbyInstances", "GroupTerminatingInstances", "GroupTotalInstances"]
  launch_configuration = aws_launch_configuration.default-a-k8s-eu-west-1.id
  max_size             = 3
  metrics_granularity  = "1Minute"
  min_size             = 1
  name                 = "default-a.k8s.eu-west-1"
  vpc_zone_identifier = [aws_subnet.private-a-k8s-eu-west-1.id]
}

resource "aws_autoscaling_attachment" "exttg-NLB--029644a29e076535-default-a" {
  alb_target_group_arn   = "arn:aws:elasticloadbalancing:eu-west-1:072412346744:targetgroup/NLB/029644a29e076535"
  autoscaling_group_name = aws_autoscaling_group.default-a-k8s-eu-west-1.id
}

[gdavison]

Hi @EladDolev, does the configuration above show the complete aws_autoscaling_group, or are you specifying both target_group_arns in the aws_autoscaling_group and using aws_autoscaling_attachment resources? If you're using both, you'll have to add ignore_changes for load_balancers and target_group_arns on the aws_autoscaling_group resource.

This is documented for the aws_autoscaling_attachment resource, but not aws_autoscaling_group. I'll update the documentation to make it more clear.

If you're not using both target_group_arns and aws_autoscaling_attachment, please provide more of your configuration so that we can analyze the issue.

[darose]

I wonder if the issue here is occurring as a result of using a separate "aws_autoscaling_attachment" section, rather than a "load_balancers = [...]" clause in the "aws_autoscaling_group" section?

[EladDolev]

Hi @gdavison
This is the complete configuration, with only tag blocks removed, there is no target_group_arns specified.
Same goes for what @darose reported, there is no load_balancers specified, only aws_autoscaling_attachment

These are default configurations generated by Kops, but there's nothing special about them.
IMHO it should get easily reproduced.

[gdavison]

Hi @darose and @EladDolev, I've managed to track this down. This is due to a change made in version 3.0 of the provider, and described at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-3-upgrade#drift-detection-enabled-for-load_balancers-and-target_group_arns-arguments. It should be better documented on the resource page itself, and I'll take care of that.

This will have to be addressed in Kops, and it looks like there are several issues open for it, including #9913 and #9891.

For more reference, the issue #3265 describes the problem if target_group_arns and load_balancers are not tracked.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!