Setting tags to have empty values or changing tags from having a value to having no value is not correctly handled

[petarlishov]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform version 0.14.7
AWS provider version v4.11.0

Affected Resource(s)

Multiple. Examples of this issue have been seen in:

• aws_acm_certificate
• aws_api_gateway_api_key
• aws_api_gateway_domain_name
• aws_api_gateway_rest_api
• aws_api_gateway_stage
• aws_api_gateway_usage_plan
• aws_cloudwatch_event_rule
• aws_cloudwatch_log_group
• aws_config_config_rule
• aws_db_parameter_group
• aws_db_proxy
• aws_db_subnet_group
• aws_iam_instance_profile
• aws_iam_role
• aws_instance
• aws_kinesis_firehose_delivery_stream
• aws_kms_key
• aws_rds_cluster_parameter_group
• aws_secretsmanager_secret
• aws_security_group
• aws_sns_topic
• aws_sqs_queue
• aws_wafv2_ip_set
• aws_wafv2_web_acl
• aws_lambda_function
• aws_s3_bucket
• aws_rds_cluster
• aws_rds_cluster_instance

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

First scenario:

resource "aws_s3_bucket" "bucket" {
  bucket ="somethingthatisnttaken"
  tags = {
    example = ""
  }
}

Output of terraform plan upon creation:

  # aws_s3_bucket.bucket will be created
   + resource "aws_s3_bucket" "bucket" {
...
       + bucket                      = "somethingthatisnttaken"
...
       + tags                        = {
           + "example" = ""
         }
...

Every subsequent output of terraform plan:

  # aws_s3_bucket.bucket will be updated in-place
  ~ resource "aws_s3_bucket" "bucket" {
        id                          = "somethingthatisnttaken"
      ~ tags                        = {
          + "example" = ""
        }
      ~ tags_all                    = {
          + "example" = ""
        }
        # (9 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }
...

Second scenario:

First step:

resource "aws_s3_bucket" "bucket" {
  bucket ="somethingthatisnttaken"
  tags = {
    example = "something"
  }
}

Output:

   + resource "aws_s3_bucket" "bucket" {
...
       + bucket                      = "somethingthatisnttaken"
...
       + tags                        = {
           + "example" = "something"
         }
...

Second step - terraform apply these changes:

resource "aws_s3_bucket" "bucket" {
  bucket ="somethingthatisnttaken"
  tags = {
    example = ""
  }
}

Every subsequent output of terraform plan:

  # aws_s3_bucket.bucket will be updated in-place
  ~ resource "aws_s3_bucket" "bucket" {
        id                          = "somethingthatisnttaken"
      ~ tags                        = {
          ~ "example" = "something" -> ""
        }
        # (10 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }
...

Debug Output

I apologise but I cannot get the debug output without some decent amount of research into what needs to be obfuscated in the API calls to AWSand potentially other sensitive information. If you believe this is essential for debugging this issue as you cannot replicate, I will look into what I can do

Panic Output

Expected Behavior

First scenario

Upon the first terraform apply, the S3 bucket should have the "example" tag set with no value.
A subsequent run of terraform apply should not attempt to change the resource from its initial state (the one it was left in after the first terraform apply)

Second scenario

For the first step, upon the first terraform apply, the S3 bucket should have the "example" tag set with the value of "something".
For the second step, upon the terraform apply the S3 bucket should have the "example" tag changed to have no value (not the same as tag deletion)
A subsequent run of terraform apply should not attempt to change the resource from its previous state (the one it was left in after the last terraform apply)

Actual Behavior

First scenario

Upon the first terraform apply, the AWS bucket is created with no tags
Every subsequent run of terraform apply attempts to add the "example" tag but that seems to never happen (no errors shown though, Terraform is happy and assumes that everything went well)

Second scenario

For the first step, everything goes as planned and the S3 bucket is created with the "example" tag set with the value of "something".
For the second step, upon the terraform apply the S3 bucket does not change the state of the "example" tag and it is left with the value of "something"
Every subsequent run of terraform apply attempts to change the "example" tag but that seems to never happen (no errors shown though, Terraform is happy and assumes that everything went well)

Steps to Reproduce

First scenario

1. terraform apply
2. terraform apply

Second scenario

1. terraform apply
2. change the Terraform configuration to match part 2
3. terraform apply
4. terraform apply

Important Factoids

This has been seen on a few really old accounts from what I am aware of so I would not be surprised if this has to do with the account setup. I can have an ask if you know what might be useful to know about (I am not the account owner)

References

[justinretzolk]

Related:

• Unable to updates tags with empty list with aws provider version 3.74.3  #24032
• TF fails to create tags with empty value #21896
• [Bug]: aws_launch_template can't have "aws-node-termination-handler/managed" as a key #30166
• Some aws resources don't work correctly with tags with empty/null string as a value #20371
• aws_launch_template tag_specifications cannot contain empty or null tags #13155
• aws_launch_template Tag on create values cannot be empty #11164
• aws_autoscaling_group removes empty tag "value" attributes #9049

Hopefully addressed via #29747.

[johnsonaj]

Issue was resolved in #30793 and merged to main in #31392. Will be released in v5.0.0

[github-actions]

This functionality has been released in v5.0.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.