r/aws_s3_bucket_object: Updating just tags creates new object version

[ewbankkit]

I noticed this while adding support for S3 object legal hold and retention: #6634.

Updating just the tags for an S3 object in a versioned S3 bucket causes the creation of a new object version. This is not the behavior when performing the same action via the AWS console where the object version remains constant and just its tags are updated.

resource "aws_s3_bucket_object" "a" {
  bucket  = "my-versioned-bucket"
  key     = "data.txt"
  content = "A"
}

$ terraform apply
aws_s3_bucket_object.a: Creating...
  acl:                    "" => "private"
  bucket:                 "" => "my-versioned-bucket"
  content:                "" => "A"
  content_type:           "" => "<computed>"
  etag:                   "" => "<computed>"
  key:                    "" => "data.txt"
  server_side_encryption: "" => "<computed>"
  storage_class:          "" => "<computed>"
  version_id:             "" => "<computed>"
aws_s3_bucket_object.a: Creation complete after 0s (ID: data.txt)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

$ grep version_id terraform.tfstate 
                            "version_id": "bznhV5wG05yXBhxyqdHmM90Jv_eROybX",

update to:

resource "aws_s3_bucket_object" "a" {
  bucket  = "my-versioned-bucket"
  key     = "data.txt"
  content = "A"

  tags {
    Key1 = "Value1"
  }
}

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

aws_s3_bucket_object.a: Refreshing state... (ID: data.txt)

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_s3_bucket_object.a
      tags.%:    "0" => "1"
      tags.Key1: "" => "Value1"


Plan: 0 to add, 1 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

$ terraform apply
aws_s3_bucket_object.a: Refreshing state... (ID: data.txt)
aws_s3_bucket_object.a: Modifying... (ID: data.txt)
  tags.%:    "0" => "1"
  tags.Key1: "" => "Value1"
aws_s3_bucket_object.a: Modifications complete after 1s (ID: data.txt)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

$ grep version_id terraform.tfstate
                            "version_id": "Jy7xRCGut7yuc7VOaMcOrOnmwCw7Vsfv",

[ewbankkit]

It seems that this behavior is being used to force a new object version - Would this fix be seen as a breaking change?

[ewbankkit]

The same happens when just the canned ACL is updated.

[ewbankkit]

To me it seems the confusion is caused by the fact that, for versioned buckets, the aws_s3_bucket_object resource represents the latest version of the S3 object and not a specific version of the object.
Therefore only changes to the bucket or key attributes force a new instance of the resource and changes to any of the other attributes forces a new version of the S3 object but just an update to the existing resource instance.
It seems there are management and cost overheads associated with creating unnecessary object versions.

[bflad]

This has been released in version 1.56.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

@ewbankkit
thought that have been fixed
for one reason is created a new version with 0 size
please see the attached photo

aws_s3_bucket_object.***************_bucket_object_tag[738]: Creating...
  acl:                    "" => "private"
  bucket:                 "" => "************************"
  content_type:           "" => "<computed>"
  etag:                   "" => "<computed>"
  force_destroy:          "" => "false"
  key:                    "" => "output/dt=2019-11-19/hr=10/246d2c40-9646-451b-853b-20806b935b6f.json.gzip"
  server_side_encryption: "" => "<computed>"
  storage_class:          "" => "<computed>"
  tags.%:                 "" => "1"
  tags.CostArea:          "" => "Data Lake"
  version_id:             "" => "<computed>"

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!