Add option to compress state file for s3 remote state

[SebastianCzoch]

Thanks to that option (optional, default to false), we will be able to use gzip compression for state file.
More info available in feature request #20328

Acceptance tests output

$ make testacc TEST=./backend/remote-state/s3 TESTARGS='-run=TestRemoteClient_stateChecksum'
TF_ACC=1 go test ./backend/remote-state/s3 -v -run=TestRemoteClient_stateChecksum -mod=vendor -timeout 120m
=== RUN   TestRemoteClient_stateChecksum
=== RUN   TestRemoteClient_stateChecksum/with_compression
=== RUN   TestRemoteClient_stateChecksum/without_compression
--- PASS: TestRemoteClient_stateChecksum (31.97s)
    --- PASS: TestRemoteClient_stateChecksum/with_compression (17.22s)
        client_test.go:225: TestBackendConfig on *s3.Backend with configs.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-s3-test-5c647859"), "compression":cty.True, "dynamodb_table":cty.StringVal("terraform-remote-s3-test-5c647859"), "key":cty.StringVal("te
stState")}}
        backend_test.go:370: creating S3 bucket terraform-remote-s3-test-5c647859 in eu-west-2
        client_test.go:258: TestBackendConfig on *s3.Backend with configs.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-s3-test-5c647859"), "key":cty.StringVal("testState")}}
    --- PASS: TestRemoteClient_stateChecksum/without_compression (14.76s)
        client_test.go:225: TestBackendConfig on *s3.Backend with configs.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-s3-test-5c64786a"), "compression":cty.False, "dynamodb_table":cty.StringVal("terraform-remote-s3-test-5c64786a"), "key":cty.StringVal("t
estState")}}
        backend_test.go:370: creating S3 bucket terraform-remote-s3-test-5c64786a in eu-west-2
        client_test.go:258: TestBackendConfig on *s3.Backend with configs.synthBody{Filename:"<TestWrapConfig>", Values:map[string]cty.Value{"bucket":cty.StringVal("terraform-remote-s3-test-5c64786a"), "key":cty.StringVal("testState")}}
PASS
ok      github.com/hashicorp/terraform/backend/remote-state/s3  32.017s

Logs truncated

Solves #20328

[hashicorp-cla]

All committers have signed the CLA.

[j3parker]

Bump on this? (from Hashicorp, really.)

I would make two slight modifications:

• Set the Content-Encoding metadata on the object to gzip when doing PutObject (but don't set it to anything if compression isn't enabled): this would be the most compatible with other tools. For example, in an extreme case, if you're proxying the bucket over HTTP for some reason browsers will transparently handle Content-Encoding.
• When fetching the state I like how you're not paying attention to the compression flag -- this makes it easier to migrate. I would maybe be a bit more strict and decompress if-and-only-if the Content-Encoding metadata for the response object is gzip.

It would be nice to default to compression but I doubt Hashicorp would go for that :)

[nilya]

Suggested change

	Description: "Enable gzip compression before sending sate file into bucket",
	Description: "Enable gzip compression before sending state file into bucket",