Error during Terraform Apply, tfstate too large

[andrestorresGL]

Hi there,

During terraform apply execution I'm facing next error message

terraform plan -var="edge_hostname=myhostname.com.edgesuite.net"                                                                                                                                            
Acquiring state lock. This may take a few moments...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

module.akamai.data.local_file.hostnames: Refreshing state...
module.akamai.data.local_file.rules: Refreshing state...
module.akamai.akamai_property.property: Refreshing state... [id=prp_607038]
module.akamai.akamai_property_activation.production: Refreshing state... [id=atv_8802082]

------------------------------------------------------------------------

Error: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (4375970 vs. 4194304)


Releasing state lock. This may take a few moments...

Seems to be related to the tfstate file size.

I'm using S3 as backend with DynamoDB to lock the tfstate file.

The goal is to update an Akamai property passing the json rules as a file, the rules file is 2Mb.

terraform -v                                                                                                                                                                                                        
Terraform v0.12.26

Expected Behavior

Execute the terraform apply without errors

Actual Behavior

terraform apply fails because tfstate file is too large

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:
Create a

1. terraform init
2. terraform apply

References

hashicorp/terraform#20879

[martinstibbe]

@andrestorresGL Just to isolate storage backend RPC payload can you run apply storing state locally as a test ?

[andrestorresGL]

I had to reduce the number of rules and hostnames created using TF, I tested it locally without errors generating a tfstate file of 4.2Mb and a rules.json file of 1.6Mb

I'll be adding new rules and hostnames to test it locally with a tfstate file bigger than 4.5Mb

[andrestorresGL]

Hi @martinstibbe I was able to execute terraform apply locally you could say that the issue is related to the backend RPC

terraform apply

....

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

[andrestorresGL]

Is there any update with this issue?

[martinstibbe]

@andrestorresGL This would be an issue that would need to be logged with s3 backend provider
Similar issues ... etcd and S3 storage hit those as well

hashicorp/terraform#25745
hashicorp/terraform#20328

[zstlaw]

This has been noted as an AWS S3 issue to be submitted to the S3 provider project. Closing this issue.