aws_msk_cluster : default value for client_broker : "TLS"

[kunsalvi]

Hi

I am creating MSK cluster and if i do not keep any encryption_in_transit. It picks up "TLS" value for client_broker and cluster is created. But if I do terrform apply (without making any changes.). It forces below change :
encryption_info.0.encryption_in_transit.0.client_broker: "TLS" => "TLS_PLAINTEXT" (forces new resource)

Moreover if I provide value as :
encryption_in_transit {
client_broker = "TLS"
in_cluster = true
}

Cluster creation fails with :
Error: Error refreshing state: 1 error occurred:
* aws_msk_cluster.atropos_prod: 1 error occurred:
* aws_msk_cluster.atropos_prod: aws_msk_cluster.XXXX: failed requesting bootstrap broker info for "arn:aws:kafka:ap-south-1:286817589435:cluster/XXXX/684bd0d9-eecf-454d-b209-306ead63dda6-4" : BadRequestException: You can't get bootstrap broker nodes for a cluster in FAILED state.
status code: 400, request id: 81a984a9-2646-46e7-86af-236c0a88b329

Terrform version : Terraform v0.11.14

[bflad]

Thanks for submitting this, @kunsalvi.

A few weeks after general availability launch, the API default for encryption switched from TLS_PLAINTEXT to TLS. We unfortunately cannot update the Terraform default to match the updated API default until a major version update of the Terraform AWS Provider as it would represent a potentially breaking change for existing Terraform configurations. I have marked this in the 3.0.0 milestone for now to ensure this change is included in upcoming planning for that major version.

Regarding the BadRequestException you're seeing, it sounds like you may want to check in with AWS Support on why the cluster is entering a FAILED state -- the maintainers here most likely would not be able to help directly with that unless the MSK team can pinpoint that the Terraform resource is doing something incorrectly.

[kunsalvi]

@bflad

Thank you for the response. For now, I am explicitly specifying "client_broker" and its working for me.

And "BadRequestException" is due to my certificate issue.

[wvidana]

This is even worst now.

If not specified it forces a new resource since now it wants to be null

          - encryption_in_transit {
              - client_broker = "TLS" -> null # forces replacement
              - in_cluster    = true -> null
            }

[bflad]

The attribute default update has been merged and will release with version 3.0.0 of the Terraform AWS Provider, likely in the next two weeks. 👍

[ghost]

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!