Add New MSK Broker Node Data Resource

[james-bjss]

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

What?

This is a new Data Source to lookup details of MSK Broker Nodes

Why?

The current aws_msk_cluster resource has a bootstrap broker attribute that returns a comma delimited string of hostnames and ports; however this only returns one hostname per availability zone. Meaning in a 6 Node cluster only 3 hosts are returned.

Furthermore, the returned list is not stable and returns different hostnames on subsequent calls/refreshes.

Use Case

We would like to enumerate the hostnames of each broker node so we can create Consul/DNS aliases. This data source will return a list of brokers with their associated hostnames.

More Info

• I considered adding this as an attribute on the existing msk_cluster resource , but thought that it may be useful to split this out into its own resource. But happy to change if others disagree.
• The Go SDK docs also mention being able to return a list of ZK nodes however these do not seem to be returned when checking on the CLI. Would it be worth distinguishing the resource name?
• The datasource is plural, hopefully this is acceptable? Should we consider a singular form for single brokers?

Kafka API Docs

Output From Acceptance Testing

$ make testacc TESTARGS='-run=TestAccAWSMskNodesDataSource_Name'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSMskNodesDataSource_Name -timeout 180m
=== RUN   TestAccAWSMskNodesDataSource_Name
=== PAUSE TestAccAWSMskNodesDataSource_Name
=== CONT  TestAccAWSMskNodesDataSource_Name
--- PASS: TestAccAWSMskNodesDataSource_Name (1838.65s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       1838.776s

Relates: #19659
Closes: #20940
Closes: #20628
Closes: #19638
Closes: #11085
Closes: #15731.

[github-actions]

Welcome @james-bjss 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

[ewbankkit]

LGTM 🚀.

Commercial

% make testacc TESTARGS='-run=TestAccAWSMskBrokerNodesDataSource_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSMskBrokerNodesDataSource_basic -timeout 180m
=== RUN   TestAccAWSMskBrokerNodesDataSource_basic
=== PAUSE TestAccAWSMskBrokerNodesDataSource_basic
=== CONT  TestAccAWSMskBrokerNodesDataSource_basic
--- PASS: TestAccAWSMskBrokerNodesDataSource_basic (1919.42s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	1922.956s

GovCloud

% make testacc TESTARGS='-run=TestAccAWSMskBrokerNodesDataSource_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSMskBrokerNodesDataSource_basic -timeout 180m
=== RUN   TestAccAWSMskBrokerNodesDataSource_basic
=== PAUSE TestAccAWSMskBrokerNodesDataSource_basic
=== CONT  TestAccAWSMskBrokerNodesDataSource_basic
--- PASS: TestAccAWSMskBrokerNodesDataSource_basic (1172.83s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	1176.275s

[ewbankkit]

@james-bjss Thanks for the contribution 🎉 👏.
In order to get this merged quicker I went ahead and made a few changes:

• Renamed the data source to aws_msk_broker_nodes
• Renamed the output list to node_info_list to match the underlying AWS API
• Added the client_vpc_ip_address and node_arn attributes

[james-bjss]

@james-bjss Thanks for the contribution 🎉 👏.
In order to get this merged quicker I went ahead and made a few changes:

• Renamed the data source to aws_msk_broker_nodes
• Renamed the output list to node_info_list to match the underlying AWS API
• Added the client_vpc_ip_address and node_arn attributes

@ewbankkit No problem, thanks for the changes and for reviewing. 👍

[github-actions]

This functionality has been released in v3.60.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.