Adds Kinesis Stream Consumer resource and data source (Continued)

[rockycore]

This adds support for AWS Kinesis Stream Consumers, that allow for a dedicated consumer using enhanced fan-out

It is based on the original contribution of #6565

Changes proposed in this pull request:

Adds aws_kinesis_stream_consumer as new resource
Adds aws_kinesis_stream_consumer as new data source

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Relates OR Closes #7611

Release note for CHANGELOG:

FEATURES:
New Data Source: aws_kinesis_stream_consumer
New Resource: aws_kinesis_stream_consumer

Output from acceptance testing:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSKinesisStreamConsumer'

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSKinesisStreamConsumer -timeout 120m
=== RUN   TestAccAWSKinesisStreamConsumerDataSource
=== PAUSE TestAccAWSKinesisStreamConsumerDataSource
=== RUN   TestAccAWSKinesisStreamConsumer_basic
=== PAUSE TestAccAWSKinesisStreamConsumer_basic
=== RUN   TestAccAWSKinesisStreamConsumer_createMultipleConcurrentStreamConsumers
=== PAUSE TestAccAWSKinesisStreamConsumer_createMultipleConcurrentStreamConsumers
=== CONT  TestAccAWSKinesisStreamConsumerDataSource
=== CONT  TestAccAWSKinesisStreamConsumer_createMultipleConcurrentStreamConsumers
=== CONT  TestAccAWSKinesisStreamConsumer_basic
--- PASS: TestAccAWSKinesisStreamConsumer_basic (94.23s)
--- PASS: TestAccAWSKinesisStreamConsumer_createMultipleConcurrentStreamConsumers (94.70s)
--- PASS: TestAccAWSKinesisStreamConsumerDataSource (99.00s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       99.083s

...

[jbergknoff-rival]

@bflad @nywilken @aeschright can somebody please review this PR? This functionality would be really nice to have in the provider.

[codingdiaz]

+1

[jbergknoff-rival]

Nice work! I ran into an issue with import after pulling these changes in, but it looks like an easy fix.

[rockycore]

@jbergknoff-rival please, could you provide another review.

[jbergknoff-rival]

Well I'm not a maintainer of the library, and I'm not the most experienced Gopher, but it looks fine to me. I am happy to report that my team's been pulling this PR into our fork/build of the provider for over a month now, and using it extensively, and it's worked great.

The only issues we've seen were:

• the import problem, which you've addressed

• updating a Lambda's event source mapping, when the event source ARN is a Kinesis stream consumer, will always fail with

  Error updating Lambda event source mapping: InvalidParameterValueException: Unsupported MaximumBatchingWindowInSeconds parameter for given event source mapping type.
  

  To be clear: we're not using the MaximumBatchingWindowInSeconds attribute, but the attribute is currently invalid for dedicated consumer event source mappings, and there's a bug in the event source mapping resource that includes it in the update payload. That's an easy fix in the event source mapping resource, but I don't know if it falls in the scope of this PR.

[abirhoss]

Can we get this merged?

[iyuroch]

Any update on this feature? When can we expect this to be merged?

[assertnotnull]

Adding my voice here. I have a whole stack set up to find out I'm missing fan-out and that it's missing in this provider. I need this.

[nickpoorman]

@bflad @aareet @austingebauer @chrisarcand @jrasell @justincampbell @kmoe @manicminer @nicsnet @pwelch @radeksimko

What's the holdup on this? This is blocking a significant number of people from the 32 thumbs up on it and it's been open since Oct 2019.

What can we do to get this merged?

[neongazer]

Please any progress on this please? This quite important feature to implement AWS streaming data consumers

[breathingdust]

Hi all! 👋

Just letting you know that this item is in our backlog and will be reviewed as soon as we are able. We are currently working through roadmap items and once that is complete we will be in a better position to give feedback and hopefully merge.

We are currently unable to give you a timeline, but stay tuned for an update! We appreciate the contributions and your patience!

[teamterraform]

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Action Required: Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Action Required: Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

[bevanbennett]

Is anyone still reviewing this? People are still waiting for it... I can't edit the PR to remove aws.erb as requested.

[anthonyroach]

I made a new PR for this: #17149 with only minor changes, so hopefully it will get reviewed quickly.

[anGie44]

Hi @rockycore, thank you for this PR! Given #17149 builds upon the work here, I've taken your commits and merged them there so the work here doesn't get lost 👍 I'll be closing this PR in favor or merging in #17149 which has your changes and those made by @anthonyroach. For updates to this feature request, please refer to #17149 .

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!