Support AWS MSK's IAM authentication via SASL_OAUTHBEARER

[brianV]

Problem
AWS MSK is AWS's managed Kafka service. It offers an IAM access mode that restricts access based on IAM policies. They have a Serverless option, which only supports IAM access control.

You can authenticate this way by using the SASL_OAUTHBEARER mechanism as described here: https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#configure-clients-for-iam-access-control

The underlying Signer library for Go generates an OAuth token from AWS credentials in the current AWS credential provider chain. The README shows an example using Sarama here: https://github.com/aws/aws-msk-iam-sasl-signer-go/tree/main

Persona:
Which persona is this feature for? Event producers and consumers

Exit Criteria
Users can specify configuration options for a Kafka Broker that will allow it to authenticate using SASL_OATHBEARER and a token provider that generates tokens from the user's default IAM credential provider chain.

Time Estimate (optional):
How many developer-days do you think this may take to resolve?

Additional context (optional)
Add any other context about the feature request here.

[Cali0707]

cc @pierDipi @creydr

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[brianV]

Regarding the above lifecycle bot comment - should feature requests be closed just because they haven't been prioritized?

[pierDipi]

@brianV thanks for the request, our concern is that we will not be able to test the integration with AWS MSK and the library is AWS specific.

Is there any way to implement the integration with a generic oauth library that would work with components like Strimzi + IDP (like keycloak) so that we can test and verify it works and doesn't break in future releases?

/remove-lifecycle stale

[pierDipi]

I'm not sure if the approach suggested here would work with AWS identities IBM/sarama#1320 (comment)

[adrien-f]

Greetings 👋

I am currently implementing MSK IAM authentication for Keda at kedacore/keda#5692 and based on that experience I would be open also to contributing it to Knative.

I understand the worries about not being able to try out the integration with a real cluster. It is possible to implement SASL_OAUTBEARER similar to Keda:

• A generic OAuth provider that should be easily testable with any IDP
• An MSK provider with support for Pod Identity, Credentials & Credentials + Assume Role

Speaking of Keda, I know that the source can be scaled up and down with it, I suppose that once merged, this would also work with MSK brokers.

Does that sound alright?

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.