Add support for temporary AWS IAM credentials (using session tokens) from secrets

[JacobHenner]

Hello,

In my environment there are situations where temporary (sts-generated) IAM role credentials are stored in Kubernetes secrets and updated as needed. In this environment IRSA is unavailable, and we cannot permit KEDA to assume-role.

Instead, it'd be great if we could use the existing static credentials mechanism, and add support for specifying session tokens (needed for temporary credentials). For example:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger-auth-aws-credentials
  namespace: keda-test
spec:
  secretTargetRef:
  - parameter: awsAccessKeyID     # Required.
    name: test-secrets            # Required.
    key: AWS_ACCESS_KEY_ID        # Required.
  - parameter: awsSecretAccessKey # Required.
    name: test-secrets            # Required.
    key: AWS_SECRET_ACCESS_KEY    # Required.
  - parameter: awsSessionToken # Optional.
    name: test-secrets # Optional.
    key: AWS_SESSION_TOKEN # Optional.

If this proposal is acceptable, I can craft the PR. Adding support for the extra value seems to be straightforward. I just need to research how credential refreshing would be implemented (assuming it's needed), as these temporary creds would obviously expire after some time.

[tomkerkhove]

That sounds like a reasonable ask and nice improvement. Thoughts @kedacore/keda-maintainers?

Thanks for suggesting to contributing this!

[JorTurFer]

I'm not totally sure about this.
Providing a temporally token when the IRSA is unavailable seems like change the authentication method from pod-identity to another IMHO. It is possible to change it directly in the same moment when you'd add the temp token, isn't?
Maybe I'm missing something because AWS and I are not very friends... xD

Could you share a high level description about the solution you are thinking on? (how to manage the token expiration, how to use one or the other,...)

[JacobHenner]

I'm not totally sure about this. Providing a temporally token when the IRSA is unavailable seems like change the authentication method from pod-identity to another IMHO. It is possible to change it directly in the same moment when you'd add the temp token, isn't? Maybe I'm missing something because AWS and I are not very friends... xD

When I look at the docs for the sqs scaler, I see identityOwner: pod broken down into three categories: pod-identity based authn, role based authn, and credential based authn. My proposal is to extend the credential based authn to optionally support a third value (awsSessionToken). If set, that value would replace the empty string currently supplied to NewStaticCredentials.

keda/pkg/scalers/aws_sqs_queue_scaler.go

Line 124 in 2446b8e

creds := credentials.NewStaticCredentials(metadata.awsAuthorization.awsAccessKeyID, metadata.awsAuthorization.awsSecretAccessKey, "")

I haven't yet explored the lifetime of these creds objects, or how KEDA behaves if static credentials become invalid or the underlying secret changes. I'd assume this has been considered already, as static credentials can become invalid and the secrets containing the credentials can change. This would just happen more frequently if temporary creds (using session tokens) were supported.

If the concept of adding support for temporary credentials sounds acceptable in general, I'll proceed by examining the lifetime of these credential objects, determine how KEDA currently handles expirations/secret changes (assuming the credential object lifetimes are long enough to warrant this), and if needed, propose any changes that would be necessary to support temporary credentials.

[JorTurFer]

Hi @JacobHenner
I have to apologize because I have been busy last week :(
I misunderstood your plan, sounds totally reasonable, :) I didn't know that you can use a session token instead of the other methods

[JacobHenner]

I've got the session tokens working, but only the keda-operator component seems to re-read the secret if auth starts failing due to token expiration. I've spent some time looking into how this could be handled in the metrics server component, and haven't yet figured it out. I'll revisit this in the next few days.

See also: #511, #563.