Provide CloudEvent design proposal to extend KEDA

[tomkerkhove]

Provide CloudEvent that are emitted by the runtime so that customers can extend KEDA.

Events that come to mind are:

• New Trigger Authentication Created
• Deployment Scaled to Zero
• New Scaled Object Created
• Information on scaling action
• Information about new scaled object referencing existing trigger authentication

Only concern I have is that folks could go through the Kubernetes eventing approach to get the information, but if we would emit it ourselves we could make it more user friendly rather than having to figure this out via the Kube way

Tasklist

• Create proposal
• Agree on proposal
• Define list of events & payloads to emit
• Implement
• Document

[tomkerkhove]

Thoughts @jeffhollan?

[zroubalik]

We should definitely implement the Kuberenetes eventing approach (created #530). This could be lower priority item.

[tomkerkhove]

Thanks for that! Since Kubernetes doens't support CloudEvents (yet), I would leverage this as a seperate component as well to make it easier to integrate with other products/tools that are not Kubernetes event specific.

See kubernetes/kubernetes#85544 for CloudEvents support in Kubernetes.

[tomkerkhove]

The reason for that is that, for most scenarios you'd want to react on these events, and these workloads are not always in Kubernetes. If we only support Kubernetes events how it is today, events are stuck inside the cluster so they are, in my opinion, often pointless.

If we have CloudEvents, we can just forward them to things like Azure Event Grid and take it from there with FaaS/SaaS/PaaS services like Azure Functions.

[tomkerkhove]

I will provide a more thorough proposal before we start on this.

[tomkerkhove]

Thanks for that! And seconding v3 :D

[JorTurFer]

I see the business value and I think it's easy to do xD The thing is that it requires information about the infrastructure status and I know KEDA can get it, but I'm not sure if KEDA should do it. I mean, why only max replicas and not min/current replicas?
I feel that monitoring this information like infrastructure/workload monitoring and that should be done by other tools, prometheus/newrelic/datadog/whatever.

It's not a blocker for me, if we think that it's needed, we can do it

[tomkerkhove]

I don't see this as monitoring since neither Prometheus nor the rest will tell you when the workload needs 11 replicas but it's not scaling because 10 is the max.

Imagine having an event for which you can subscribe, page somebody and mitigate it before your system is flooded.

[JorTurFer]

The problem is that "in theory" KEDA doesn't know when you need more than 10 replicas, it's the HPA Controller who knows it. The owner of that knowledge, it's not KEDA IMO.
Apart from that, we cannot guarantee the consistency of this event because CPU and memory run totally outside KEDA scope

[tomkerkhove]

But similarly to how we offer scale-to-zero, we can leverage this as well in my opinion. If it's technically possible, which I think it should be but I do agree that this could be in a separate component if need be.

[dmeytin]

Can you please help me to understand how rising Cloud Events will help with multi-cluster use-case #1587 ?

[tomkerkhove]

Events will allow you to extend KEDA to close our current multi-cluster gap and take required actions. For example if we're at max replicas you can scale out in another region or if it fails to scale, etc.

However, we are open to event requests that can help you more though.

[tomkerkhove]

I've completed my design proposal and anyone who is interested is welcome to review and comment on it: docs.google.com/document/d/14Z04AWiMBkSX88uiaVS-kg-8QjjNcDhVy6-u5FMUk_w/edit?usp=sharing

In terms of implementation, I would identify a first batch of events to support and create dedicated items per event scenario so that they can be added afterwards in a phased-approach.

/cc @kedacore/keda-maintainers

Last call for reviews before we close it on Tuesday EOB European time.

/cc @kedacore/keda-maintainers

[JorTurFer]

I have taken another look and I agree with all but Maximum replicas exceeded for workload, I still think that it's not under KEDA scope because KEDA doesn't know the current needed replicas, it's the HPA Controller who knows that.
You have done a great job designing this

[tomkerkhove]

I have taken another look and I agree with all but Maximum replicas exceeded for workload, I still think that it's not under KEDA scope because KEDA doesn't know the current needed replicas, it's the HPA Controller who knows that.

I tend to disagree and say that this is implementation detail because KEDA's focus is making application autoscaling simple and using HPA is just an implementation detail that some/most KEDA end-users don't even know about.

Because of that, it's up to us to provide a unified autoscaling solution imo.

[dmeytin]

I hope we'll agree to start mult-cluster support for KEDA. It's a crucial feature for enterprise scale systems.

[tomkerkhove]

@dmeytin Feel free to post in #1587 with a proposal but it's not a top prio at the moment

[tomkerkhove]

All work items have been created and can be progress can be found on https://github.com/orgs/kedacore/projects/2/views/23?query=is%3Aopen+sort%3Aupdated-desc