STATUS | Experimental |
Sponsoring WG | Eventing |
Warning: Still under development. Not meant for production deployment.
To enable KEDA Autoscaling of Knative Event Sources (and other components in the
future) there is a separate controller implemented, ie. no hard dependency in
Knative. This controller watches CustomResourcesDefinitions
resources in the
cluster. If the newly installed CRD is supported by this controller, it creates
a new dynamic controller which watches custom objects of kind specified by the
CRD.
Currently there is support for Kafka Source. We also have experimental support for RabbitMQ Broker and Redis Stream Source.
User can enable and configure autoscaling on a particular Source or Broker by a set of annotations.
metadata:
annotations:
autoscaling.knative.dev/class: keda.autoscaling.knative.dev
autoscaling.knative.dev/minScale: "0"
autoscaling.knative.dev/maxScale: "5"
keda.autoscaling.knative.dev/pollingInterval: "30"
keda.autoscaling.knative.dev/cooldownPeriod: "30"
# Kafka Source
keda.autoscaling.knative.dev/kafkaLagThreshold: "10"
# Redis Stream Source
keda.autoscaling.knative.dev/redisStreamPendingEntriesCount: "5"
autoscaling.knative.dev/class: keda.autoscaling.knative.dev
- needs to be specified on a Source to enable KEDA autoscalingautoscaling.knative.dev/minScale
- minimum number of replicas to scale down to. Default:0
autoscaling.knative.dev/maxScale
- maximum number of replicas to scale out to. Default:50
keda.autoscaling.knative.dev/pollingInterval
- interval in seconds KEDA uses to poll metrics. Default:30
keda.autoscaling.knative.dev/cooldownPeriod
- period of time in seconds KEDA waits until it scales down. Default:300
keda.autoscaling.knative.dev/kafkaLagThreshold
- only for Kafka Source, refers to the lag on the current consumer group that's used for scaling (1<->N). Default:10
keda.autoscaling.knative.dev/kafkaActivationLagThreshold
- only for Kafka Source, refers to the lag on the current consumer group that's used for activation (0<->1). Default:0
keda.autoscaling.knative.dev/rabbitMQQueueLength
- only for RabbitMQ broker, refers to the target value for number of messages in a RabbitMQ brokers trigger queue:1
keda.autoscaling.knative.dev/redisStreamPendingEntriesCount
- only for Redis Stream Source, refers to the target value for number of entries in the Pending Entries List for the specified consumer group in the Redis Stream. Default:5
It is needed to install KEDA v2, which is using different namespace for it's
CRDs (keda.k8s.io
-> keda.sh
). KEDA v1 is not supported.
To install KEDA, please follow installation instructions.
Confirm there are 2 pods running in keda
namespace:
$ kubectl get pods -n keda
NAME READY STATUS RESTARTS AGE
keda-metrics-apiserver-7cf7765dc8-k9lnc 1/1 Running 0 5m2s
keda-operator-55658855fc-rc9rb 1/1 Running 0 5m3s
export KO_DOCKER_REPO=...
ko apply -f config/
Confirm there is a pod of eventing-autoscaler-keda
controller running in
knative-eventing
namespace:
$ kubectl get pod -n knative-eventing -l app=eventing-autoscaler-keda-controller
NAME READY STATUS RESTARTS AGE
eventing-autoscaler-keda-controller-69bf565cb8-r5922 1/1 Running 0 3m7s
-
Set up Kafka Cluster, eg. use Strimzi operator
-
Install Knative Serving and Eventing
-
Install Knative Eventing Kafka Source
-
Create
KafkaSource
resource, with annotationautoscaling.knative.dev/class: keda.autoscaling.knative.dev
. There are other KEDA related annotations, see the example:
apiVersion: sources.knative.dev/v1alpha1
kind: KafkaSource
metadata:
name: kafka-source
namespace: default
annotations:
autoscaling.knative.dev/class: keda.autoscaling.knative.dev
autoscaling.knative.dev/minScale: "0"
autoscaling.knative.dev/maxScale: "5"
keda.autoscaling.knative.dev/pollingInterval: "30"
keda.autoscaling.knative.dev/cooldownPeriod: "30"
keda.autoscaling.knative.dev/kafkaLagThreshold: "10"
spec:
consumerGroup: knative-group
bootstrapServers:
- my-cluster-kafka-bootstrap.openshift-operators:9092
topics:
- my-topic
sink:
ref:
apiVersion: serving.knative.dev/v1
kind: Service
name: event-display
- Check that
ScaledObject
was created for thisKafkaSource
:
$ kubectl get scaledobjects
NAME SCALETARGETKIND SCALETARGETNAME TRIGGERS AUTHENTICATION READY ACTIVE AGE
so-f87369e5-c320-4f44-b23a-8c535a523e3a apps/v1.Deployment kafkasource-kafka-source-f87369e5-c320-4f44-b23a-8c535a523e3a kafka True False 6m5s
-
Install Knative Serving and Eventing
-
Install RabbitMQ Broker
-
Install a Broker / Trigger and sources as directed in the above guide.
-
Enable the autoscaler by applying the KEDA patch:
kubectl patch broker default --type merge --patch '{"metadata": {"annotations": {"autoscaling.knative.dev/class": "keda.autoscaling.knative.dev"}}}'
- Check that the scaled resources were created and are ready
vaikas-a01:eventing-autoscaler-keda vaikas$ kubectl get triggerauthentications
NAME PODIDENTITY SECRET ENV
default-trigger-auth default-broker-rabbit
vaikas-a01:eventing-autoscaler-keda vaikas$ kubectl get scaledobjects
NAME SCALETARGETKIND SCALETARGETNAME TRIGGERS AUTHENTICATION READY ACTIVE AGE
ping-trigger apps/v1.Deployment ping-trigger-dispatcher rabbitmq default-trigger-auth True True 14m