This repository has been archived by the owner on Jun 19, 2022. It is now read-only.
Add KEDA-scaling for Sources #423
Labels
priority/2
Nice to have feature but doesn't block current release defined by release/*
Milestone
Problem
Our PubSub-based Sources are Pull-based. Given that our receive adapter polls for new messages, it’s hard to decide when to scale it out. In other words, the draining of the PubSub subscriptions will eventually become a throughput bottleneck.
There are some initial talks in the Sources WG regarding Pull-based sources and how to scale them. @aslom suggested KEDA. Either with KEDA or something else, we will need to read metrics from Stackdriver to see how “congested” the subscription is and inform the scaling mechanism. See details on metrics here.
Exit Criteria
Being able to scale out our receive adapters.
Time Estimate (optional):
How many developer-days do you think this may take to resolve?
2-3 weeks
The text was updated successfully, but these errors were encountered: