Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(namespaces): increase QPS and burst rate limits #147

Merged
merged 1 commit into from
Oct 31, 2024

Conversation

nobbs
Copy link
Contributor

@nobbs nobbs commented Oct 29, 2024

This PR increases the rate limits (QPS and burst) of the namespace switching component. This fixes the issue of the namespace switcher timing out while fetching the namespaces from a cluster due to cache discovery of the kubernetes API / client-go taking too long on clusters with lots of CRDs. This is a known issue, cf. this blog post from 2022 and other users have also reported issues for similar timeouts in #127.

I've updated QPS to 50.0 and burst to 300 from the defaults specified in

const (
DefaultQPS float32 = 5.0
DefaultBurst int = 10
)

The default burst for cache discovery is set in

func setDiscoveryDefaults(config *restclient.Config) error {
to 300, but I'm not sure if it really is being properly propagated here.

Anyway, the new values 50/ 300 are based on the defaults in the kubectl code found in https://github.com/kubernetes/kubernetes/blob/5f594f42159f2ee84f805d382638f77dd5967177/staging/src/k8s.io/kubectl/pkg/cmd/cmd.go#L95-L97

Setting this substantially increased the responsiveness and I haven't since seen this issue on clusters with ~350 CRDs where the namespacer switcher previously was not able to grab namespaces at all.

@danielfoehrKn
Copy link
Owner

Thank you very much for the PR. Could you try to rebase it on master - golangci-lint is failing for this build. Master contains an updated version of golangci-lint. Thank you!

@nobbs nobbs force-pushed the fix/increase-rate-limits branch from cd26100 to f6a2e9c Compare October 30, 2024 07:30
@nobbs
Copy link
Contributor Author

nobbs commented Oct 30, 2024

Thank you very much for the PR. Could you try to rebase it on master - golangci-lint is failing for this build. Master contains an updated version of golangci-lint. Thank you!

Right, sorry - apparently I had a previous fork and forgot to pull in changes. Done, should pass the ci workflow now.

@danielfoehrKn danielfoehrKn merged commit 27dffa2 into danielfoehrKn:master Oct 31, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants