Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use parallel podManagementPolicy for stateful sets #2054

Closed
fmassot opened this issue Oct 1, 2022 · 1 comment
Closed

Use parallel podManagementPolicy for stateful sets #2054

fmassot opened this issue Oct 1, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@fmassot
Copy link
Contributor

fmassot commented Oct 1, 2022

Currently, scaling indexers and searchers is slow as we use the OrderedReady pod management policy (it's the default). It seems very interesting to use the parallel policy to start all pods at the same time, at least this is very cool for a demo ^^.

For searchers, it seems only beneficial to use the parallel policy. Starting a searcher is very light, there is no risk to overload the system by making many queries on the metastore.

For indexers, there is a risk to overload the metastore. On the contrary, by starting together 10 nodes, there is a chance that we will avoid making several rebalances.

I would go for the parallel policy at least for searchers, for indexers, we could progressively start indexers in groups of 10 so that you don't have to wait that much when you need to start 100 indexers.

See docs for more details.

@guilload
Copy link
Member

guilload commented Nov 26, 2022

Closing in favor of issue linked above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants