Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New rebalance #909

Merged
merged 1 commit into from
Dec 9, 2024
Merged

New rebalance #909

merged 1 commit into from
Dec 9, 2024

Conversation

leonidc
Copy link
Collaborator

@leonidc leonidc commented Oct 20, 2024

Auto rebalance is done from the separate thread

@leonidc leonidc force-pushed the new-rebalance branch 2 times, most recently from ccc24da to 45aa326 Compare October 20, 2024 09:39
@leonidc leonidc force-pushed the new-rebalance branch 8 times, most recently from d13bc55 to d3c2bc5 Compare November 10, 2024 15:33
@leonidc leonidc force-pushed the new-rebalance branch 13 times, most recently from 543a9ba to 91b5b35 Compare November 14, 2024 15:38
control/grpc.py Outdated Show resolved Hide resolved
tests/ha/auto_load_balance.sh Outdated Show resolved Hide resolved
@baum
Copy link
Collaborator

baum commented Nov 17, 2024

@leonidc great work! could you please extract the new rebalance task into a separate class and file, control/rebalance.py? This appears to be a significant piece of logic that is not directly related to serving gRPC. Rebalance-specific state could be managed within the new class, which would also help clarify which state is shared between threads. Let me know if you need any assistance with this.

To ensure thread safety:

  • Identify shared data structures and protect them with a mutex.
  • Verify that shared objects, such as cephutil, which are currently used across multiple threads, are thread-safe.

@leonidc leonidc force-pushed the new-rebalance branch 4 times, most recently from 27d0041 to e5577cb Compare November 19, 2024 08:09
control/cephutils.py Outdated Show resolved Hide resolved
control/grpc.py Show resolved Hide resolved
control/grpc.py Outdated Show resolved Hide resolved
control/grpc.py Outdated Show resolved Hide resolved
control/grpc.py Outdated Show resolved Hide resolved
control/rebalance.py Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
control/rebalance.py Show resolved Hide resolved
tests/ha/auto_load_balance.sh Show resolved Hide resolved
@leonidc leonidc force-pushed the new-rebalance branch 2 times, most recently from 313c07a to 4852e5f Compare November 27, 2024 14:11
@leonidc leonidc force-pushed the new-rebalance branch 3 times, most recently from 1947101 to 294cf0d Compare December 8, 2024 19:43
Copy link
Contributor

@gbregman gbregman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor issues still need to be changed

control/cephutils.py Outdated Show resolved Hide resolved
control/cephutils.py Outdated Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
control/rebalance.py Outdated Show resolved Hide resolved
perform auto rebalance from the separate thread,
redesign to comply fair rebalance per ana-grp and subsystem
works for 3 GWs and 3 subsystem. Just 1 GW in time performs rebalance
added test for github, merged again,
accelerate rebalance when several ANA are in Optimized state on some GW.
added protection for shared data.

put rebalance code to the new file rebalance.py
code review fixes

additional param in config file :  max_gws_in_grp

Signed-off-by: Leonid Chernin <[email protected]>
@leonidc leonidc merged commit 88e1e67 into ceph:devel Dec 9, 2024
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants