Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Honor goals when increasing topic replication factor. #789

Merged
merged 2 commits into from
Jul 10, 2019
Merged

Honor goals when increasing topic replication factor. #789

merged 2 commits into from
Jul 10, 2019

Conversation

kidkun
Copy link

@kidkun kidkun commented Jun 21, 2019

Fix issue #713 and issue #722

@kidkun kidkun requested a review from efeg June 21, 2019 05:37
efeg
efeg reviewed Jun 22, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch! Started the review and left some initial comments.

cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Show resolved Hide resolved
@kidkun kidkun force-pushed the rf_change_improvement branch from d2bf5de to a5a9060 Compare June 22, 2019 06:20
efeg
efeg requested changes Jun 25, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates -- added more comments.

cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControlUtils.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/GoalOptimizer.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/GoalOptimizer.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/GoalOptimizer.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/OptimizationOptions.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/CapacityGoal.java Outdated Show resolved Hide resolved
efeg
efeg requested changes Jun 26, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates! Left some comments.

cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/GoalOptimizer.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/GoalOptimizer.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/GoalOptimizer.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/model/ClusterModel.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/model/ClusterModel.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
...rc/test/java/com/linkedin/kafka/cruisecontrol/analyzer/ExcludedBrokersForLeadershipTest.java Show resolved Hide resolved
...rol/src/test/java/com/linkedin/kafka/cruisecontrol/analyzer/ReplicationFactorChangeTest.java Outdated Show resolved Hide resolved
...rol/src/test/java/com/linkedin/kafka/cruisecontrol/analyzer/ReplicationFactorChangeTest.java Outdated Show resolved Hide resolved
cruise-control/src/test/java/com/linkedin/kafka/cruisecontrol/common/DeterministicCluster.java Show resolved Hide resolved
efeg
efeg requested changes Jun 27, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates! Left more comments.

cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/model/ClusterModel.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/model/ClusterModel.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/model/ClusterModel.java Show resolved Hide resolved
...l/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/ReplicaDistributionGoal.java Show resolved Hide resolved
...l/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/ReplicaDistributionGoal.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/CapacityGoal.java Outdated Show resolved Hide resolved
.../main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/TopicReplicaDistributionGoal.java Show resolved Hide resolved
...rol/src/test/java/com/linkedin/kafka/cruisecontrol/analyzer/ReplicationFactorChangeTest.java Show resolved Hide resolved
@kidkun kidkun force-pushed the rf_change_improvement branch from cce2958 to 380dd08 Compare June 28, 2019 17:24
efeg
efeg requested changes Jun 29, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updates -- left some comments.

cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/KafkaCruiseControl.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/model/ClusterModel.java Outdated Show resolved Hide resolved
cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/RackAwareGoal.java
// Satisfy rack awareness requirement.
SortedSet<Replica> replicas = new TreeSet<>(broker.replicas());
for (Replica replica : replicas) {
if ((broker.isAlive() && satisfiedRackAwareness(replica, clusterModel))
|| shouldExclude(replica, excludedTopics)) {
|| shouldExclude(replica, excludedTopics)
|| (onlyMoveImmigrantReplicas && !replica.isImmigrant())) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! -- thanks for creating #802

.../src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/ResourceDistributionGoal.java Show resolved Hide resolved
@kidkun kidkun mentioned this pull request Jul 6, 2019
Closed
efeg
efeg requested changes Jul 8, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update -- added a comment.

...ntrol/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/ReplicaCapacityGoal.java Show resolved Hide resolved
@kidkun kidkun force-pushed the rf_change_improvement branch from f5e593f to 2485ce3 Compare July 9, 2019 22:19
efeg
efeg approved these changes Jul 10, 2019
View reviewed changes
Copy link
Collaborator

@efeg efeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kidkun kidkun merged commit 4b2b2f4 into linkedin:master Jul 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@efeg efeg efeg approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kidkun @efeg