Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic configuration #16408

Merged
merged 3 commits into from
Aug 18, 2022
Merged

[fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic configuration #16408

merged 3 commits into from
Aug 18, 2022

Conversation

lordcheng10
Copy link
Contributor

@lordcheng10 lordcheng10 commented Jul 5, 2022 •
edited
Loading

Motivation

Although the configuration loadBalancerSheddingIntervalMinutes is marked as dynamic=true, it cannot be changed dynamically.
When loadSheddingTask is initialized, the shed interval is fixed:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java

Lines 1051 to 1053 in 4d64e2e

loadSheddingTask = loadManagerExecutor.scheduleAtFixedRate(
new LoadSheddingTask(loadManager),
loadSheddingInterval, loadSheddingInterval, TimeUnit.MILLISECONDS);

pulsar/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java

Lines 2087 to 2093 in 4d64e2e

@FieldContext(
category = CATEGORY_LOAD_BALANCER,
dynamic = true,
doc = "Load shedding interval. \n\nBroker periodically checks whether some traffic"
+ " should be offload from some over-loaded broker to other under-loaded brokers"
)
private int loadBalancerSheddingIntervalMinutes = 1;

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jul 5, 2022
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

@eolivelli @hangc0276 @codelipenghui PTAL,thanks!

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

5 similar comments
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

@codelipenghui @eolivelli @Jason918 PTAL,thanks!

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

Jason918
Jason918 reviewed Jul 10, 2022
View reviewed changes
Copy link
Contributor

@Jason918 Jason918 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this same as loadBalancerSheddingIntervalMinutes?

@lordcheng10
Copy link
Contributor Author

loadBalancerSheddingIntervalMinutes

@Jason918
loadBalancerSheddingIntervalMinutes cannot really be modified dynamically. It is used to initialize the cycle execution time. Once initialized, it cannot be changed：
image

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10 lordcheng10 changed the title [improve][broker] support dynamic configuration of the unload interval for shedding [fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic configuration Jul 17, 2022
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

12 similar comments
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10 lordcheng10 force-pushed the add_dynamic_config_unloadBundleIntervalMinutes branch from c25443c to 4dfb3c1 Compare August 10, 2022 07:02
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10 lordcheng10 force-pushed the add_dynamic_config_unloadBundleIntervalMinutes branch from 653daa9 to bc13bec Compare August 10, 2022 11:37
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

2 similar comments
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

Do we have a test for the loadBalancerSheddingIntervalMinutes dynamic support? If no, we need to add a test.

@codelipenghui Fixed, PTAL,thanks!

@lordcheng10 lordcheng10 force-pushed the add_dynamic_config_unloadBundleIntervalMinutes branch from bc13bec to caba922 Compare August 12, 2022 15:48
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

2 similar comments
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@lordcheng10
Copy link
Contributor Author

Do we have a test for the loadBalancerSheddingIntervalMinutes dynamic support? If no, we need to add a test.

Fixed.PTAL,thanks! @codelipenghui

@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@lordcheng10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@codelipenghui codelipenghui merged commit e4dcf5a into apache:master Aug 18, 2022
@Technoboy- Technoboy- modified the milestones: 2.12.0, 2.11.0 Aug 18, 2022
Technoboy- pushed a commit that referenced this pull request Aug 18, 2022
@lordcheng10 @Technoboy-
[fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic con…
d8c32ca 
…figuration (#16408)
Jason918 pushed a commit that referenced this pull request Sep 2, 2022
@lordcheng10 @Jason918
[fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic con…
18cd7f4 
…figuration (#16408)

(cherry picked from commit e4dcf5a)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Sep 2, 2022
@lordcheng10 @nicoloboschi
[fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic con…
3275c82 
…figuration (apache#16408)

(cherry picked from commit e4dcf5a)
(cherry picked from commit 18cd7f4)
@gaoran10 gaoran10 mentioned this pull request Sep 14, 2022
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aloyszhang aloyszhang aloyszhang approved these changes

@codelipenghui codelipenghui codelipenghui approved these changes

@Jason918 Jason918 Jason918 approved these changes

Assignees

@lordcheng10 lordcheng10

Labels
area/broker cherry-picked/branch-2.10 cherry-picked/branch-2.11 doc-not-needed Your PR changes do not impact docs release/2.10.2 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
2.11.0
Development

Successfully merging this pull request may close these issues.
5 participants
@lordcheng10 @Jason918 @codelipenghui @aloyszhang @Technoboy-