Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improve][broker] PIP-192: Implement broker registry #7

Open
wants to merge 152 commits into
base: master
Choose a base branch
from
Open

[improve][broker] PIP-192: Implement broker registry #7

wants to merge 152 commits into from

Conversation

Demogorgon314
Copy link
Owner

@Demogorgon314 Demogorgon314 commented Dec 7, 2022 •
edited
Loading

Master Issue: apache#16691

Motivation

We will start raising PRs to implement PIP-192, apache#16691

Modifications

This PR should not impact the existing broker load balance behavior.

For the PIP-192 project, this PR

  • Implemented broker registry.
  • Added units test.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@Demogorgon314 Demogorgon314 self-assigned this Dec 7, 2022
@Demogorgon314 Demogorgon314 mentioned this pull request Dec 8, 2022
Merged
4 tasks
tisonkun pushed a commit that referenced this pull request Dec 10, 2022
@Pomelongan
[cleanup][broker]remove methods no longer used in AdminResource #7 (a…
ca648b2 
…pache#17969)
tisonkun and others added 27 commits December 29, 2022 19:08
@tisonkun
[improve][sec] Suppress false positive OWASP reports (apache#19105)
62a2058 
Signed-off-by: tison <[email protected]>
@tisonkun
[refactor] Remove docs from the main repo (apache#19100)
c13f389
@AnonHxy
[fix][broker] Fix estimateBacklogFromPosition if position is greater …
c611a1a 
…than the greatest ledgerId (apache#19017)
@Technoboy-
[fix][ci] Fix pulsar shell license issue (apache#19101)
ed33fb3
@maxsxu
[fix][cli] Fix wrong description of pulsar-admin flags (apache#19108)
36cff70 
Signed-off-by: Max Xu <[email protected]>
Motivation
pulsar-admin sources list has the wrong parameters description.
Modifications
Should be pulsar-admin sources list --namespace <The source's namespace> --tenant <The source's tenant>
@vineeth1995
[improve][admin,broker] Add option to unloadNamespaceBundle with bund…
e194c01 
…le Affinity broker url (apache#18663)

Co-authored-by: Vineeth <[email protected]>
@aymkhalil
[fix][admin] Keep new inputSpecs when updating sink configs (apache#1…
b3f3e0b 
…9082)
@nicoloboschi
[fix][sec] Upgrade jettison to 1.5.3 (apache#19038)
1736396
@lhotari
[fix][build] Resolve OWASP Dependency Check false positives (apache#1…
f912fb3 
…9120)
@coderzc
[feat][broker][PIP-195]Implement Filter out all delayed messages and …
9ec1d07 
…skip them when reading messages from bookies - part7 (apache#19035)
@michaeljmarshall
[cleanup][broker] Fix ClusterDataImpl#clone and add test (apache#19126)
a2ec02d
@coderzc
[fix][broker][PIP-195] fix cursor skip read (apache#19124)
3bb93cb
@heesung-sn
[fix][monitor] Fix the partitioned publisher topic stat aggregation b…
8790ed1 
…ug (apache#18807)
@nicoloboschi
[fix][broker] Topic could be in fenced state forever if deletion fails (
a6516a8 
apache#19129)
@codelipenghui
[improve][schema] Do not print error log with stacktrace for 404 (apa…
3afc291 
…che#19130)
@coderzc
[feat][broker][PIP-195] Implement BucketDelayedDeliveryTrackerFactory…
4b8f447 
… and load BucketDelayedDeliveryTracker - part6 (apache#17756)
[improve][io] Elasticsearch sink: Support loading config from secrets (…
593fcb8 
…apache#18986)
@lhotari
[improve][ci] Improve CI ssh access in forks, don't fail build if set…
4028ad3 
…ting up ssh access fails (apache#19127)
@michaeljmarshall
[revert] "[fix][broker] change name limitTime to limitTimeInSec (apac…
c862356 
…he#19053)" (apache#19152)
@aloyszhang
[fix][broker] fix ttl expiration block due to no-recoverable exceptio…
badd69b 
…n even if autoSkipNonRecoverableData=true (apache#19132)
@mattisonchao
[improve][broker] Add oshi library to help control OS resources. (apa…
96f8262 
…che#18990)
@mattisonchao
[fix][broker] Reject create non existent persistent partitions. (apac…
56a7b89 
…he#19086)
@gaozhangmin
[improve][admin]internalGetMessageById shouldn't be allowed on partit…
b05fddb 
…ioned topic (apache#19013)

Co-authored-by: gavingaozhangmin <[email protected]>
@nicoloboschi
[cleanup][txn] Use MLTransactionMetadataStore in integration tests (a…
6335fa1 
…pache#19140)
@mattisonchao
[improve][broker] Add parameter check for create/update cluster. (apa…
e829672 
…che#19151)
@nicoloboschi
[fix][txn] Always send correct transaction id in end txn response (ap…
90764e7 
…ache#19137)
@aloyszhang
[improve][cli] Expose updateLocalTopicOnly to UpdatePartitionedCmd (a…
9ef54fd 
…pache#19162)
liangyepianzhou and others added 26 commits January 29, 2023 18:19
@liangyepianzhou
[improve][txn]Clean up redundant logic in transaction client (apache#…
c17f355 
…19345)
@Pomelongan
[improve][broker] Optimize the logic of internalCreatePartitionedTopic (
8ea41f1 
apache#18666)
@poorbarcode
[fix] [ml] The atomicity of multiple fields of ml is broken (apache#1…
e2851da 
…9346)
@mattisonchao
[fix][client] Fix reader listener can't auto ack with pooled message. (…
1b2fa1f 
…apache#19354)
@gaozhangmin
[improve][admin] Improve partitioned-topic condition evaluation (apac…
3bab099 
…he#19015)

Co-authored-by: gavingaozhangmin <[email protected]>
@michaeljmarshall
[fix][client] Set fields earlier for correct ClientCnx initialization (…
3d8b52a 
…apache#19327)
@lhotari
[improve][test] Refactor TestPulsarService to PulsarTestContext and a…
d9a71d7 
…dd support for starting (apache#19337)
@mattisonchao
[improve][txn] Move checked exception into builder when newTransactio…
4165090 
…n. (apache#19356)
@poorbarcode
[fix] [ml] messagesConsumedCounter of NonDurableCursor was initialize…
fc9e8bf 
…d incorrectly (apache#19355)
@crossoverJie
[improve][broker] Add lowerBoundarySheddingEnabled conf (apache#19351)
785fb82
@poorbarcode
[fix] [broker] print warn log if compaction task skipped cause by ex (a…
72b2e7e 
…pache#19360)
@heesung-sn
[improve][broker] PIP-192 Added ServiceUnitStateCompactionStrategy (a…
1cd1aef 
…pache#19045)
@Demogorgon314
Implement broker registry for new load manager
66b0d7c
@Demogorgon314
Refactor
723f154
@Demogorgon314
Cleanup the listeners first when close
3fe33e1
@Demogorgon314
Add more unit tests
c6f8f13
@Demogorgon314
Add more unit tests
ec723fb
@Demogorgon314
Add test group
01335ab
@Demogorgon314
Change the lookup address to broker id
582925b
@Demogorgon314
Remove cache and add more tests
b66cd7f
@Demogorgon314
Address reviewer's comments
48c1cbc
@Demogorgon314
Add new available broker lookup data method
1d20a5f
@Demogorgon314
Add log to warn empty lookup data
99ba81b
@Demogorgon314
Use state field to store states.
0fa3392
@Demogorgon314
Use try finally when close
d4c2326
@Demogorgon314
Merge master into current branch
fdb9ebd
@Demogorgon314 Demogorgon314 force-pushed the Demogorgon314/Implement_broker_registry branch from 38c0beb to fdb9ebd Compare February 1, 2023 03:27
@github-actions
Copy link

github-actions bot commented Mar 4, 2023

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Mar 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Demogorgon314 Demogorgon314

Labels
Stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.