[improve][broker][PIP-318] Support not retaining null-key message during topic compaction #21578
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
coderzc
force-pushed
the
compaction_not_retain_nullkey
branch
from
7fbe7b5 to
cb1b764
Compare
coderzc
changed the title
pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
Outdated
Show resolved
Hide resolved
coderzc
force-pushed
the
compaction_not_retain_nullkey
branch
from
ad93df2 to
4ff06cf
Compare
pulsar-broker/src/main/java/org/apache/pulsar/client/impl/RawBatchConverter.java
Show resolved
Hide resolved
Technoboy-
approved these changes
Nov 16, 2023
mattisonchao
approved these changes
Nov 16, 2023
coderzc
force-pushed
the
compaction_not_retain_nullkey
branch
from
a1ae5f1 to
1ca9ce5
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #21578 +/- ##
============================================
+ Coverage 73.27% 73.29% +0.02%
- Complexity 32676 32705 +29
============================================
Files 1892 1892
Lines 140632 140721 +89
Branches 15467 15489 +22
============================================
+ Hits 103044 103146 +102
+ Misses 29497 29470 -27
- Partials 8091 8105 +14
Flags with carried forward coverage won't be shown. Click here to find out more.
|
coderzc
added
type/bug
coderzc
added a commit
to coderzc/pulsar
that referenced
this pull request
Nov 21, 2023
…during topic compaction (apache#21578)
4 tasks
coderzc
added
type/enhancement
Technoboy-
pushed a commit
that referenced
this pull request
Nov 22, 2023
coderzc
added a commit
to coderzc/pulsar
that referenced
this pull request
Dec 4, 2023
Merged
4 tasks
Technoboy-
pushed a commit
that referenced
this pull request
Dec 4, 2023
coderzc
added a commit
to coderzc/pulsar
that referenced
this pull request
Dec 4, 2023
Merged
5 tasks
coderzc
added a commit
to coderzc/pulsar
that referenced
this pull request
Dec 5, 2023
merlimat
reviewed
Dec 6, 2023
| @@ -538,6 +538,9 @@ brokerServiceCompactionThresholdInBytes=0 | |||
| # If the execution time of the compaction phase one loop exceeds this time, the compaction will not proceed. | |||
| brokerServiceCompactionPhaseOneLoopTimeInSeconds=30 | |||
|
|
|||
| # Whether retain null-key message during topic compaction | |||
| topicCompactionRemainNullKey=false | |||
There was a problem hiding this comment.
Suggested change
| topicCompactionRemainNullKey=false | |
| topicCompactionRetainNullKey=false |
There was a problem hiding this comment.
@coderzc I just saw this typo in the config key :)
There was a problem hiding this comment.
Sorry, I open a new PR to fix it. #21690
4 tasks
Technoboy-
pushed a commit
that referenced
this pull request
Dec 14, 2023
… compaction (#21578) (#21665) Co-authored-by: fengyubiao <[email protected]>
nikhil-ctds
pushed a commit
to datastax/pulsar
that referenced
this pull request
Dec 20, 2023
srinath-ctds
pushed a commit
to datastax/pulsar
that referenced
this pull request
Dec 20, 2023
coderzc
added a commit
that referenced
this pull request
Jan 4, 2024
nodece
pushed a commit
to nodece/pulsar
that referenced
this pull request
Feb 23, 2024
mukesh-ctds
pushed a commit
to datastax/pulsar
that referenced
this pull request
Mar 1, 2024
… compaction (apache#21578) (apache#21601) (cherry picked from commit 6ddf83d)
mukesh-ctds
pushed a commit
to datastax/pulsar
that referenced
this pull request
Mar 6, 2024
… compaction (apache#21578) (apache#21601) (cherry picked from commit 6ddf83d)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PIP: #21541
Motivation
Implement #21541
Modifications
topicCompactionRetainNullKeyin the broker.conf, the default value is false.Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: