ZOOKEEPER-2260: Paginated getChildren call #50
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mprimi
force-pushed
the
ZOOKEEPER-2260
branch
from
cfc4e2f to
32a7eb6
Compare
| @@ -69,6 +69,8 @@ | |||
|
|
|||
| public final int deleteContainer = 20; | |||
|
|
|||
| public final int getChildrenPaginated = 71; | |||
There was a problem hiding this comment.
Instead of random number, I'd prefer to use next sequence getChildrenPaginated = 21;
|
@rakeshadr Thanks for the comments. Just pushed a new commit that addresses all of them (except the new suggested iterator interface), let me know what you think. I'll keep the patch sets incremental but squash at the very end, if this gets accepted. Still to do:
|
|
@rakeshadr thanks for the comments, just uploaded a new patch |
mprimi
force-pushed
the
ZOOKEEPER-2260
branch
from
380f175 to
7e30292
Compare
mprimi
force-pushed
the
ZOOKEEPER-2260
branch
from
ae9977a to
0484ea9
Compare
|
Jenkins test this please |
added 7 commits
April 11, 2017 12:53
Motivation Being able to get children with pagination. Two main benefits: - Allow listing children that normally would exceed the buffer limit - Get stat for children in the same call as the list Modifications - Implemetation of DataTree#getPaginatedChildren - Added the call to the zk client - Handle the call in the FinalRequestProcessor - Added PathWithStat, GetChildrenPaginatedRequest and GetChildrenPaginatedResponse to the protos - Added tests for pagination and watch
- Follow convention for Stat field in jute PathWithStat - Change opcode for getChildrenPaginated - Rename ZooKeeper#getChildrenPaginated -> ZooKeeper#getChildren - Better javadoc for ZooKeeper#getChildren - Use znode creation zkid for comparison and sorting of children - Add test timeout - Use wait() rather than sleep() in tests - Update test to make sure node modification does not affect order or creates duplicates
…ater) - Adding iterator interface, and leaving the lower-level interface in place for the moment - Adding test for iterator interface
- s/minZkid/minZxid - Introduce RemoteIterator interface - Switch ChildrenBatchIterator to RemoteIterator - New test for pagination - Change ChildrenBatchIterator logic as tests exposed serious issues
…ater) - Updating RemoteIterator interface to avoid throwing from hasNext() - Updating ChildrenIterator for new interface, and simplifying implementation using deque
- Adding to CHANGES file - s/zkid/czxid - Better javadoc and more comments - Minor refactoring and renaming - Add test for empty children list and iteration beyond last element
Previous iteration would not work properly when children have non-unique creation id (i.e. as result of multi()). Add an extra offset parameter in request, used by client to track how many children with a given creation zxid it has seen. Server uses this to skip. Also changes semantic of minzkid: now returns children with creation >= (before was strictly >) Adding tests that cover these new behaviors.
huizhilu
added a commit
to huizhilu/zookeeper
that referenced
this pull request
Nov 11, 2020
Apply PR of paginated getChildren call(apache#50)
huizhilu
added a commit
to linkedin/zookeeper
that referenced
this pull request
Nov 12, 2020
Apply the patch of ZOOKEEPER-2260: paginated getChildren call (apache#50) And adjust code style to pass the check style verification Credit goest to the original author of ZOOKEEPER-2260
junkaixue
pushed a commit
to linkedin/zookeeper
that referenced
this pull request
Dec 21, 2020
Apply the patch of ZOOKEEPER-2260: paginated getChildren call (apache#50) And adjust code style to pass the check style verification Credit goest to the original author of ZOOKEEPER-2260
narendly
pushed a commit
to linkedin/zookeeper
that referenced
this pull request
Nov 23, 2021
Apply the patch of ZOOKEEPER-2260: paginated getChildren call (apache#50) And adjust code style to pass the check style verification Credit goest to the original author of ZOOKEEPER-2260
|
@anmolnar may I ask the reason this PR gets closed? From JIRA and PR it seems the author keeps working on this issue over years and there's a consensus it's a good feature to have. |
mgao0
pushed a commit
to mgao0/zookeeper
that referenced
this pull request
Jan 25, 2022
…e#50) ZooSave & ZooRestore, a new feature that enables server-side ZooKeeper snapshot backup, has its own naming scheme for naming backup files (mostly adding an ending ZXID interval). However, this naming scheme was not compatible with the newly-introduced feature to the upstream ZooKeeper main branch for snapshot compression. This is because snapshot compression appends a StreamMode extension if it is enabled. This commit modifies the backup naming logic so that it considers the case where snapshot compression is enabled and thus the snapshot files on the server have a StreamMode extension. It also updates the BackupFileInfo initialization logic so that when the backed up snapshots are read back in, their standard names are populated correctly. A test is added in BackupManagerTest: testBackupFileNamingOnStreamingMode().
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/ZOOKEEPER-2260