Implements concurrent segmented queues #638
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 9309d92.
andsel
force-pushed
the
feature/disk_queues
branch
from
c834b24
to
a680d74
Compare
…data in the same buffer and writing that raw data
…load that crossed 2 segments
…PayloadFromSegments
|
Is there a way to discard/close a queue? For example when the client reconnects with cleanSession=true? |
|
@hylkevds good to remind it, I haven't yet thought about :-) |
… offsets to the respective physical segments, which permits to have not monitically incrementing segments and put the basis for segments reuse and limiting holes (external fragmentation) in page files
…when reload from checksum file
1 task
- one that advanced twice a pointer (newSegmentPointer) - fixed calculation of availableDataLength in a segment while writing payload across two segments
|
Superseeded by #691 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release notes
Introduce new persistent queue implementation based on concurrent access to segmented queues.
What does this PR do?
All queues are saved in segments, each segment in stored in a memory mapped region named page where each page can contains multiple segments of different queues. The access to the segment is based on CAS operator on such named
SegmentPointerobjects, when the CAS is not possible a retry is repeated until newSegmentaccess is required, in which case the access is managed by a usual lock guarded sectionWhy is it important/What is the impact to the user?