storage: include memtable in for_update snapshots #6146
Conversation
Signed-off-by: Wish <[email protected]>
…hot-for-update Signed-off-by: Wish <[email protected]>
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
ti-chi-bot
added
release-note-none
Signed-off-by: Wish <[email protected]>
Signed-off-by: Wish <[email protected]>
…wish/tiflash into wenxuan/unify-snapshot-for-update
|
/run-all-tests |
| WriteBatches & wbs) const | ||
| { | ||
| auto [new_mem_files, flushed_mem_files] = mem_table_set->diffColumnFiles({}); | ||
| RUNTIME_CHECK(flushed_mem_files.empty()); |
There was a problem hiding this comment.
Why flushed_mem_files should be empty? Can you add some comments?
Signed-off-by: Wish <[email protected]>
ti-chi-bot
added
status/LGT2
|
/merge |
|
@JaySon-Huang: It seems you want to merge this PR, I will help you trigger all the tests: /run-all-tests You only need to trigger If you have any questions about the PR merge process, please refer to pr process. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
This pull request has been accepted and is ready to merge. Commit hash: aaf0243
|
ti-chi-bot
added
the
status/can-merge
|
The failed unit test is caused by a newly added test merged into master. I will fix it soon. |
Signed-off-by: Wish <[email protected]>
ti-chi-bot
removed
the
status/can-merge
|
/merge |
|
@breezewish: It seems you want to merge this PR, I will help you trigger all the tests: /run-all-tests You only need to trigger If you have any questions about the PR merge process, please refer to pr process. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
This pull request has been accepted and is ready to merge. Commit hash: c88f947
|
ti-chi-bot
added
the
status/can-merge
|
@breezewish: Your PR was out of date, I have automatically updated it for you. At the same time I will also trigger all tests for you: /run-all-tests If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
What problem does this PR solve?
Issue Number: close #6045
Problem Summary:
What is changed and how it works?
Include memtable in the ForUpdate snapshot. The appendable ColumnFileTiny will be changed to not appendable when ForUpdate snapshot is created.
Consequently, the DeltaMerge, Merge and Split process is changed. Take DeltaMerge as an example, the new process looks like:
In the apply stage, we only include new memtable CFs and new persisted CFs in the new segment. There may be CFs flushed from memtable into the persisted. These CFs will not be included. This is handled by the new
DeltaVS::cloneNewlyAppendedColumnFiles. Here is a visualization of its step:merge(start, end)in the DeltaMergeStore test framework, to merge segments in[start, end).Check List
Tests
Side effects
Documentation
Release note