feat(subscription): Support specified pk read log store #19274
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xxhZs
force-pushed
the
xxh/timeout-base-recovery-cursor
branch
from
d53bf7a to
7f4e0cd
Compare
hzxa21
reviewed
Nov 11, 2024
src/expr/impl/src/scalar/cast.rs
Outdated
| @@ -47,6 +47,15 @@ where | |||
| }) | |||
| } | |||
|
|
|||
| #[function("cast(varchar) -> struct", type_infer = "unreachable")] | |||
| pub fn str_parse_struct(elem: &str, ctx: &Context) -> Result<StructValue> { | |||
There was a problem hiding this comment.
Is this change relevant to this PR?
There was a problem hiding this comment.
is to enable recovery from a single point, his logic = select * from table where (pk1,pk2,pk3)>(1,1,1);
hzxa21
reviewed
Dec 20, 2024
| @@ -740,6 +742,52 @@ impl SubscriptionCursor { | |||
| Ok(row) | |||
| } | |||
|
|
|||
| pub fn process_output_desc_row( | |||
There was a problem hiding this comment.
Given that the Field and the row structure (i.e. which columns in which idx) are fixed, there are several things can be simplified in this method:
- We can use
row.project(<pk_col_idx_in_row>)to get the pk instead of doing the filter_map and map checking every time. - We only need to calculate the
pk_fieldsonce and cache it in SubscriptionCursor instead of doing it every time when we get rows.
| @@ -770,33 +818,116 @@ impl SubscriptionCursor { | |||
| } | |||
| }) | |||
| .collect::<Vec<_>>(); | |||
| let output_col_idx_with_out_hidden = output_col_idx | |||
| let max_split_range_gap = context.session_ctx().config().max_split_range_gap() as u64; | |||
There was a problem hiding this comment.
Why do we need this? I think we only have one range here.
There was a problem hiding this comment.
The call to the split_to_scan_ranges method requires that, here, it won't work
hzxa21
reviewed
Jan 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
In this pr we will save the pk consumed by the cursor every time, and support the specified pk consumption log, this but this pr does not use them at present, we will open the cursor recovery in the next pr
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.