refactor(meta): generalize replace table to replace job #18935

xxchan · 2024-10-16T06:46:18Z

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Style improvements (e.g., extract logic into methods) to make the code easier to understand and able to generalize to replace job.

prepare for #19063

Checklist

I have written necessary rustdoc comments
I have added necessary unit tests and integration tests
I have added test labels as necessary. See details.
I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
All checks passed in ./risedev check (or alias, ./risedev c)
My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

xxchan · 2024-10-16T06:46:37Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

xxchan · 2024-12-03T05:33:42Z

src/meta/src/controller/fragment.rs

+    /// - For source, it should have one `Source` fragment.
+    ///
+    /// In other words, it's the `MView` fragment if it exists, otherwise it's the `Source` fragment.
+    pub async fn get_root_fragments(


The logic is not related whether it is "upstream", so renamed.

xxchan · 2024-12-03T05:34:30Z

src/meta/src/controller/fragment.rs

+    }
+
+    /// Get the downstream fragments connected to the specified job.
+    pub async fn get_downstream_fragments(


The name Chain fragments is outdated. Just call it downstream_fragments. I guess there's no confusion.

xxchan · 2024-12-03T05:35:01Z

src/meta/src/controller/fragment.rs

-                (fragment_id, strategy)
-            })
-            .collect();
+        let (root_fragment, actors) = self.get_root_fragment(job_id).await?;


Refactored from mview_fragment -> root_fragment, to make it more general.

xxchan · 2024-12-03T06:15:41Z

Merge activity

Dec 3, 2:15 PM GMT+8: Graphite rebased this pull request after merging its parent, because this pull request is set to merge when ready.
Dec 3, 5:09 PM GMT+8: Graphite couldn't merge this PR because it failed for an unknown reason (This repository has GitHub's merge queue enabled, which is currently incompatible with Graphite).

yezizp2012

LGTM

github-actions bot added type/feature ci/run-e2e-single-node-tests labels Oct 16, 2024

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from 7ee40ee to f74b951 Compare October 16, 2024 06:52

xxchan changed the base branch from main to xxchan/share-source-config October 16, 2024 06:52

xxchan mentioned this pull request Oct 16, 2024

feat: enable shared source in session variable by default, and add cluster-level config to disable #18749

Merged

12 tasks

xxchan force-pushed the xxchan/share-source-config branch from 899a05f to 7326f9d Compare October 17, 2024 06:52

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from f74b951 to 9c3cea6 Compare October 17, 2024 06:52

xxchan force-pushed the xxchan/share-source-config branch from 7326f9d to 2bf03cd Compare October 18, 2024 00:49

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from 9c3cea6 to a7949b2 Compare October 18, 2024 00:49

xxchan force-pushed the xxchan/share-source-config branch from 2bf03cd to 06bd105 Compare October 21, 2024 01:40

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from a7949b2 to 975cc21 Compare October 21, 2024 01:40

Base automatically changed from xxchan/share-source-config to main October 21, 2024 02:34

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch 2 times, most recently from 90d1fab to 1c9feb2 Compare October 23, 2024 02:47

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch 2 times, most recently from d43372e to 9bf7633 Compare October 30, 2024 02:19

xxchan changed the base branch from main to xxchan/refactor-ddl October 30, 2024 02:19

This was referenced Oct 30, 2024

refactor(meta): simplify stream job table/source id assignment #19171

Merged

refactor: rename ConnectorSchema/SourceSchema to FormatEncode #19174

Merged

github-actions bot added the ci/run-e2e-test-other-backends label Oct 30, 2024

xxchan force-pushed the xxchan/refactor-ddl branch from f33a42a to d43598b Compare November 4, 2024 03:59

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from 9bf7633 to c117687 Compare November 4, 2024 03:59

xxchan force-pushed the xxchan/refactor-ddl branch from d43598b to 0012e27 Compare November 4, 2024 05:50

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from c117687 to 9be32c1 Compare November 4, 2024 05:50

xxchan force-pushed the xxchan/refactor-ddl branch from 0012e27 to 613044f Compare November 4, 2024 05:59

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch 2 times, most recently from 4acdc3b to 42bb88c Compare November 4, 2024 06:19

Base automatically changed from xxchan/refactor-ddl to main November 4, 2024 10:58

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from 42bb88c to 7e3529d Compare November 20, 2024 09:59

xxchan marked this pull request as ready for review December 3, 2024 04:43

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch 2 times, most recently from dcf9e2b to 5b06546 Compare December 3, 2024 05:16

xxchan changed the title ~~feat: support alter shared source~~ feat(meta): generalize replace table to replace job Dec 3, 2024

xxchan changed the title ~~feat(meta): generalize replace table to replace job~~ refactor(meta): generalize replace table to replace job Dec 3, 2024

github-actions bot added the type/refactor label Dec 3, 2024

xxchan commented Dec 3, 2024

View reviewed changes

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch 4 times, most recently from 1d9faeb to 2a4081e Compare December 3, 2024 05:45

xxchan requested review from yezizp2012, BugenZhao and shanicky December 3, 2024 05:45

xxchan mentioned this pull request Dec 3, 2024

feat(meta): support replace source #19650

Merged

9 tasks

Base automatically changed from xxchan/administrative-baboon to main December 3, 2024 06:14

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from 2a4081e to 016e23b Compare December 3, 2024 06:15

refactor(meta): prepare for replace stream job

a373c13

xxchan force-pushed the 09-17-feat_support_alter_shared_source branch from 016e23b to a373c13 Compare December 3, 2024 08:28

yezizp2012 approved these changes Dec 3, 2024

View reviewed changes

xxchan enabled auto-merge December 3, 2024 08:38

xxchan added this pull request to the merge queue Dec 3, 2024

Merged via the queue into main with commit c2ee827 Dec 3, 2024
34 of 35 checks passed

xxchan deleted the 09-17-feat_support_alter_shared_source branch December 3, 2024 09:45

cyliu0 mentioned this pull request Dec 4, 2024

nightly-20241203 perf degradation #19666

Open

xxchan mentioned this pull request Dec 10, 2024

feat(frontend): support alter source refresh schema for shared source #19740

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(meta): generalize replace table to replace job #18935

refactor(meta): generalize replace table to replace job #18935

xxchan commented Oct 16, 2024 •

edited

Loading

xxchan commented Oct 16, 2024 •

edited

Loading

xxchan Dec 3, 2024

xxchan Dec 3, 2024

xxchan Dec 3, 2024

xxchan commented Dec 3, 2024 •

edited

Loading

yezizp2012 left a comment

refactor(meta): generalize replace table to replace job #18935

refactor(meta): generalize replace table to replace job #18935

Conversation

xxchan commented Oct 16, 2024 • edited Loading

What's changed and what's your intention?

Checklist

Documentation

Release note

xxchan commented Oct 16, 2024 • edited Loading

xxchan Dec 3, 2024

Choose a reason for hiding this comment

xxchan Dec 3, 2024

Choose a reason for hiding this comment

xxchan Dec 3, 2024

Choose a reason for hiding this comment

xxchan commented Dec 3, 2024 • edited Loading

Merge activity

yezizp2012 left a comment

Choose a reason for hiding this comment

xxchan commented Oct 16, 2024 •

edited

Loading

xxchan commented Oct 16, 2024 •

edited

Loading

xxchan commented Dec 3, 2024 •

edited

Loading