Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: try adding FragmentRelation in meta store #20035

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

shanicky
Copy link
Contributor

@shanicky shanicky commented Jan 6, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR attempts to add a FragmentRelation table to eventually replace the existing Actor and ActorDispatcher tables, using dynamically generated Actors and Dispatchers.

This PR currently only generates the FragmentRelation entity when inserting into the ActorDispatcher and then performs the insert; Our streaming logic doesn't use it.

Part of #19106 .

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

  • My PR needs documentation updates.
Release note

@shanicky shanicky force-pushed the peng/fragment-relation branch from 5cf9ecb to a68fd78 Compare January 6, 2025 10:10
@shanicky shanicky marked this pull request as draft January 6, 2025 10:23
@shanicky shanicky requested a review from yezizp2012 January 7, 2025 14:40
@shanicky shanicky marked this pull request as ready for review January 7, 2025 14:40
Copy link
Member

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


let mut fragment_relations = BTreeMap::new();

for (actor_id, dispatchers) in &new_actor_dispatchers {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some comments for it.

@@ -429,6 +429,35 @@ impl CatalogController {
let inner = self.inner.write().await;
let txn = inner.db.begin().await?;

let mut fragment_relations = BTreeMap::new();

for (fragment, actors, actor_dispatchers) in &fragment_actors {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants