Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Substrait logical plan #704

Merged

Conversation

MichaelScofield
Copy link
Collaborator

@MichaelScofield MichaelScofield commented Dec 5, 2022

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Use Substrait logical plan to execute query between Frontend and Datanode, in distributed mode. Formerly we are converting table scan to SQL, which is not elegant.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

#682

#597

#532

@codecov
Copy link

codecov bot commented Dec 5, 2022

Codecov Report

Merging #704 (911fafc) into develop (55e6be7) will decrease coverage by 0.00%.
The diff coverage is 77.88%.

@@             Coverage Diff             @@
##           develop     #704      +/-   ##
===========================================
- Coverage    86.06%   86.05%   -0.01%     
===========================================
  Files          451      451              
  Lines        58564    58661      +97     
===========================================
+ Hits         50401    50479      +78     
- Misses        8163     8182      +19     
Flag Coverage Δ
rust 86.05% <77.88%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/catalog/src/helper.rs 86.00% <ø> (ø)
src/catalog/src/lib.rs 84.90% <ø> (ø)
src/catalog/src/remote/manager.rs 69.35% <ø> (ø)
src/common/catalog/src/lib.rs 100.00% <ø> (ø)
src/common/substrait/src/error.rs 0.00% <0.00%> (ø)
src/common/substrait/src/lib.rs 100.00% <ø> (ø)
src/frontend/src/catalog.rs 6.79% <0.00%> (ø)
src/frontend/src/error.rs 8.47% <0.00%> (-0.15%) ⬇️
src/frontend/src/instance/distributed.rs 75.66% <ø> (ø)
src/meta-srv/src/keys.rs 100.00% <ø> (ø)
... and 23 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

src/api/greptime/v1/database.proto Outdated Show resolved Hide resolved
src/datanode/src/instance/grpc.rs Outdated Show resolved Hide resolved
src/common/substrait/src/df_logical.rs Outdated Show resolved Hide resolved
src/common/substrait/src/df_logical.rs Outdated Show resolved Hide resolved
@MichaelScofield
Copy link
Collaborator Author

@waynexia resolved, PTAL

Copy link
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General looks good to me, and I left a few comments about style.

src/table/src/metadata.rs Outdated Show resolved Hide resolved
src/common/substrait/src/df_logical.rs Show resolved Hide resolved
src/common/substrait/src/df_logical.rs Outdated Show resolved Hide resolved
@MichaelScofield
Copy link
Collaborator Author

@waynexia resolved, PTAL

Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@killme2008 killme2008 merged commit 8959dbc into GreptimeTeam:develop Dec 6, 2022
@MichaelScofield MichaelScofield deleted the feat/dist-scan-substrait branch December 6, 2022 11:24
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* feat: use Substrait logical plan to query data from Datanode in Frontend in distributed mode

* fix: resolve PR comments

* fix: resolve PR comments

* fix: resolve PR comments

Co-authored-by: luofucong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants