Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add corr aggregate function #1561

Merged
merged 5 commits into from
Jan 16, 2022
Merged

Add corr aggregate function #1561

merged 5 commits into from
Jan 16, 2022

Conversation

realno
Copy link
Contributor

@realno realno commented Jan 14, 2022

Which issue does this PR close?

This PR add the corr function which is part of #1486

Rationale for this change

Add correlation function to DataFusion.

What changes are included in this PR?

New function corr(col1, co2) is added.

Are there any user-facing changes?

New function corr(col1, co2) is added.

No breaking change.

@github-actions github-actions bot added ballista datafusion Changes in the datafusion crate labels Jan 14, 2022
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jan 14, 2022
@liukun4515
Copy link
Contributor

liukun4515 commented Jan 14, 2022

why not support covar_pop and covar_samp together?
reference https://www.postgresql.org/docs/14/functions-aggregate.html
@realno

@realno
Copy link
Contributor Author

realno commented Jan 14, 2022

why not support covar_pop and covar_samp together? reference https://www.postgresql.org/docs/14/functions-aggregate.html @realno

covar_pop and covar_samp were merged here #1551

Copy link
Contributor

@liukun4515 liukun4515 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
you can rebase the master, after the Clippy fixed

if let ScalarValue::Float64(Some(c)) = covar {
if let ScalarValue::Float64(Some(s1)) = stddev1 {
if let ScalarValue::Float64(Some(s2)) = stddev2 {
return Ok(ScalarValue::Float64(Some(c / s1 / s2)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can s1 or s2 be 0?

Copy link
Contributor Author

@realno realno Jan 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They may be 0, pushed a fix. Thanks for finding this.

@alamb alamb merged commit b743610 into apache:master Jan 16, 2022
@alamb
Copy link
Contributor

alamb commented Jan 16, 2022

Thanks @realno !

@realno realno deleted the add-corr-operator branch February 9, 2022 20:09
@alamb alamb changed the title add correlation function Add corr aggregate function Feb 10, 2022
@alamb alamb added enhancement New feature or request and removed documentation Improvements or additions to documentation labels Feb 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants