Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of REPEAT functions #12015

Merged
merged 4 commits into from
Aug 16, 2024
Merged

Conversation

tlm365
Copy link
Contributor

@tlm365 tlm365 commented Aug 15, 2024

Which issue does this PR close?

Closes #11990.

Rationale for this change

Refactor code and improve performance.

What changes are included in this PR?

repeat function and benchmark.

Are these changes tested?

Yes

Are there any user-facing changes?

No


Benchmark stats on my local machine:

  • AMD Ryzen 5 PRO 5650U (6 cores 12 threads) - 24Gb RAM

2024-08-16_11-31-57

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tlm365 this PR looks promising. Would you mind adding bench results to the PR description?

@tlm365
Copy link
Contributor Author

tlm365 commented Aug 15, 2024

Thanks @tlm365 this PR looks promising. Would you mind adding bench results to the PR description?

@comphead Thanks for reviewing, I have updated it.

datafusion/functions/benches/repeat.rs Outdated Show resolved Hide resolved
datafusion/functions/src/string/repeat.rs Outdated Show resolved Hide resolved
(Some(_), Some(_)) => Some("".to_string()),
_ => None,
}
.for_each(|(string, number)| match (string, number) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 -- very nice

@@ -251,6 +252,22 @@ impl<'a> ColumnarValueRef<'a> {
}
}

pub trait StringArrayType<'a>: ArrayAccessor<Item = &'a str> + Sized {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense -- thank you @tlm365 and @Omega359 -- I'll make a PR to add some additional comments to this trait as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#12027 is a PR to document this trait and how to use it

@alamb alamb merged commit bb92181 into apache:main Aug 16, 2024
26 checks passed
@alamb
Copy link
Contributor

alamb commented Aug 16, 2024

Thank you for the reviews @tlm365 @Omega359 and @comphead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve performance of REPEAT functions
4 participants