Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize take_bits to optimize take_boolean / take_primitive / take_byte_view: up to -25% #6622

Merged
merged 17 commits into from
Oct 24, 2024

Conversation

Dandandan
Copy link
Contributor

@Dandandan Dandandan commented Oct 24, 2024

Which issue does this PR close?

Closes #6621

Rationale for this change

Further optimization

take bool 1024          time:   [864.69 ns 865.71 ns 866.55 ns]
                        change: [-25.978% -25.819% -25.660%] (p = 0.00 < 0.05)
                        Performance has improved.

take bool null indices 1024
                        time:   [1.5549 µs 1.5560 µs 1.5572 µs]
                        change: [-16.412% -15.787% -15.182%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
  2 (2.00%) low mild
  6 (6.00%) high mild
  7 (7.00%) high severe

take bool null values 1024
                        time:   [1.6879 µs 1.6897 µs 1.6916 µs]
                        change: [-11.554% -11.427% -11.294%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

take bool null values null indices 1024
                        time:   [3.5142 µs 3.5281 µs 3.5463 µs]
                        change: [-16.009% -15.478% -14.934%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) low mild
  8 (8.00%) high mild
  2 (2.00%) high severe

take i32 null values 1024
                        time:   [1.2923 µs 1.2929 µs 1.2935 µs]
                        change: [-19.133% -19.022% -18.913%] (p = 0.00 < 0.05)
                        Performance has improved.


take stringview null values 1024
                        time:   [1.5800 µs 1.5815 µs 1.5832 µs]
                        change: [-16.514% -16.324% -16.129%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 25 outliers among 100 measurements (25.00%)
  5 (5.00%) low severe
  10 (10.00%) low mild
  10 (10.00%) high severe

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 24, 2024
@Dandandan Dandandan changed the title Optimize take_bits and take_bytes Optimize take_bits / take_boolean Oct 24, 2024
@Dandandan Dandandan changed the title Optimize take_bits / take_boolean Optimize take_bits / take_boolean: up to -25% Oct 24, 2024
arrow-select/src/take.rs Outdated Show resolved Hide resolved
arrow-select/src/take.rs Show resolved Hide resolved
@Dandandan Dandandan merged commit a9294d7 into apache:master Oct 24, 2024
26 checks passed
@Dandandan
Copy link
Contributor Author

Thanks for the review @tustvold

@Dandandan
Copy link
Contributor Author

I found this also improves take for primitive / stringview with null values (15 - 20 %, added results in PR comment)

@Dandandan Dandandan changed the title Optimize take_bits / take_boolean: up to -25% Optimize take_bits to optimize take_boolean / take_primitive / take_byte_view: up to -25% Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Some take optimizations
2 participants