Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize buffer build #77

Open
wants to merge 10 commits into
base: arrow-4.0.0-oap
Choose a base branch
from

Conversation

FelixYBW
Copy link

There are 3 optimizations included:

  1. Added prefetch to buffer_build, assumption is if we append one value, we may append the second one soon
  2. Since most of the value isn't NULL, we reset the null bitmap buffer to 0xffffffff once we allocate it. Later we skip the bitset if value is true. Now we still need to update the bitlength, and one branch. Two more optimizations can be done: 1) skip memory allocation until first false value met. 2) in binary_builder etc, skip the bitmap append until first false value
  3. specify int32_t for typedbufferbuild, in which we use *(pos)=value, instead of memcpy

@github-actions
Copy link

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW

Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

binwei added 5 commits January 12, 2022 16:18
2. reset null bitmap to all true after allocation, skip bit update if it's true
…nto arrow-4.0.0-oap

Conflicts:
	cpp/src/arrow/buffer_builder.h
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant