Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Fixed wrong null_count when slicing a sliced Bitmap #848

Merged

Conversation

satlank
Copy link
Contributor

@satlank satlank commented Feb 17, 2022

When pre-calculating the null_count on a Bitmap we need to start from
the correct place in the underlying byte-array (i.e. take into account
that we may already be looking at a slice). Currently, when we slice of
a small part (so that we enter the first branch of the null_count
choice), the null_count assumes that the current offset is 0, but it
should not.

This adds a test for this situation and fixes the issue.

@codecov
Copy link

codecov bot commented Feb 17, 2022

Codecov Report

Merging #848 (5559917) into main (6ccb154) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #848   +/-   ##
=======================================
  Coverage   71.49%   71.49%           
=======================================
  Files         335      335           
  Lines       17910    17910           
=======================================
  Hits        12805    12805           
  Misses       5105     5105           
Impacted Files Coverage Δ
src/bitmap/immutable.rs 87.80% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ccb154...5559917. Read the comment docs.

When pre-calculating the null_count on a Bitmap we need to start from
the correct place in the underlying byte-array (i.e. take into account
that we may already be looking at a slice).  Currently, when we slice of
a small part (so that we enter the first branch of the null_count
choice), the null_count assumes that the current offset is 0, but it
should not.

This adds a test for this situation and fixes the issue.
@ritchie46
Copy link
Collaborator

Looks good @satlank, thanks a lot!

@jorgecarleitao
Copy link
Owner

this is a serious semantic bug. Thank you so much @satlank for finding and fixing! 💯

@jorgecarleitao jorgecarleitao merged commit 107f118 into jorgecarleitao:main Feb 17, 2022
@jorgecarleitao jorgecarleitao added the bug Something isn't working label Feb 17, 2022
@jorgecarleitao jorgecarleitao changed the title Fix wrong null_count when slicing a sliced Bitmap Fixed wrong null_count when slicing a sliced Bitmap Feb 17, 2022
@satlank satlank deleted the fixWrongNullCountOnSubSlice branch February 17, 2022 17:25
@satlank
Copy link
Contributor Author

satlank commented Feb 17, 2022

My pleasure @jorgecarleitao, did cause a bit of head-scratching but luckily easy fix ;-) Thanks for this crate, it is proving quite useful!

sydduckworth pushed a commit to mindx/arrow2 that referenced this pull request Mar 2, 2022
When pre-calculating the null_count on a Bitmap we need to start from
the correct place in the underlying byte-array (i.e. take into account
that we may already be looking at a slice).  Currently, when we slice of
a small part (so that we enter the first branch of the null_count
choice), the null_count assumes that the current offset is 0, but it
should not.

This adds a test for this situation and fixes the issue.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants