Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GenericBinaryArray::from_list and GenericStringArray::from_list Ignores Offset #1726

Closed
tustvold opened this issue May 23, 2022 · 2 comments · Fixed by #2277
Closed

GenericBinaryArray::from_list and GenericStringArray::from_list Ignores Offset #1726

tustvold opened this issue May 23, 2022 · 2 comments · Fixed by #2277
Assignees
Labels

Comments

@tustvold
Copy link
Contributor

Describe the bug

Whilst perusing the code I noticed that GenericBinaryArray::from_list and GenericBinaryArray::from_list have the following logic

let mut builder = ArrayData::builder(Self::get_data_type())
    .len(v.len())
    .add_buffer(v.data_ref().buffers()[0].clone())
    .add_buffer(v.data_ref().child_data()[0].buffers()[0].clone());
if let Some(bitmap) = v.data_ref().null_bitmap() {
    builder = builder.null_bit_buffer(bitmap.bits.clone())
}

let data = unsafe { builder.build_unchecked() };

This does not propagate any offset that might be present in the source data

To Reproduce

I suspect if you slice an ArrayList and then pass it to this method, you will lose the offset position.

Expected behavior

The offset, and probably the null count, should be preserved by both methods

@tustvold
Copy link
Contributor Author

The issue also seems to appear in From<FixedSizeListArray> for FixedSizeBinaryArray and similar methods

@HaoYang670
Copy link
Contributor

I will work on this if no one else what to do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants