Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StringArray::from_list and from(binary_array) -> string_array are unsound #2205

Closed
HaoYang670 opened this issue Jul 27, 2022 · 1 comment · Fixed by #2686
Closed

StringArray::from_list and from(binary_array) -> string_array are unsound #2205

HaoYang670 opened this issue Jul 27, 2022 · 1 comment · Fixed by #2686
Labels
bug

Comments

@HaoYang670
Copy link
Contributor

HaoYang670 commented Jul 27, 2022 •
edited
Loading

This would allow non-utf8 data within a StringArray using safe APIs, which would break our safety guarantees

Originally posted by @tustvold in #2181 (comment)

Describe the solution you'd like

  1. rename the current method to from_list_unchecked and mark it as unsafe
  2. implement the safe version in impl From<ListArray> for StringArray
  3. update the docs and tests.

We should also do the same change for from_binary after merging #2181
@HaoYang670 HaoYang670 mentioned this issue Jul 27, 2022
Merged
@HaoYang670
Copy link
Contributor Author

We should do this after solving #1726.

@tustvold tustvold changed the title StringArray::from_list is unsafe StringArray::from_list is unsound Jul 28, 2022
@tustvold tustvold added the bug label Jul 28, 2022
@HaoYang670 HaoYang670 changed the title StringArray::from_list is unsound StringArray::from_list and from(binary_array) -> string_array are unsound Jul 28, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Sep 8, 2022
@tustvold
Verify valid UTF-8 when converting byte array (apache#2205)
e797d64
tustvold added a commit to tustvold/arrow-rs that referenced this issue Sep 8, 2022
@tustvold
Verify valid UTF-8 when converting byte array (apache#2205)
a095675
@tustvold tustvold mentioned this issue Sep 8, 2022
Merged
tustvold added a commit that referenced this issue Sep 11, 2022
@tustvold
Verify valid UTF-8 when converting byte array (#2205) (#2686)
8206f01 
* Verify valid UTF-8 when converting byte array (#2205)

* Add doc comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Verify valid UTF-8 when converting byte array (#2205) tustvold/arrow-rs
2 participants
@tustvold @HaoYang670