Faster BinaryArray to StringArray conversion (~67%) #3168
Conversation
tustvold
force-pushed
the
faster-byte-array-string-conversion
branch
3 times, most recently
from
9c133c2
to
ae3ba7b
Compare
tustvold
force-pushed
the
faster-byte-array-string-conversion
branch
from
ae3ba7b
to
5598db0
Compare
| let builder = v.into_data().into_builder().data_type(Self::DATA_TYPE); | ||
| Self::from(builder.build().unwrap()) | ||
| // SAFETY: | ||
| // Validated UTF-8 above |
There was a problem hiding this comment.
👍
How much performance improvement can we get?
There was a problem hiding this comment.
byte_array_to_string_array 20000
time: [13.252 µs 13.254 µs 13.257 µs]
change: [-67.094% -67.073% -67.049%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) high mild
6 (6.00%) high severe
tustvold
changed the title
| let builder = v.into_data().into_builder().data_type(Self::DATA_TYPE); | ||
| Self::from(builder.build().unwrap()) |
There was a problem hiding this comment.
Does it basically remove full validation cost?
| for offset in offsets.iter() { | ||
| assert!(validated.is_char_boundary(offset.as_usize())) | ||
| } |
There was a problem hiding this comment.
Do we need to exclude last offset?
There was a problem hiding this comment.
No, in case the string is sliced
tustvold
changed the title
|
Benchmark runs are scheduled for baseline = 1d22fe3 and contender = 8ba7842. 8ba7842 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #.
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?