Skip to content

Commit

Permalink
Optimize String#to_utf16 (#14671)
Browse files Browse the repository at this point in the history
A series of optimizations to improve performance of `String#to_utf16`:

* Remove branches for impossible codepoints. `each_char` already excludes invalid codepoints, so we only have to handle the encoding as one or two UInt16.
* Drop `ascii_only?` branch. The performance benefit is questionable because `ascii_only?` iterates the string. With optimizations to the regular loop, this special case doesn't provide much extra performance, so it's expendable.
* Use pointer appender to avoid bounds checks on every slice assignment. It also improves convenience.
* Use wrapping math operators. These operations cannot overflow, so we can use the unsafe variants for better performance.
  • Loading branch information
straight-shoota authored Jun 12, 2024
1 parent 70fb98e commit ef04b2e
Showing 1 changed file with 9 additions and 28 deletions.
37 changes: 9 additions & 28 deletions src/string/utf16.cr
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,6 @@ class String
# "hi 𐂥".to_utf16 # => Slice[104_u16, 105_u16, 32_u16, 55296_u16, 56485_u16]
# ```
def to_utf16 : Slice(UInt16)
if ascii_only?
# size == bytesize, so each char fits in one UInt16

# This is essentially equivalent to `to_slice.map(&.to_u16)` but also makes
# sure to allocate a null byte after the string.
slice = Slice(UInt16).new(bytesize + 1) do |i|
if i == bytesize
0_u16
else
to_unsafe[i].to_u16
end
end
return slice[0, bytesize]
end

# size < bytesize, so we need to count the number of characters that are
# two UInt16 wide.
u16_size = 0
Expand All @@ -37,28 +22,24 @@ class String
# Allocate one extra character for trailing null
slice = Slice(UInt16).new(u16_size + 1)

i = 0
appender = slice.to_unsafe.appender
each_char do |char|
ord = char.ord
if ord <= 0xd800 || (0xe000 <= ord < 0x1_0000)
if ord < 0x1_0000
# One UInt16 is enough
slice[i] = ord.to_u16
elsif ord >= 0x1_0000
# Needs surrogate pair
ord -= 0x1_0000
slice[i] = 0xd800_u16 + ((ord >> 10) & 0x3ff) # Keep top 10 bits
i += 1
slice[i] = 0xdc00_u16 + (ord & 0x3ff) # Keep low 10 bits
appender << ord.to_u16!
else
# Invalid char: use replacement
slice[i] = 0xfffd_u16
# Needs surrogate pair
ord &-= 0x1_0000
appender << 0xd800_u16 &+ ((ord >> 10) & 0x3ff) # Keep top 10 bits
appender << 0xdc00_u16 &+ (ord & 0x3ff) # Keep low 10 bits
end
i += 1
end

# Append null byte
slice[i] = 0_u16
appender << 0_u16

# The trailing null is not part of the returned slice
slice[0, u16_size]
end

Expand Down

0 comments on commit ef04b2e

Please sign in to comment.