You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently Fury xlang serialization use utf8 for string encoding, which is not performance efficient in many languages.
We introduced utf16 in https://fury.apache.org/docs/specification/fury_xlang_serialization_spec#string . But c++ doesn't support utf16, and most users assume the std::string is utf-8 encoded if it's used as a string instead of buffer. we should support to transcode utf16 encoded string to utf8 string in fury C++ deserialization.
Describe the solution you'd like
Implement utf16 to utf8 convertion in fury c++. The implementation should use SIMD to provide faster speed.
chaokunyang
changed the title
[Go] Support convert utf16 encoded string to utf8 string #1545
[C++] Support convert utf16 encoded string to utf8 string
Apr 19, 2024
Is your feature request related to a problem? Please describe.
Currently Fury xlang serialization use utf8 for string encoding, which is not performance efficient in many languages.
We introduced utf16 in https://fury.apache.org/docs/specification/fury_xlang_serialization_spec#string . But c++ doesn't support utf16, and most users assume the
std::string
is utf-8 encoded if it's used as a string instead of buffer. we should support to transcode utf16 encoded string to utf8 string in fury C++ deserialization.Describe the solution you'd like
Implement utf16 to utf8 convertion in fury c++. The implementation should use SIMD to provide faster speed.
Additional context
#1413
The text was updated successfully, but these errors were encountered: