Fix Presto Java UUID serialization #11197
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
|
please review @aditi-pandit @Yuhta @mbasmanova |
Yuhta
reviewed
Oct 8, 2024
| @@ -63,10 +63,13 @@ TEST_F(UuidFunctionsTest, castAsVarchar) { | |||
| // Verify that CAST results as the same as boost::lexical_cast. We do not use | |||
| // boost::lexical_cast to implement CAST because it is too slow. | |||
| auto expected = makeFlatVector<std::string>(size, [&](auto row) { | |||
| const auto uuid = uuids->valueAt(row); | |||
| auto uuid = uuids->valueAt(row); | |||
There was a problem hiding this comment.
Just do this and rest of the file can be left unchanged:
auto uuid = folly::Endian::big(uuids->valueAt(row));| @@ -97,8 +97,8 @@ class UuidCastOperator : public exec::CastOperator { | |||
|
|
|||
| size_t offset = 0; | |||
| for (auto i = 0; i < 16; ++i) { | |||
| result.data()[offset] = kHexTable[uuidBytes[i] * 2]; | |||
| result.data()[offset + 1] = kHexTable[uuidBytes[i] * 2 + 1]; | |||
| result.data()[offset] = kHexTable[uuidBytes[15 - i] * 2]; | |||
There was a problem hiding this comment.
const auto uuid = folly::Endian::big(uuids->valueAt(row));| @@ -125,7 +125,10 @@ class UuidCastOperator : public exec::CastOperator { | |||
| auto uuid = boost::lexical_cast<boost::uuids::uuid>(uuidString); | |||
|
|
|||
| int128_t u; | |||
| memcpy(&u, &uuid, 16); | |||
| auto charPtr = reinterpret_cast<char*>(&u); | |||
There was a problem hiding this comment.
memcpy(&u, &uuid, 16);
u = folly::Endian::big(u);
|
Thanks @Yuhta , but |
|
@BryanCutler You can add a utility to |
BryanCutler
force-pushed
the
fix-uuid-java-serde-23311
branch
from
af1deb8
to
beb2740
Compare
Added DecimalUtil::big for reversing int128_t byte order
BryanCutler
force-pushed
the
fix-uuid-java-serde-23311
branch
from
beb2740
to
d608cfe
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes the
PrestoSerializerto put UUID values in the correct format that is expected by Presto Java so that the values will match those from a Java worker. First, when converting UUID to/from string, the values are no longer in big endian format (as taken from boost::uuid) and are instead stored as a little endian in an int128_t. Secondly, Presto Java will read UUID values from anInt128ArrayBlockwith the first value as the most significant bits. To correct this, the upper/lower parts of the int128_t are swapped during serialization/deserialization.A unit test for checking roundtrip UUID serializaiton was added and manual testing of Presto with a native worker to verify the problem from the issue description is fixed.
From prestodb/presto#23311