sql: set hash-sharded index column type to INT8 #76930
Conversation
|
This change is |
There was a problem hiding this comment.
This is awesome~
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)
mgartner
force-pushed
the
use-int8-for-shard-column
branch
from
4722ea9 to
0c154e8
Compare
There was a problem hiding this comment.
Does this mean we could / should use fnv64 instead of fnv32?
Reviewable status:
There was a problem hiding this comment.
Does this mean we could / should use fnv64 instead of fnv32?
I think this would only be useful when the bucket count is greater than 2^32 - 1, which seems unlikely. In fact, I just tried to create a hash-sharded index with a bucket count of 999999999 (which is less than 2^32 - 1), and the CREATE TABLE statement locked up. I created an issue to track this: #77000.
Reviewable status:
mgartner
force-pushed
the
use-int8-for-shard-column
branch
4 times, most recently
from
b1d64e9 to
6772437
Compare
mgartner
force-pushed
the
use-int8-for-shard-column
branch
from
6772437 to
ccd582c
Compare
Shard columns of hash-sharded indexes are computed columns with an expression in the form `mod(fnv32(crdb_internal.datums_to_bytes(...)))`. The `mod()` builtin returns a value of type `INT8`, while the shard column's type was previously `INT4`. Because these types did not match, `mod` expressions were wrapped in the assignment cast function, `crdb_internal.assignment_cast`, in mutation query plans. This commit changes the type of newly created shard columns to `INT8`, eliminating the need for an assignment cast. Because all integers are encoded as varints in keys and values, this will not increase the amount of space on disk required for these columns. Release justification: This is a minor change to hash-sharded indexes, which are a newly un-experimentalized feature in the upcoming release. Release note (sql change): The type of shard columns created for hash-sharded indexes have changed from `INT4` to `INT8`. This should have no effect on behavior or performance.
mgartner
force-pushed
the
use-int8-for-shard-column
branch
from
ccd582c to
fde3881
Compare
|
@ajwerner can you take a look at this and sign-off if you're ok with it. |
|
TFTRs! bors r+ |
|
🕐 Waiting for PR status (Github check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set. |
|
bors r+ |
|
Stopped waiting for PR status (Github check) without running due to duplicate requests to run. You may check Bors to see that this PR is included in a batch by one of the other requests. |
|
Build succeeded: |
Shard columns of hash-sharded indexes are computed columns with an
expression in the form
mod(fnv32(crdb_internal.datums_to_bytes(...))).The
mod()builtin returns a value of typeINT8, while the shardcolumn's type was previously
INT4. Because these types did not match,modexpressions were wrapped in the assignment cast function,crdb_internal.assignment_cast, in mutation query plans.This commit changes the type of newly created shard columns to
INT8,eliminating the need for an assignment cast. Because all integers are
encoded as varints in keys and values, this will not increase the amount
of space on disk required for these columns.
Release justification: This is a minor change to hash-sharded indexes,
which are a newly un-experimentalized feature in the upcoming release.
Release note (sql change): The type of shard columns created for
hash-sharded indexes have changed from
INT4toINT8. This shouldhave no effect on behavior or performance.