release-20.2: importccl: avoid random() collisions between KV batches #61630
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
miretskiy
approved these changes
Mar 8, 2021
pbardea
force-pushed
the
backport20.2-61214
branch
from
b4554d6 to
47392df
Compare
|
Huh, this is failing due to an error when trying to export to userfile locations. Put up #61789 and will rebase once that's in. |
pbardea
force-pushed
the
backport20.2-61214
branch
from
47392df to
f0d93e6
Compare
An import may parallelize the work to convert SQL rows into KVs. During this phase, default expressions are evaluated. Previously, IMPORT's implementations of random number generates that are evaluated in default expressions assumed that all of the given row IDs were contiguous. This is not the case since 1 row converter may be responsible for converting several non-contiguous batches of rows. This resulted in random_values colliding between different batches of KV space. This commit fixes this bug by feeding in the current position and resetting the random source backing these methods. This ensures that import treats a new contiguous batch of rows separately. Release justification: bug fix Release note (bug fix): Fix a bug where random numbers generated as default expressions during IMPORT would collide a few hundred rows apart from each-other.
pbardea
force-pushed
the
backport20.2-61214
branch
from
f0d93e6 to
8a311c8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #61214.
/cc @cockroachdb/release
An import may parallelize the work to convert SQL rows into KVs. During
this phase, default expressions are evaluated. Previously, IMPORT's
implementations of random number generates that are evaluated in default
expressions assumed that all of the given row IDs were contiguous. This
is not the case since 1 row converter may be responsible for converting
several non-contiguous batches of rows. This resulted in random_values
colliding between different batches of KV space.
This commit fixes this bug by feeding in the current position and
resetting the random source backing these methods. This ensures that
import treats a new contiguous batch of rows separately.
Fixes #61203.
Release justification: bug fix
Release note (bug fix): Fix a bug where random numbers generated as
default expressions during IMPORT would collide a few hundred rows apart
from eachother.