pkg/ccl/spanconfigccl/spanconfigreconcilerccl/spanconfigreconcilerccl_test: TestDataDriven failed #98038
Assignees
Labels
branch-master
Failures and bugs on the master branch.
C-test-failure
Broken test (automatically or manually discovered).
O-robot
Originated from a bot.
T-kv
KV Team
Milestone
Comments
cockroach-teamcity
added
branch-master
|
Doesn't immediately reproduce under stress: But I think I see what's going on -- it's just raciness in the test itself rather an a real underlying bug. At some point in the test we have protection records over /Table/10{6-8} at ts=3 and ts=4: Then we release those protection records: cockroach/pkg/ccl/spanconfigccl/spanconfigreconcilerccl/testdata/protectedts Lines 133 to 138 in 7416aab And expect that for /Table/10{6-8}, we should observe mutations that replace the existing record with one that has no protected timestamps. That does happen, but its possible to observe the intermediate mutation getting rid of just one protected timestamp (the test is not getting rid of pts records atomically). We should just assert on the final state instead of intermediate mutations. |
craig bot
pushed a commit
that referenced
this issue
Mar 6, 2023
98082: spanconfig: deflake spanconfigreconcilerccl/TestDataDriven r=irfansharif a=irfansharif Fixes #98038. This test set up two protection records over two schema objects at two timestamps ts=3 and ts=4. ``` /Table/10{6-7} protection_policies=[{ts: 3} {ts: 4}] /Table/10{7-8} protection_policies=[{ts: 3} {ts: 4}] ``` When it later released those protection records: ``` release record-id=3 release record-id=4 ---- ``` It asserted that the span config mutations showed that we did infact get rid of the protected state: ``` mutations ---- delete /Table/10{6-7} upsert /Table/10{6-7} range default delete /Table/10{7-8} upsert /Table/10{7-8} range default ``` But since release of these protections was non-atomic, in #98038 we observed the following transition instead. ``` delete /Table/10{6-7} upsert /Table/10{6-7} protection_policies=[{ts: 4}] delete /Table/10{7-8} upsert /Table/10{7-8} protection_policies=[{ts: 4}] delete /Table/10{6-7} upsert /Table/10{6-7} range default delete /Table/10{7-8} upsert /Table/10{7-8} range default ``` That is, we first got rid of the record with ts=3 and only then got rid of ts=4. We just rewrite the test to assert on the final state of the records that show no remaining protections, instead of trying to add synchronization for mutations. Release note: None Co-authored-by: irfan sharif <[email protected]>
blathers-crl bot
pushed a commit
that referenced
this issue
Jun 16, 2023
Fixes #98038. This test set up two protection records over two schema objects at two timestamps ts=3 and ts=4. /Table/10{6-7} protection_policies=[{ts: 3} {ts: 4}] /Table/10{7-8} protection_policies=[{ts: 3} {ts: 4}] When it later released those protection records: release record-id=3 release record-id=4 ---- It asserted that the span config mutations showed that we did infact get rid of the protected state: mutations ---- delete /Table/10{6-7} upsert /Table/10{6-7} range default delete /Table/10{7-8} upsert /Table/10{7-8} range default But since release of these protections was non-atomic, in #98038 we observed the following transition instead. delete /Table/10{6-7} upsert /Table/10{6-7} protection_policies=[{ts: 4}] delete /Table/10{7-8} upsert /Table/10{7-8} protection_policies=[{ts: 4}] delete /Table/10{6-7} upsert /Table/10{6-7} range default delete /Table/10{7-8} upsert /Table/10{7-8} range default That is, we first got rid of the record with ts=3 and only then got rid of ts=4. We just rewrite the test to assert on the final state of the records that show no remaining protections, instead of trying to add synchronization for mutations. Release note: None
irfansharif
added a commit
that referenced
this issue
Jun 27, 2023
Fixes #98038. This test set up two protection records over two schema objects at two timestamps ts=3 and ts=4. /Table/10{6-7} protection_policies=[{ts: 3} {ts: 4}] /Table/10{7-8} protection_policies=[{ts: 3} {ts: 4}] When it later released those protection records: release record-id=3 release record-id=4 ---- It asserted that the span config mutations showed that we did infact get rid of the protected state: mutations ---- delete /Table/10{6-7} upsert /Table/10{6-7} range default delete /Table/10{7-8} upsert /Table/10{7-8} range default But since release of these protections was non-atomic, in #98038 we observed the following transition instead. delete /Table/10{6-7} upsert /Table/10{6-7} protection_policies=[{ts: 4}] delete /Table/10{7-8} upsert /Table/10{7-8} protection_policies=[{ts: 4}] delete /Table/10{6-7} upsert /Table/10{6-7} range default delete /Table/10{7-8} upsert /Table/10{7-8} range default That is, we first got rid of the record with ts=3 and only then got rid of ts=4. We just rewrite the test to assert on the final state of the records that show no remaining protections, instead of trying to add synchronization for mutations. Release note: None
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
pkg/ccl/spanconfigccl/spanconfigreconcilerccl/spanconfigreconcilerccl_test.TestDataDriven failed with artifacts on master @ cf14ad694ee562676f53e36fa8495206c3aed61f:
Parameters:
TAGS=bazel,gss,deadlockHelp
See also: How To Investigate a Go Test Failure (internal)
This test on roachdash | Improve this report!
Jira issue: CRDB-25036
The text was updated successfully, but these errors were encountered: