kvnemesis: add transaction savepoints #97444
Assignees
Labels
A-testing
Testing tools and infrastructure
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
P-3
Issues/test failures with no fix SLA
T-kv
KV Team
Comments
nvanbenschoten
added
C-enhancement
|
This (probably) would have found the bug discovered in #94337. |
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Sep 12, 2023
This patch adds basic support for savepoints in kvnemesis. It does not handle validating retries using initial savepoints. Fixes: cockroachdb#97444 Release note: None
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Sep 13, 2023
This patch adds basic support for savepoints in kvnemesis. It does not handle validating retries using initial savepoints. Fixes: cockroachdb#97444 Release note: None
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Oct 30, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
``` engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Oct 31, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Nov 2, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Nov 2, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
craig bot
pushed a commit
that referenced
this issue
Nov 2, 2023
113306: storage: correctly update the stats for rolled back deletes r=nvanbenschoten a=miraradeva
Previously, a delete on a key rolled back by a savepoint did not correctly update the MVCCStats when the intent for that key was resolved. In particular, if the rollback revealed a non-deleted value for the key, the stats considered that key/value non-live and included other inaccuracies. This is an example from the output of kvnemesis, which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent resolution and adds testing for rolled back deletes.
Informs: #97444
Release note (bug fix): Rolled back deletes no longer cause a discrepancy between computed stats and the actual stored values.
Co-authored-by: Mira Radeva <[email protected]>
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Nov 6, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
annrpom
pushed a commit
to annrpom/cockroach
that referenced
this issue
Nov 29, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
miretskiy
pushed a commit
to miretskiy/cockroach
that referenced
this issue
Nov 29, 2023
Previously, a delete on a key rolled back by a savepoint did not
correctly update the MVCCStats when the intent for that key was
resolved. In particular, if the rollback revealed a non-deleted value
for the key, the stats considered that key/value non-live and included
other inaccuracies. This is an example from the output of kvnemesis,
which caught the bug:
```
engine stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:771 LiveCount:13 KeyBytes:1350 KeyCount:34
ValBytes:714 ValCount:53 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:254 SysCount:6 AbortSpanBytes:0}
computed stats:
{ContainsEstimates:0 LastUpdateNanos:1698322873881949000 LockAge:0
GCBytesAge:0 LiveBytes:161 LiveCount:1 KeyBytes:0 KeyCount:0
ValBytes:8 ValCount:0 IntentBytes:8 IntentCount:0 LockBytes:0
LockCount:0 RangeKeyCount:0 RangeKeyBytes:0 RangeValCount:0
RangeValBytes:0 SysBytes:0 SysCount:0 AbortSpanBytes:0}
```
This patch fixes the logic responsible for computing stats upon intent
resolution and adds testing for rolled back deletes.
Informs: cockroachdb#97444
Release note (bug fix): Rolled back deletes no longer cause a
discrepancy between computed stats and the actual stored values.
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Nov 29, 2023
This patch adds support for generating and validating savepoints in kvnemesis. Fixes: cockroachdb#97444 Release note: None
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Dec 11, 2023
This patch adds support for generating and validating savepoints in kvnemesis. Fixes: cockroachdb#97444 Release note: None
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Jan 2, 2024
This patch adds support for generating and validating savepoints in kvnemesis. Fixes: cockroachdb#97444 Release note: None
craig bot
pushed a commit
that referenced
this issue
Jan 10, 2024
110494: kvnemesis: add support for savepoints r=arulajmani a=miraradeva This patch adds support for generating and validating savepoints in kvnemesis. Fixes: #97444 Release note: None Co-authored-by: Mira Radeva <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We should extend kvnemesis to use transaction savepoints.
Jira issue: CRDB-24698
Epic CRDB-34183
The text was updated successfully, but these errors were encountered: