-
Notifications
You must be signed in to change notification settings - Fork 241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem: versiondb write wrong data #1683
Comments
yihuang
added a commit
to yihuang/cronos
that referenced
this issue
Nov 11, 2024
Closes: crypto-org-chain#1683 Solution: - add fix command to fix corrupted data in versiondb
yihuang
added a commit
to yihuang/cronos
that referenced
this issue
Nov 11, 2024
Closes: crypto-org-chain#1683 Solution: - add fix command to fix corrupted data in versiondb
13 tasks
yihuang
added a commit
to yihuang/cronos
that referenced
this issue
Nov 11, 2024
…pto-org-chain#1685) Closes: crypto-org-chain#1683 Solution: - add fix command to fix corrupted data in versiondb rename support SkipVersionZero support SkipVersionZero cleanup
13 tasks
yihuang
added a commit
to yihuang/cronos
that referenced
this issue
Nov 12, 2024
…pto-org-chain#1685) Closes: crypto-org-chain#1683 Solution: - add fix command to fix corrupted data in versiondb rename support SkipVersionZero support SkipVersionZero cleanup cleanup cleanup fix test cleanup destroy log store name fix data manually cli Update versiondb/client/fixdata.go Signed-off-by: yihuang <[email protected]> Update versiondb/client/fixdata.go Signed-off-by: yihuang <[email protected]> rnemae Update CHANGELOG.md Signed-off-by: yihuang <[email protected]> fix test don't return nil as empty slice add dryrun mode separete read from iteration add stores flag debug add timestamp api
yihuang
added a commit
that referenced
this issue
Nov 12, 2024
…) (#1686) * Problem: nixpkgs-fmt is deprecated (backport: #1677) Solution: - switch to nixfmt-rfc-style * Problem: no command to fix corrupted data in versiondb (backport: #1685) Closes: #1683 Solution: - add fix command to fix corrupted data in versiondb rename support SkipVersionZero support SkipVersionZero cleanup cleanup cleanup fix test cleanup destroy log store name fix data manually cli Update versiondb/client/fixdata.go Signed-off-by: yihuang <[email protected]> Update versiondb/client/fixdata.go Signed-off-by: yihuang <[email protected]> rnemae Update CHANGELOG.md Signed-off-by: yihuang <[email protected]> fix test don't return nil as empty slice add dryrun mode separete read from iteration add stores flag debug add timestamp api * update rocksdb * changelog * fix flag name conflict * validate timestamp * Update CHANGELOG.md Signed-off-by: yihuang <[email protected]> * skip non-zero version * flush after fix * only flush if not dry-run --------- Signed-off-by: yihuang <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this issue
Nov 12, 2024
* Problem: no command to fix corrupted data in versiondb Closes: #1683 Solution: - add fix command to fix corrupted data in versiondb * rename * support SkipVersionZero * support SkipVersionZero * cleanup * cleanup * cleanup * destroy * fix data manually * Update CHANGELOG.md Signed-off-by: yihuang <[email protected]> * Update versiondb/tsrocksdb/store.go Co-authored-by: mmsqe <[email protected]> Signed-off-by: yihuang <[email protected]> * cli * cleanup * Update versiondb/client/fixdata.go Signed-off-by: yihuang <[email protected]> * rename * fix test * check nil * don't return nil as empty slice * add dryrun mode * cleanup * separete read from iteration * store name flag * validate timestamp * skip non-zero version * update gomod2nix * fix build * flush after fix * fix * revert --------- Signed-off-by: yihuang <[email protected]> Co-authored-by: mmsqe <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
suspected to be related to a rocksdb version upgrade happens in the past, which breaks user-defined timestamp feature. will develop a fix tool, should not happen again in the future.
Description
versiondb using rocksdb's user-defined timestamp feature, which assign a timestamp with a key, the timestamp is encoded with little endianness and append to the key as postfix, and the UDT feature will handle the timestamp specially, like a tuple
(key, timestamp)
.But for the wrong blocks, the data are apparently written without the UDT special handling, and read out as
(key | timestamp, 0)
.Fix
We adopt a non-intrusive strategy to fix:
cronosd changeset fixdata <dir>
command to re-insert the wrong data.0
data.In this way, we don't automatically trim the timestamp 0 data, so if the same issue happen again in the future, we can still apply the same command to recover it.
The text was updated successfully, but these errors were encountered: