storage: update debug range-data to account for remote files #119960
Assignees
Labels
A-storage
Relating to our storage engine (Pebble) on-disk storage.
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Comments
itsbilal
added
C-enhancement
craig bot
pushed a commit
that referenced
this issue
Mar 20, 2024
120674: sql: allow procedures to call other procedures r=mgartner a=mgartner #### sql/logictest: add more tests for UDFs calling UDFs This commit rearranges some existing tests and adds a few new tests for UDFs that invoke other UDFs. Release note: None #### sql: allow procedures to call other procedures Fixes #88198 Release note (sql change): Stored procedures can now invoke other stored procedures via `CALL` statements. 120767: go.mod: bump Pebble to 7b8b3d5a8211 r=RaduBerinde a=jbowens Changes: * [`7b8b3d5a`](cockroachdb/pebble@7b8b3d5a) *,sstable: upgrade zstd to v1.5.6 * [`3a7021c5`](cockroachdb/pebble@3a7021c5) db: reuse backings for external ingestions This commit bumps the zstd dependency as well, working around #105568 by disabling the assembly routines for now. Informs #105568. Release note: none. Epic: none. 120776: cli: add support for remote files in debug range-data r=RaduBerinde a=itsbilal Previously, if a Checkpoint was created due to a storage-level replica corruption, we wouldn't be able to read the checkpoint if it contained any remote (shared or external) files as the pebble instance opened with `debug range-data` would not be started with the relevant RemoteStorageFactory. This change adds the same SharedStorage/RemoteStorageFactory params as the usual Pebble start in pkg/server/config.go, so that these checkpoints can be opened. Fixes #119960. Epic: None. Release note: None Co-authored-by: Marcus Gartner <[email protected]> Co-authored-by: Jackson Owens <[email protected]> Co-authored-by: Bilal Akhtar <[email protected]>
rharding6373
pushed a commit
to rharding6373/cockroach
that referenced
this issue
Mar 21, 2024
Previously, if a Checkpoint was created due to a storage-level replica corruption, we wouldn't be able to read the checkpoint if it contained any remote (shared or external) files as the pebble instance opened with `debug range-data` would not be started with the relevant RemoteStorageFactory. This change adds the same SharedStorage/RemoteStorageFactory params as the usual Pebble start in pkg/server/config.go, so that these checkpoints can be opened. Fixes cockroachdb#119960. Epic: None. Release note: None
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently, when we crash due to a range inconsistency, we checkpoint the range's contents into a checkpoint directory. Once cockroachdb/pebble#3362 updates the checkpoint operation to also store the object provider's catalog there, we should update
cockroach debug range-datato be able to read the catalog and also take in a RemoteStorageFactory so it can spit out all contents of the range, even if they're on external or shared files.Jira issue: CRDB-36414
The text was updated successfully, but these errors were encountered: