Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: allow setting IngestSplit #115430

Closed
nicktrav opened this issue Dec 1, 2023 · 1 comment · Fixed by #115462
Closed

storage: allow setting IngestSplit #115430

nicktrav opened this issue Dec 1, 2023 · 1 comment · Fixed by #115462
Assignees
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker T-storage Storage Team

Comments

@nicktrav
Copy link
Collaborator

nicktrav commented Dec 1, 2023

Pebble now has support for virtualizing tables on ingest. This is currently controlled via a Pebble configuration option, IngestSplit, defaulting to false.

Enable dynamically altering this param, via a cluster setting, so that it can be trialled on 23.2 clusters.

We should also default to true on master.

Jira issue: CRDB-34030

@nicktrav nicktrav added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-storage Relating to our storage engine (Pebble) on-disk storage. T-storage Storage Team labels Dec 1, 2023
@itsbilal itsbilal self-assigned this Dec 1, 2023
@nicktrav
Copy link
Collaborator Author

nicktrav commented Dec 4, 2023

Adding GA blocker label, as we should have this for 23.2.

@nicktrav nicktrav added GA-blocker branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 labels Dec 4, 2023
craig bot pushed a commit that referenced this issue Dec 5, 2023
115447: roachtest: indicate when version is too old in `UploadWorkload` r=DarrylWong,srosenberg a=renatolabs

In #115061, a `UploadWorkload` API was added to allow tests to upload "arbitrary" versions of the `workload` binary, especially useful in upgrade testing.

However, we only have `workload` binaries for v22.2+. In this commit, we add a `bool` return value to `UplodWorkload` indicating whether a binary was actually uploaded. Callers are expected to check this value before attempting to use a workload binary.

The `acceptance/version-upgrade` test is updated in this commit in the way described.

Epic: none

Release note: None

115462: storage: add cluster setting for ingest splits r=RaduBerinde a=itsbilal

Previously, the ingest-time splitting feature was only tunable through a code change. This change adds a cluster setting to allow this feature to be turned on dynamically.

Fixes #115430.

Epic: none

Release note: None

115593: ui: change some references to tenant to virtual cluster r=dhartunian a=stevendanna

This changes the "Tenant" selector on the metrics pages to use the new "Virtual Cluster" terminology.

Fixes #114170

Release note: None

<img width="1037" alt="Screenshot 2023-12-05 at 11 05 10" src="https://github.com/cockroachdb/cockroach/assets/852371/54f9ce9c-fc0b-4457-8214-de22c45b32bc">
<img width="1086" alt="Screenshot 2023-12-05 at 11 05 05" src="https://github.com/cockroachdb/cockroach/assets/852371/d0cb2f32-20a6-4522-8727-3c19ce0a91ab">


115595: server: support strict MemFS in StickyVFSRegistry r=jbowens a=pavelkalinnikov

The strict `MemFS` can be used for emulating crashes, and testing for data durability loss that they may cause if fsync is not used correctly.

Touches #113135, #114411
Epic: none
Release note: none

Co-authored-by: Renato Costa <[email protected]>
Co-authored-by: Bilal Akhtar <[email protected]>
Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: Pavel Kalinnikov <[email protected]>
@craig craig bot closed this as completed in 57ac7b6 Dec 5, 2023
blathers-crl bot pushed a commit that referenced this issue Dec 5, 2023
Previously, the ingest-time splitting feature was only
tunable through a code change. This change adds a cluster
setting to allow this feature to be turned on dynamically.

Fixes #115430.

Epic: none

Release note: None
itsbilal added a commit that referenced this issue Dec 6, 2023
Previously, the ingest-time splitting feature was only
tunable through a code change. This change adds a cluster
setting to allow this feature to be turned on dynamically.

Fixes #115430.

Epic: none

Release note: None

Release justification: low-risk addition of a cluster setting to
be able to toggle a new, experimental feature.
@jbowens jbowens moved this to Done in [Deprecated] Storage Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker T-storage Storage Team
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants