Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: spill to disk incurs unnecessary I/O costs #65623

Open
cucaroach opened this issue May 24, 2021 · 3 comments
Open

sql: spill to disk incurs unnecessary I/O costs #65623

cucaroach opened this issue May 24, 2021 · 3 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) E-quick-win Likely to be a quick win for someone experienced. E-starter Might be suitable for a starter project for new employees or team members. T-sql-queries SQL Queries Team T-storage Storage Team

Comments

@cucaroach
Copy link
Contributor

cucaroach commented May 24, 2021

As currently implemented spill to disk data is sync'd to the filesystem however we don't need to do this for correctness, in fact if there's ample memory the information need never leave the kernel's buffers since the data is ephemeral and only ever read by the writing process.

Jira issue: CRDB-7697

@cucaroach cucaroach added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label May 24, 2021
@jlinder jlinder added T-sql-queries SQL Queries Team T-storage Storage Team labels Jun 16, 2021
@jbowens
Copy link
Collaborator

jbowens commented Apr 27, 2022

This seems at odds with the periodic background syncing performed to smooth disk latencies. We don't have exposure into when the operating system might write back to the underlying storage, so we sync periodically to prevent an eventual writeback from causing a spike in write latencies.

Linking to cockroachdb/pebble#873 and cockroachdb/pebble#1159.

@mwang1026
Copy link

We should flip this flag in the tmp store cockroachdb/pebble#873

@nicktrav nicktrav added E-starter Might be suitable for a starter project for new employees or team members. E-quick-win Likely to be a quick win for someone experienced. labels Jun 6, 2022
@mgartner mgartner moved this to New Backlog in SQL Queries Jul 24, 2023
Copy link

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

@exalate-issue-sync exalate-issue-sync bot removed the T-sql-queries SQL Queries Team label Mar 15, 2024
@yuzefovich yuzefovich added the T-sql-queries SQL Queries Team label May 2, 2024
@jbowens jbowens moved this to Backlog in [Deprecated] Storage Jun 4, 2024
@exalate-issue-sync exalate-issue-sync bot removed the T-sql-queries SQL Queries Team label Jul 16, 2024
@yuzefovich yuzefovich added the T-sql-queries SQL Queries Team label Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) E-quick-win Likely to be a quick win for someone experienced. E-starter Might be suitable for a starter project for new employees or team members. T-sql-queries SQL Queries Team T-storage Storage Team
Projects
Status: Backlog
Status: Backlog
Development

No branches or pull requests

6 participants