Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kv/bulk: send ssts with lower admission control priority #79126

Merged
merged 4 commits into from
Apr 4, 2022

Conversation

dt
Copy link
Member

@dt dt commented Mar 31, 2022

small refactor while I'm here since plumbing args to/from the kv.DB "helpers" has been a source of toil and merge conflicts lately.

Jira issue: CRDB-14737
Epic CRDB-2264

@dt dt requested review from erikgrinaker and stevendanna March 31, 2022 13:49
@dt dt requested a review from a team as a code owner March 31, 2022 13:49
@dt dt changed the title kv/bulk: send ssts with lower admission control priority. kv/bulk: send ssts with lower admission control priority Mar 31, 2022
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@dt dt requested a review from sumeerbhola March 31, 2022 14:33
@dt dt force-pushed the sst-admission branch from 269d48b to c2405d4 Compare March 31, 2022 14:36
@dt dt requested a review from a team March 31, 2022 14:36
@dt
Copy link
Member Author

dt commented Mar 31, 2022

first two commits are #79086, ignore them here.

dt added 3 commits March 31, 2022 17:17
passing args/unpacking returns is getting annoying.

Release note: none.
The detour through the kv.DB wrapper was just adding more plumbing, signatures, etc
to update to get an argument or return value through. Perhaps we should remove them
in a follow-up as this is the only non-test caller.

Release note: none.
Release note (performance improvement): Bulk ingestion writes now use a lower priority for admission control.
@dt dt force-pushed the sst-admission branch from c2405d4 to 05cb78a Compare March 31, 2022 17:17
@erikgrinaker
Copy link
Contributor

erikgrinaker commented Apr 1, 2022

The detour through the kv.DB wrapper was just adding more plumbing, signatures, etc
to update to get an argument or return value through. Perhaps we should remove them
in a follow-up as this is the only non-test caller.

Agreed, I find the non-trivial DB helpers to mostly be a hassle. Should make it easy to construct and send requests instead.

@dt
Copy link
Member Author

dt commented Apr 4, 2022

I'm a little wary about back-porting this so last minute, particularly given how much recent ingestion performance work we've been doing, so hopefully we can get it in asap so it has at least a few betas in which to bake.

@erikgrinaker
Copy link
Contributor

I'm a little wary about back-porting this so last minute, particularly given how much recent ingestion performance work we've been doing, so hopefully we can get it in asap so it has at least a few betas in which to bake.

I'm fine with merging this as-is.

@dt
Copy link
Member Author

dt commented Apr 4, 2022

I'm fine with merging this as-is.

👍 was just waiting on the open questions in #79086.

Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: for the admission control bits

Reviewed 1 of 1 files at r2, 1 of 1 files at r3, 1 of 2 files at r4, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @stevendanna)

@dt
Copy link
Member Author

dt commented Apr 4, 2022

TFTRs!

bors r+

@craig
Copy link
Contributor

craig bot commented Apr 4, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Apr 4, 2022

Build succeeded:

@blathers-crl
Copy link

blathers-crl bot commented Apr 4, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from ef29359 to blathers/backport-release-21.2-79126: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 21.2.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

@irfansharif
Copy link
Contributor

blathers backport 21.2

@blathers-crl
Copy link

blathers-crl bot commented Apr 21, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from ef29359 to blathers/backport-release-21.2-79126: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 21.2 failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bulk: AddSSTable should use lower admission priority
5 participants