Runs accounts delta hash calculation in thread pool #3389
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Backports to the beta branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. Exceptions include CI/metrics changes, CLI improvements and documentation updates on a case by case basis. |
steviez
approved these changes
Oct 30, 2024
|
Merging this one so it is ready for the backport meeting 😸 |
mergify bot
pushed a commit
that referenced
this pull request
Oct 30, 2024
(cherry picked from commit ffd261e)
brooksprumo
added a commit
that referenced
this pull request
Oct 31, 2024
…#3389) (#3391) Runs accounts delta hash calculation in thread pool (#3389) (cherry picked from commit ffd261e) Co-authored-by: Brooks <[email protected]>
ray-kast
pushed a commit
to abklabs/agave
that referenced
this pull request
Nov 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
In Bank::freeze() we calculate the accounts delta hash for the slot. This is in the critical path, so we do the delta hash calc in parallel. Since an explicit thread pool is not specified, the delta hash calc ends up running in the global thread pool. This may not be ideal, if other tasks are running in the global pool (or get added later).
There is already a thread pool for foreground tasks, so we should use that one here.
h/t to @steviez for bringing this up
Summary of Changes
Run the accounts delta hash calc in the accounts-db foreground thread pool.
Results
Here's metrics from
mce3andmce4before and after picking up this change:(Note, mce3 restarted twice, which is why it has two spikes. mce4 only restarted once, hence one spike.)
Comparing pre and post, we see the accounts delta hash time got faster for both machines by quite a noticeable amount! ~1 ms for mce3 and ~0.5 ms for mce4.