Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.17: Use BankForks on tests - Part 1 (backport of #34206) #34354

Closed
wants to merge 2 commits into from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Dec 7, 2023

This is an automatic backport of pull request #34206 done by Mergify.
Cherry-pick of e832765 has failed:

Fixing these tests are needed to remove WorkingSlot from LoadedPrograms cache. We need to remove WorkingSlot to support cooperative program loading (for cold programs). The cooperative program loading is needed to be backported to v1.17, unless we find a minimal solution that works.

---------

Signed-off-by: Lucas Steuernagel <[email protected]>
(cherry picked from commit e832765)

# Conflicts:
#	runtime/src/bank/tests.rs
@mergify mergify bot added the conflicts label Dec 7, 2023
@brooksprumo
Copy link
Contributor

Hi, @LucasSte. Can you include why this PR should/needs to be backported?

@pgarg66
Copy link
Contributor

pgarg66 commented Dec 7, 2023

Hi, @LucasSte. Can you include why this PR should/needs to be backported?

Fixing these tests are needed to remove WorkingSlot from LoadedPrograms cache. And, we need to remove WorkingSlot to support cooperative program loading (for cold programs). The cooperative program loading is needed to be backported to v1.17, unless we find a minimal solution that works.

@CriesofCarrots
Copy link
Contributor

The cooperative program loading is needed to be backported to v1.17, unless we find a minimal solution that works.

Why? Can you please explain more? Is there a bug this is fixing?

@pgarg66
Copy link
Contributor

pgarg66 commented Dec 7, 2023

The cooperative program loading is needed to be backported to v1.17, unless we find a minimal solution that works.

Why? Can you please explain more? Is there a bug this is fixing?

The problem manifests as a performance issue during loading of missing programs in the cache. If multiple transaction batches try to load the same program, they all JIT the same program in parallel. That results into unnecessary work, and memory thrashing, leading to other issues.

Here's the PR that will fix it, but it's dependent on fixing these tests: #33715

.unwrap()
.insert(bank)
.clone_without_scheduler()
bank_forks.write().unwrap().insert(bank)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Insert returns BankWithScheduler, but the function returns Arc<Bank>, so I think we need the clone_without_scheduler here, don't we?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v1.17 doesn't seem to have that.

Copy link

codecov bot commented Dec 7, 2023

Codecov Report

Merging #34354 (6e2ee79) into v1.17 (d17243f) will decrease coverage by 0.1%.
Report is 2 commits behind head on v1.17.
The diff coverage is 98.6%.

Additional details and impacted files
@@            Coverage Diff            @@
##            v1.17   #34354     +/-   ##
=========================================
- Coverage    81.8%    81.8%   -0.1%     
=========================================
  Files         803      803             
  Lines      218106   218152     +46     
=========================================
+ Hits       178538   178561     +23     
- Misses      39568    39591     +23     

@pgarg66 pgarg66 closed this Dec 13, 2023
@mergify mergify bot deleted the mergify/bp/v1.17/pr-34206 branch December 13, 2023 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants