Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move ideal_storage_size calculation before it is used #3250

Merged
merged 2 commits into from
Oct 29, 2024

Conversation

dmakarov
Copy link

Problem

The ideal_storage_size tuning parameter is incorrectly calculated after it needs to be used.

Summary of Changes

Move the update of the ideal_storage_size tuning parameter to an earlier point in the control flow before it is used.

Adjust tests.

) -> AncientSlotInfos {
let mut ancient_slot_infos = self.calc_ancient_slot_info(slots, tuning);
// ideal storage size is total alive bytes of ancient storages

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for this, i.e. after we call collect_sort_filter_ancinet_slots() make sure that ideal_storage_size is updated?

Copy link

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good to me. Please re-request a review when PR is ready for review again/has the test. Thanks!

@jeffwashington
Copy link

definitely looks like it is working:
image

it used to write 130GB each time, now it writes 57MB (5.7MB * 10).
We may still want to adjust the 10 constant. But, at least we are respecting the limit correctly now.

@dmakarov
Copy link
Author

definitely looks like it is working: image

it used to write 130GB each time, now it writes 57MB (5.7MB * 10). We may still want to adjust the 10 constant. But, at least we are respecting the limit correctly now.

that's good. the constant is for another PR.

Copy link

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link

@jeffwashington jeffwashington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link

@HaoranYi HaoranYi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Thanks for adding the test.

@dmakarov dmakarov merged commit 91d0a5e into anza-xyz:master Oct 29, 2024
40 checks passed
@dmakarov dmakarov deleted the iss branch October 29, 2024 16:20
ray-kast pushed a commit to abklabs/agave that referenced this pull request Nov 27, 2024
* Move ideal_storage_size calculation before it is used

* Add test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants