Skip to content
This repository has been archived by the owner on Jan 13, 2025. It is now read-only.

Remove potentially too costly Packets::default() #14821

Merged
merged 4 commits into from
Jan 29, 2021

Conversation

ryoqun
Copy link
Contributor

@ryoqun ryoqun commented Jan 25, 2021

Problem

I needed some cleanup while creating #14806 to ensure there is no similar issue.

The root problem is that current api is too easy to misuse. This follow-up pr is to fix the root cause and never reoccur similar issue in the future.

Summary of Changes

  • Removed Packets::default(). As far as I read the code, this not need to be Default. By convention, Default::default() should be cheap but Pakcets violates that by over-allocating silently.
    • Also, implementing Default generally opens unbounded call-graph. So, I want to avoid it unless there is clear benefit/necessity to define Default.
  • Then, identical functionality is renamed to Packets::for_test_and_bench to aptly indicate the purpose. New name is intentionally verbose and indicates this shouldn't be used in the production code.
    • Also checked that all current call-sites are tests and benches. (Yeah, I did this boring work once for all... lol)
  • Demoted to_packets to #cfg(test), another source of potential cause of overallocation. It seems that production code doesn't actually need it as well.

This pr's cleanup motto is that let's make the ultimate call-site decide the allocation behavior by explicit call of to_packets_chunked.

This pr won't be back-ported to v1.4 intentionally. Maybe not worth do it.

Also, there is no functional change.

Fixes #

@ryoqun ryoqun added the v1.5 label Jan 25, 2021
@ryoqun ryoqun requested a review from sakridge January 25, 2021 05:53
@codecov
Copy link

codecov bot commented Jan 25, 2021

Codecov Report

Merging #14821 (5fb74f2) into master (015058e) will increase coverage by 0.0%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##           master   #14821   +/-   ##
=======================================
  Coverage    80.3%    80.3%           
=======================================
  Files         403      403           
  Lines      102747   102734   -13     
=======================================
- Hits        82551    82542    -9     
+ Misses      20196    20192    -4     

@sakridge
Copy link
Contributor

How about just making Packets::default do a PinnedVec::default which just does a Vec::new(). Only reason to do with_capacity is performance, but if it's just used in the test context, then we don't really need that. I'm a bit mixed on the rename to test_or_bench.

I don't think the benchmarks do much allocating in the timing sections. In general it generates the packet vector up front, then runs the code on that.

@ryoqun
Copy link
Contributor Author

ryoqun commented Jan 28, 2021

How about just making Packets::default do a PinnedVec::default which just does a Vec::new(). Only reason to do with_capacity is performance, but if it's just used in the test context, then we don't really need that. I'm a bit mixed on the rename to test_or_bench.

I don't think the benchmarks do much allocating in the timing sections. In general it generates the packet vector up front, then runs the code on that.

@sakridge Thanks for review! That makes sense. I reverted most of changes. How does this pr look now?

Copy link
Contributor

@sakridge sakridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ryoqun ryoqun merged commit d6873b8 into solana-labs:master Jan 29, 2021
mergify bot pushed a commit that referenced this pull request Jan 29, 2021
* Remove potentially too costly Packets::default()

* Fix test...

* Restore Packets::default()

* Restore Packets::default() more

(cherry picked from commit d6873b8)
ryoqun added a commit that referenced this pull request Jan 29, 2021
* Remove potentially too costly Packets::default()

* Fix test...

* Restore Packets::default()

* Restore Packets::default() more

(cherry picked from commit d6873b8)

Co-authored-by: Ryo Onodera <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants