crypto/merkle: pre-allocate data slice in innherHash #6443
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cuonglm
requested review from
alexanderbez,
cmwaters,
ebuchman,
tessr and
tychoish
as code owners
|
cc @odeke-em |
Codecov Report
@@ Coverage Diff @@
## master #6443 +/- ##
==========================================
+ Coverage 60.92% 60.97% +0.05%
==========================================
Files 287 287
Lines 27087 27091 +4
==========================================
+ Hits 16502 16518 +16
+ Misses 8883 8870 -13
- Partials 1702 1703 +1
|
tac0turtle
approved these changes
May 10, 2021
Done |
|
One other thing we could do much later on after this PR is to add a sync.Pool to reuse slices since the byte slices are passed into tmhash.Sum and read-only. |
tychoish
approved these changes
May 10, 2021
odeke-em
reviewed
May 10, 2021
odeke-em
reviewed
May 11, 2021
odeke-em
reviewed
May 11, 2021
crypto/merkle/hash.go
Outdated
| return tmhash.Sum(append(innerPrefix, append(left, right...)...)) | ||
| data := make([]byte, 0, len(innerPrefix)+len(left)+len(right)) | ||
| n := copy(data, innerPrefix) | ||
| n = copy(data[n:], left) |
There was a problem hiding this comment.
Oh oh, this is invalid, it has to be a: n += copy(data[n:], left) otherwise you'll just have the length of left
There was a problem hiding this comment.
Yeah, just realized it!
There was a problem hiding this comment.
Hmm, somehow, the copy always return 0...
There was a problem hiding this comment.
Yay, we need to pre-allocate all slice elem 😄
There was a problem hiding this comment.
Then the running time is nearly the same, compare using append vs copy:
name old time/op new time/op delta
HashAlternatives/recursive-8 22.8µs ± 0% 22.6µs ± 0% -0.69% (p=0.000 n=10+10)
HashAlternatives/iterative-8 21.9µs ± 0% 21.7µs ± 1% -0.75% (p=0.000 n=9+9)
So we can reduce pressure on runtime for checking that slice has enough capacity before appending. Benchmark result shows less memory usage and faster runtime: name old time/op new time/op delta HashAlternatives/recursive-8 25.2µs ± 1% 22.6µs ± 0% -9.98% (p=0.000 n=10+10) HashAlternatives/iterative-8 24.2µs ± 1% 21.7µs ± 1% -10.29% (p=0.000 n=10+9) name old alloc/op new alloc/op delta HashAlternatives/recursive-8 25.4kB ± 0% 19.1kB ± 0% -24.92% (p=0.000 n=10+10) HashAlternatives/iterative-8 28.1kB ± 0% 21.8kB ± 0% -22.54% (p=0.000 n=10+10) name old allocs/op new allocs/op delta HashAlternatives/recursive-8 497 ± 0% 398 ± 0% -19.92% (p=0.000 n=10+10) HashAlternatives/iterative-8 498 ± 0% 399 ± 0% -19.88% (p=0.000 n=10+10) Fixes #6442
|
@marbar3778 @tychoish please help us merge lest the PR gets stale. Thank you. |
JayT106
pushed a commit
to JayT106/tendermint
that referenced
this pull request
Sep 16, 2022
So we can reduce pressure on runtime for checking that slice has enough capacity before appending.
3 tasks
mergify bot
pushed a commit
that referenced
this pull request
Sep 21, 2022
Cherry-picking PR #6443 #### PR checklist - [x] Tests written/updated, or no tests needed - [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [x] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
So we can reduce pressure on runtime for checking that slice has enough
capacity before appending.
Benchmark result shows less memory usage and faster runtime:
Fixes #6442