Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider using pooled buffers in util.Hash #141

Closed
davecgh opened this issue Apr 24, 2016 · 1 comment
Closed

Consider using pooled buffers in util.Hash #141

davecgh opened this issue Apr 24, 2016 · 1 comment
Assignees

Comments

@davecgh
Copy link

davecgh commented Apr 24, 2016

I have been doing some profiling recently on btcsuite/btcd and both the number one and number two functions in the entire app that cause the vast majority of allocations, and hence GC pressure, are the util.Hash function and the binary.Readthat it does.

The following is a cut down version from the profile. As you can see it is a whopping ~63% of all allocations!

      flat  flat%
6074945567 36.09% github.com/syndtr/goleveldb/leveldb/util.Hash
4527845906 26.90% encoding/binary.Read
  73068951  0.43% github.com/syndtr/goleveldb/leveldb/cache.(*Cache).Get
...

Listing the util.Hash function reveals:

     14            .          . // Hash return hash of the given data. 
     15            .          . func Hash(data []byte, seed uint32) uint32 { 
     16            .          .     // Similar to murmur hash 
     17            .          .     var m uint32 = 0xc6a4a793 
     18            .          .     var r uint32 = 24 
     19            .          .     h := seed ^ (uint32(len(data)) * m) 
     20            .          .  
     21   1092928017 1092928017     buf := bytes.NewBuffer(data) 
     22            .          .     for buf.Len() >= 4 { 
     23   4982017550 4982017550         var w uint32 
     24            . 4527845812         binary.Read(buf, binary.LittleEndian, &w) 
     25            .          .         h += w 
     26            .          .         h *= m 
     27            .          .         h ^= (h >> 16) 
     28            .          .     }
@syndtr syndtr self-assigned this Apr 25, 2016
@syndtr syndtr closed this as completed in a49846e Apr 25, 2016
@davecgh
Copy link
Author

davecgh commented Apr 29, 2016

I've been running with this for a while now and can confirm it is no longer an issue. I appreciate your rapid fixes. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants