Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
eth2util/keystore: lower amount of loadStoreWorkers (#2584)
When running `create cluster` with the `--split-existing-keys` flag, I ran into the charon service getting OOM killed. A bit of trial and error helped me figure this was related to the amount of keys to be split. A low amount (<30) would work fine. I found out splitting 25 existing keys resulted in a memory consumption of over 6GB, 10 keys used about 2.5GB. Curious to see what would lead to such high memory consumption, I built the binary locally and extracted a heap dump. The heap dump showed that pretty much all the memory was consumed by `golang.org/x/crypto/scrypt.Key objects` , 256MB per key (temporarily even 512MB is allocated if I'm looking at the data correctly). 64 workers (current value) * 256MB per key results in an upper limit of more than 16GB of memory consumed, with an even higher temporary spike. This PR lowers the amount of workers to 10, resulting in an upper limit of 10 * 512MB = 5GB of memory consumption. I didn't investigate why the scrypt.Key object itself needs as much memory as it seems to be using, I'm not that well versed at Go. But it does seem a bit excessive. Fixing that would likely be the more proper fix. Let me know if this deserves an issue, but it seemed trivial enough to me. category: bug ticket: none
- Loading branch information