Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve block_sim speed with 50k+ validators #1294

Merged
merged 1 commit into from
Jul 10, 2020
Merged

Improve block_sim speed with 50k+ validators #1294

merged 1 commit into from
Jul 10, 2020

Conversation

tersec
Copy link
Contributor

@tersec tersec commented Jul 9, 2020

More progress towards running 50k or 100k validators smoothly on block_sim.

  • get_beacon_committee() was pointlessly calling the full version of get_committee_count_at_slot(), when it's just a simple arithmetic function of the number of active validators, which get_beacon_committee() already has.

  • get_inclusion_delay_deltas() is part of the epoch processing, which had a quadratically growing part.

  • the get_committee_assignment() / installValidatorApiHandlers() changes prepare for dynamically subscribing and unsubscribing to attestation subnets.

Before:

$ time build/block_sim --validators=100000 --slots=64
Loaded genesim_mainnet_100000_0.12.1.ssz...
Starting simulation...
................................ slot: 32 epoch: 1
................................ slot: 64 epoch: 2
Done!
Validators: 100000, epoch length: 32
Validators per attestation (mean): 0.0
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
    4025.887,     2255.056,     2237.409,    15992.623,           62, Process non-epoch slot with block
   21661.675,    16770.872,     9802.878,    33520.473,            2, Process epoch slot with block
       4.948,        0.878,        0.018,        5.684,           64, Tree-hash block
       1.562,        0.103,        1.385,        1.842,           64, Sign block
    7754.820,      156.415,     7471.934,     8206.023,           64, Have committee attest to block
       3.745,        0.000,        3.745,        3.745,            1, Replay all produced blocks

real	13m29.612s
user	13m18.196s
sys	0m2.752s

After:

$ time build/block_sim --validators=100000 --slots=64
Loaded genesim_mainnet_100000_0.12.1.ssz...
Starting simulation...
................................ slot: 32 epoch: 1
................................ slot: 64 epoch: 2
Done!
Validators: 100000, epoch length: 32
Validators per attestation (mean): 0.0
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
    3859.758,     2263.392,     2142.557,    16094.623,           62, Process non-epoch slot with block
   11636.219,     3293.731,     9307.200,    13965.239,            2, Process epoch slot with block
       4.811,        0.852,        0.023,        5.823,           64, Tree-hash block
       1.521,        0.087,        1.355,        1.764,           64, Sign block
    4311.929,      142.398,     3978.022,     4696.124,           64, Have committee attest to block
       3.924,        0.000,        3.924,        3.924,            1, Replay all produced blocks

real	9m16.943s
user	9m9.943s
sys	0m1.356s

Non-epoch slot times only go down slightly -- that was worth fixing, but was coming up when reloading an epochRef into StateCache, where the latter was being incompletely repopulated due to EpochRef not containing all the same information. In particular, it was coming up in onAttestation, which doesn't have any other source for its cache.

Epoch slot times are reduced by 2 to 3 times.

The combination creates a ~40% increase in block sim speed overall at 100k validators. It doesn't seem to matter nearly as much up to, say, 20k validators.

…-time inclusion delay rewards/palties in epoch processing; create infrastructure for dynamically (un)subscribing to/from attestation topics
@tersec tersec merged commit 5d5957f into devel Jul 10, 2020
@tersec tersec deleted the mao branch July 10, 2020 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants