Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance tracker #8851

Merged
merged 8 commits into from
Nov 29, 2024
Merged

Conversation

mehdi-aouadi
Copy link
Contributor

@mehdi-aouadi mehdi-aouadi commented Nov 26, 2024

PR Description

Update the performance tracker to support the new Electra attestation aggregation logic.
This prevents a memory leak triggered in Electra.

Fixed Issue(s)

Fixes #8847

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

@mehdi-aouadi mehdi-aouadi marked this pull request as ready for review November 28, 2024 16:12
@mehdi-aouadi
Copy link
Contributor Author

mehdi-aouadi commented Nov 28, 2024

I run some benchmarks to compare the old and the new attestations performance algorithm and the results are encouraging.

Old Version

Phase0 Attestations

Benchmark                                                    (attestationsCount)   Mode  Cnt      Score     Error  Units
PerformanceTrackerBenchmark.calculateAttestationPerformance                  100  thrpt   10  27119.554 ± 597.657  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance                 1000  thrpt   10   2558.557 ±  99.012  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance                10000  thrpt   10    161.646 ±   1.570  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance               100000  thrpt   10     10.397 ±   0.026  ops/s

Doesn't work for the electra attestations, so no Electra attestations benchmarking


New Version

Phase0 Attestations

Benchmark                                                    (attestationsCount)   Mode  Cnt      Score     Error  Units
PerformanceTrackerBenchmark.calculateAttestationPerformance                  100  thrpt   10  26828.725 ± 410.393  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance                 1000  thrpt   10   2638.513 ±  34.554  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance                10000  thrpt   10    160.236 ±   2.944  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance               100000  thrpt   10     11.682 ±   0.054  ops/s

Electra Attestations

Benchmark                                                    (attestationsCount)   Mode  Cnt      Score     Error  Units
PerformanceTrackerBenchmark.calculateAttestationPerformance                  100  thrpt   10  13805.354 ± 120.632  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance                 1000  thrpt   10   1733.083 ±   8.922  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance                10000  thrpt   10     98.000 ±   0.780  ops/s
PerformanceTrackerBenchmark.calculateAttestationPerformance               100000  thrpt   10      9.142 ±   0.030  ops/s

Context:

  • The operations refer to the calculateAttestationPerformance algorithm
  • The attestationsCount is the number of attestations being processed. They are all attesting from the same slot and hence all aggregated (which is extreme)

Observations:

  • The new algorithm performance is almost the same for the phase0 attestations
  • The Electra attestations require almost twice the processing time but that's normal since the aggregation bit data structure is more complex. If we take into consideration that there are less attestations in Electra (more aggregation) this should be fine

Copy link
Contributor

@rolfyone rolfyone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rolfyone
Copy link
Contributor

we'll need to cherrypick this to the mekong branch too just remember...

@mehdi-aouadi mehdi-aouadi enabled auto-merge (squash) November 29, 2024 08:59
@mehdi-aouadi mehdi-aouadi merged commit 49d8d48 into Consensys:master Nov 29, 2024
17 checks passed
@mehdi-aouadi mehdi-aouadi deleted the performance-tracker branch November 29, 2024 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance tracker memory leak
3 participants