Skip to content
This repository has been archived by the owner on Nov 6, 2020. It is now read-only.

Missing benchmarks #9152

Closed
holiman opened this issue Jul 17, 2018 · 5 comments
Closed

Missing benchmarks #9152

holiman opened this issue Jul 17, 2018 · 5 comments
Labels
M4-core ⛓ Core client code / Rust. Z1-question 🙋‍♀️ Issue is a question. Closer should answer.
Milestone

Comments

@holiman
Copy link
Contributor

holiman commented Jul 17, 2018

Before Byzantium, we did some benchmarks on the metropolis-update branch, and got these nice benchmarks when doing cargo bench -p ethcore:

running 20 tests
test bn_128_add          ... bench:      11,291 ns/iter (+/- 390)
test bn_128_mul          ... bench:     683,165 ns/iter (+/- 24,812)
test bn_128_pairing      ... bench:   7,500,475 ns/iter (+/- 214,431)
test ecrecover           ... bench:     143,070 ns/iter (+/- 4,080)
test modexp_1_pow0x10001 ... bench:      69,577 ns/iter (+/- 4,967)
test modexp_1_qube       ... bench:      12,746 ns/iter (+/- 343)
test modexp_1_square     ... bench:       9,005 ns/iter (+/- 417)
test modexp_2_pow0x10001 ... bench:     178,520 ns/iter (+/- 7,649)
test modexp_2_qube       ... bench:      30,359 ns/iter (+/- 1,987)
test modexp_2_square     ... bench:      20,965 ns/iter (+/- 951)
test modexp_3_pow0x10001 ... bench:     524,879 ns/iter (+/- 12,167)
test modexp_3_qube       ... bench:      89,207 ns/iter (+/- 2,677)
test modexp_3_square     ... bench:      60,163 ns/iter (+/- 1,230)
test modexp_4_pow0x10001 ... bench:   1,664,471 ns/iter (+/- 109,330)
test modexp_4_qube       ... bench:     279,185 ns/iter (+/- 10,925)
test modexp_4_square     ... bench:     188,611 ns/iter (+/- 2,883)
test modexp_5_pow0x10001 ... bench:   5,685,696 ns/iter (+/- 213,275)
test modexp_5_qube       ... bench:     951,783 ns/iter (+/- 31,612)
test modexp_5_square     ... bench:     641,167 ns/iter (+/- 15,356)
test sha256              ... bench:       1,443 ns/iter (+/- 74)

When I redid it on master now, I only get these:

running 4 tests
test bn_128_mul     ... bench:     709,905 ns/iter (+/- 223,458)
test bn_128_pairing ... bench:   7,273,317 ns/iter (+/- 956,445)
test ecrecover      ... bench:     153,535 ns/iter (+/- 36,858)
test sha256         ... bench:         949 ns/iter (+/- 134)

Could someone please resurrect the old benchmarks that are not present on master?

@Tbaut Tbaut added Z1-question 🙋‍♀️ Issue is a question. Closer should answer. M4-core ⛓ Core client code / Rust. labels Jul 17, 2018
@Tbaut Tbaut added this to the 2.0 milestone Jul 17, 2018
@5chdn 5chdn modified the milestones: 2.0, 2.1 Jul 17, 2018
@holiman
Copy link
Contributor Author

holiman commented Jul 19, 2018

cc @arkpar -- do you know what happened to them?

@arkpar
Copy link
Collaborator

arkpar commented Jul 19, 2018

Not sure why they did not make it to master. I've restored the branch though.

@arkpar
Copy link
Collaborator

arkpar commented Jul 19, 2018

These commits should be merged:
ea60a52
e93a68c
fe3cfa8

@andresilva
Copy link
Contributor

Some of them did make it to master, added the missing ones in #9170.

@holiman
Copy link
Contributor Author

holiman commented Jul 20, 2018

Thanks! Would be great if you can help me determine the gas cost of each testcase/benchmark

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
M4-core ⛓ Core client code / Rust. Z1-question 🙋‍♀️ Issue is a question. Closer should answer.
Projects
None yet
Development

No branches or pull requests

5 participants