Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test-drive Criterion.rs #576

Merged
merged 4 commits into from
Jul 11, 2018
Merged

Test-drive Criterion.rs #576

merged 4 commits into from
Jul 11, 2018

Conversation

garious
Copy link
Contributor

@garious garious commented Jul 10, 2018

See: https://japaric.github.io/criterion.rs/book/criterion_rs.html

We gain:

  • The ability to do benchmarking from Rust stable
  • More meaningful metrics. Maybe reliable enough to block PRs introducing regressions via CI
  • gnuplot visualizations

We lose:

  • The ability to benchmark private functions
  • The use of #[bench]
  • The community's more popular workflow.

Sample command-line output:

bench_process_transaction
                        time:   [2.4687 ms 2.4882 ms 2.5087 ms]
                        change: [-24.796% -22.249% -19.699%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

Sample output from the HTML report:

	Lower bound	Estimate	Upper bound
Slope	2.5577 ms	2.6233 ms	2.7048 ms
R²	0.2362935	0.2435784	0.2324819
Mean	2.8168 ms	2.9047 ms	2.9977 ms
Std. Dev.	376.91 us	461.51 us	553.39 us
Median	2.6520 ms	2.8219 ms	3.0084 ms
MAD	339.46 us	489.37 us	594.31 us

bench_process_transaction.pdf

cc: #283

benches/bank.rs Outdated
let tx = Transaction::new(&rando0, rando1.pubkey(), 1, last_id);
assert!(bank.process_transaction(&tx.clone()).is_ok());

// Finally, return a the transaction to benchmark.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "return a the transaction"

@garious garious merged commit 0c6d2ef into solana-labs:master Jul 11, 2018
vkomenda pushed a commit to vkomenda/solana that referenced this pull request Aug 29, 2021
behzadnouri pushed a commit to behzadnouri/solana that referenced this pull request Apr 12, 2024
* change read-only-cache to keyed on pubkey only

* implement review feedbacks

---------

Co-authored-by: HaoranYi <[email protected]>
buffalojoec pushed a commit to buffalojoec/solana that referenced this pull request Apr 16, 2024
* change read-only-cache to keyed on pubkey only

* implement review feedbacks

---------

Co-authored-by: HaoranYi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants