Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase number of runs for the basic performance benchmarks #1303

Merged
merged 2 commits into from
Sep 7, 2023

Conversation

sequba
Copy link
Contributor

@sequba sequba commented Aug 30, 2023

Context

We observe a lot of noise in the performance testing results. 10 runs seem to be not enough. I increased it to 100.

How did you test your changes?

run benchmarks with the new config.

Types of changes

  • Breaking change (a fix or a feature because of which an existing functionality doesn't work as expected anymore)
  • New feature or improvement (a non-breaking change that adds functionality)
  • Bug fix (a non-breaking change that fixes an issue)
  • Additional language file, or a change to an existing language file (translations)
  • Change to the documentation
  • Internal tooling

Related issues:

#1293
#1298

Checklist:

  • I have reviewed the guidelines about Contributing to HyperFormula and I confirm that my code follows the code style of this project.
  • I have signed the Contributor License Agreement.
  • My change is compliant with the OpenDocument standard.
  • My change is compatible with Microsoft Excel.
  • My change is compatible with Google Sheets.
  • I described my changes in the CHANGELOG.md file.
  • My changes require a documentation update.
  • My changes require a migration guide.

@github-actions
Copy link

github-actions bot commented Aug 30, 2023

Performance comparison of head (0a6d589) vs base (ef996d2)

                                     testName |   base |    head |  change
--------------------------------------------------------------------------
                                      Sheet A | 1122.5 | 1062.72 |  -5.33%
                                      Sheet B |  410.6 |   408.9 |  -0.41%
                                      Sheet T |    348 |  343.11 |  -1.41%
                                Column ranges |  853.4 |  857.24 |  +0.45%
Sheet A:  change value, add/remove row/column |     51 |      54 |  +5.88%
 Sheet B: change value, add/remove row/column |    441 |     426 |  -3.40%
                   Column ranges - add column |    389 |     382 |  -1.80%
                Column ranges - without batch |   1019 |     827 | -18.84%
                        Column ranges - batch |    493 |     231 | -53.14%

@sequba sequba self-assigned this Sep 6, 2023
@sequba sequba requested a review from budnix September 6, 2023 14:13
@sequba sequba merged commit 0e39178 into develop Sep 7, 2023
21 checks passed
@sequba sequba deleted the more-accurate-performance-benchmarks branch September 7, 2023 08:37
@AMBudnik
Copy link
Contributor

Released in v2.6.0

Copy link

codecov bot commented Oct 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.23%. Comparing base (ef996d2) to head (0a6d589).
Report is 80 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #1303   +/-   ##
========================================
  Coverage    97.23%   97.23%           
========================================
  Files          167      167           
  Lines        14304    14304           
  Branches      3065     3065           
========================================
  Hits         13909    13909           
  Misses         395      395           

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants