Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic stress testing framework + counter.add test #1045

Merged
merged 9 commits into from
May 2, 2023

Conversation

lzchen
Copy link
Contributor

@lzchen lzchen commented Apr 25, 2023

Discussed in the weekly Rust SIG, creates a stress testing framework under the folder stress to test throughput of functions in our API.

Measures the amount of times a certain function is called within a sliding window (2 seconds, can be configured in the future) to using the max number of cpus available.

Note this provides basic functionality and only prints to the console.
Sample output:

Throughput: 412998.00 requests/sec
Throughput: 404226.00 requests/sec
Throughput: 417000.00 requests/sec
Throughput: 403440.00 requests/sec
Throughput: 395148.00 requests/sec
Throughput: 404172.00 requests/sec
Throughput: 395412.00 requests/sec
Throughput: 400740.00 requests/sec
Throughput: 390132.00 requests/sec
Total requests processed: 7418976

@lzchen lzchen requested a review from a team April 25, 2023 19:44
stress/README.md Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Apr 25, 2023

Codecov Report

Patch coverage: 2.2% and project coverage change: -0.2 ⚠️

Comparison is base (f42c11d) 55.1% compared to head (d7db365) 54.9%.

Additional details and impacted files
@@           Coverage Diff           @@
##            main   #1045     +/-   ##
=======================================
- Coverage   55.1%   54.9%   -0.2%     
=======================================
  Files        149     151      +2     
  Lines      18143   18242     +99     
=======================================
+ Hits       10006   10025     +19     
- Misses      8137    8217     +80     
Impacted Files Coverage Δ
stress/src/throughput.rs 0.0% <0.0%> (ø)
stress/src/metrics.rs 14.2% <14.2%> (ø)

... and 20 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jtescher
Copy link
Member

Running this locally looks like most of the time is spent waiting on the mutex contention when aggregating, we could consider more optimized methods for updating those sums. e.g. I can get more than 2x the current throughput by switching to a SkipMap for the sum's value map so there is likely quite a bit of headroom here.

Also the conversion into attribute sets will quickly become a bottleneck too once a test is added for that case, so we may want to consider an API change that allows the attribute sets to be precomputed.

Copy link
Member

@cijothomas cijothomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good start. we can iterate this in follow ups.

@TommyCpp TommyCpp merged commit cb65114 into open-telemetry:main May 2, 2023
@lzchen lzchen deleted the bench branch May 2, 2023 17:27
@lzchen
Copy link
Contributor Author

lzchen commented May 18, 2023

@jtescher

Running this locally looks like most of the time is spent waiting on the mutex contention when aggregating, we could consider more optimized methods for updating those sums. e.g. I can get more than 2x the current throughput by switching to a SkipMap for the sum's value map so there is likely quite a bit of headroom here.

I was not able to get any improvements when trying out SkipMap, perhaps I implemented it incorrectly. Could you maybe paste your code/submit a pr for an improvement on the current implementation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants