-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Add benchmark scaffolding #29
Conversation
Codecov Report
@@ Coverage Diff @@
## master #29 +/- ##
==========================================
- Coverage 98.82% 98.38% -0.44%
==========================================
Files 2 2
Lines 85 186 +101
==========================================
+ Hits 84 183 +99
- Misses 1 3 +2
Continue to review full report at Codecov.
|
test/bench/bench-batch.js
Outdated
var fs = require('fs'); | ||
var path = require('path'); | ||
var argv = require('minimist')(process.argv.slice(2)); | ||
var tape = require('tape'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@GretaCB I would propose not depending on tape
for the benchmark. Since it is not critical and does not offer anything too useful. For the assert
below you could use the var assert = require('assert')
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 awesome thank you! just committed the change.
…th minimal JS logic
@GretaCB cf913b8 adds a very expensive usage of ~/projects/node-cpp-skel[bench]$ time node test/bench/bench-batch.js --iterations 10 --concurrency 10
real 0m3.491s
user 0m11.704s
sys 0m0.535s I get my CPU usage spiking to > 500%. Running |
Currently working on adding a couple more benchmark scenarios before merging.
|
per chat with @GretaCB - next I'm going to take a look at profiling and tuning a few things in the PR. In particular I'll look at the impl of the mutex lock and make sure there is enough work being done in the async function that locks the global such that we are properly demonstrating (e.g. providing an example you can profile) the kind of thread contention programmers should avoid. |
Done in fb1fca8. Now the
If you bump up When locks are unavoidable in real-world applications, we would hope that the % of time spent in |
Just looked back at this branch. It's got some great stuff that I think we should merge soon and keep iterating on. My one reservation before merging I'm slightly uncomfortable with how we are mixing best practice/simple/hello-world style code with new code demonstrating both advanced and non-ideal scenarios. I think we should split things apart before merging such that:
|
@GretaCB I just returned here to reflect on next steps. I feel like a good approach would be to split this work into 2 phases: phase 1Land a first PR that:
^^ this gets the key structure in place to make it easy and fast to start benchmarks for any module that uses skel. This is a great first step. phase 2We could revisit adding examples of performance scenarios. However I feel like this is a really advanced topic most suitable outside of node-cpp-skel. The skel is complex enough currently without diving deep on performance. But given performance is critical it would be great to cover it and benefit from the skel structure. So, here is an idea. Instead of building into the skel directly, we could:
|
@GretaCB now that Phase 1 is done in #61 - how about closing this ticket? Phase 2 remains but I feel like my idea is not concrete enough to warrant a ticket. I'm feeling really good with what we have and don't see a major need to ticket more work here. Rather we'll apply node-cpp-skel, learn perf issues, and then - at that time - have ideas of things to build back, or add to the docs. |
Closing this. What we have are:
|
Per #25
process.memoryUsage()
. Thinking about what info is useful heresleep
option for mocking when threads are busy, but aren't doing much workPer chat with @springmeyer:
cc @mapsam @springmeyer