Performance testing

[raimohanska]

No problems with performance so far, but it might still be interesting to see how many events a sec you can pump through a bacon system like

src.filter(f1).map(f2).flatMap(f3)

Having that metrics established it would be possible to tweak the fat out of it. If we did this with as many browsers as possible, we'd also know something about browser compatibility. But a command-line node-based metric would be a good start.

[eskimoblood]

Like to start this using https://github.com/bestiejs/benchmark.js, think the easiest is to start with test for node.

[raimohanska]

Ok, first performance test added in acf8dee.

Should add more cases there and then we can experiment with internal changes and see how the affect performance.

Some optimization ideas:

1. re-implement combineAsArray without using property.combine, i.e. combining all the streams at once.
2. lazy evaluation for event values.

The combination of these two changes might give a huge improvement in cases where a property is combined from a number of inputs but the value is actually used only when sampled by a stream. I have this case at work so this is relevant to my interests.

[raimohanska]

Well, with those optimizations the performance benchmark went from 120 to 150 ops per second, which is not a huge improvement.

[raimohanska]

Some more tweaking got me to 160 ops per second. This is all in the optimization branch. Feel free to do some tuning and see if you can get the metrics up. You can run the benchmark by

coffee performance/*

Remember to run tests too, to make sure you don't break it :)

./test

[raimohanska]

Note: I'm running the benchmark on an old MBP. Each "op" in the ops per sec is actually 100 events pushed to each of the 5 Buses that are hooked up into a combineTemplate/sampledBy system.

[RoboTeddy]

Anyone doing optimizations might find chrome's console.profile and
console.profileEnd useful (
https://developers.google.com/chrome-developer-tools/docs/commandline-api#profilename
)

On Tue, Jan 29, 2013 at 3:02 PM, Juha Paananen [email protected]:

Note: I'm running the benchmark on an old MBP. Each "op" in the ops per
sec is actually 100 events pushed to each of the 5 Buses that are hooked up
into a combineTemplate/sampledBy system.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/56#issuecomment-12863919.

[raimohanska]

Added some more perf test cases.

Bacon.combineTemplate.sample = complex combineTemplate, samples on every 10th change
Bacon.combineTemplate = same, but without sampling
EventStream.map = simple stream with map (x) -> x * 2

It seems that the optimizations (lazy etc) improves speed in the complex sampled case by more than 30%, and reduces speed in simpler cases by less than 5%. I might opt for including the performance adjustments as (for me) it's a common case to sample a complex compound Property quite seldom.

Results

Master

Bacon.combineTemplate.sample x 120 ops/sec ±4.39% (62 runs sampled)
Bacon.combineTemplate x 140 ops/sec ±4.39% (63 runs sampled)
EventStream.map x 825 ops/sec ±5.56% (64 runs sampled)

Optimization

Bacon.combineTemplate.sample x 161 ops/sec ±4.54% (58 runs sampled)
Bacon.combineTemplate x 134 ops/sec ±4.95% (56 runs sampled)
EventStream.map x 821 ops/sec ±4.44% (60 runs sampled)

[raimohanska]

Optimized Dispatcher.push by removing anon functions defined inside that function. New results:

Bacon.combineTemplate.sample x 200 ops/sec ±4.53% (71 runs sampled)
Bacon.combineTemplate x 146 ops/sec ±4.58% (61 runs sampled)
EventStream.map x 985 ops/sec ±4.81% (69 runs sampled)

[raimohanska]

Oh, there was a gorilla in the room. Removed assertions from event dispatching and got

Bacon.combineTemplate.sample x 328 ops/sec ±5.09% (65 runs sampled)
Bacon.combineTemplate x 197 ops/sec ±5.15% (56 runs sampled)
EventStream.map x 1,718 ops/sec ±3.68% (56 runs sampled)

[raimohanska]

Finally pushed the improvements to master.

[raimohanska]

Closing for now. Running perf tests in actual browsers would make sense though.