Reorganize and add to benchmarks

Rearrange the benchmark code so that each implementation is self-contained. Adds the SierpinskiTriangle case that 'emotion' introduced in their fork of the 'react-native-web' benchmarks. And make it possible to run benchmarks on a per-library basis.
necolas · Dec 30, 2017 · 86263a2 · 86263a2
1 parent f6d1caa
commit 86263a2
Show file tree

Hide file tree

Showing 80 changed files with 847 additions and 227 deletions.
diff --git a/packages/benchmarks/README.md b/packages/benchmarks/README.md
@@ -6,41 +6,50 @@ To run these benchmarks:
 yarn benchmark
 ```
 
-Append `?fastest` to the URL to include the fastest "other libraries", and
-`?all` to include all the "other libraries".
+To run benchmarks for individual implementations append `?<name>,<name>` to the
+URL, e.g., `?css-modules,react-native-web`.
 
 ## Notes
 
+These benchmarks are crude approximations of extreme cases that libraries may
+encounter. The deep and wide tree cases look at the performance of mounting and
+rendering large trees of styled elements. The Triangle cases looks at the
+performance of repeated style updates to a large mounted tree. Some libraries
+must inject new styles for each "dynamic style", whereas others may not.
+Libraries without support for dynamic styles (i.e., they rely on user-authored
+inline styles) do not include the `SierpinskiTriangle` benchmark.
+
 The components used in the render benchmarks are simple enough to be
-implemented by multiple UI or style libraries. The implementations are not
-equivalent in functionality. For example, the "React Native for Web" benchmark includes a
-complete `View` implementation and the `StyleSheet` also converts React Native
-styles to DOM styles, has deterministic resolution, and supports RTL layout.
+implemented by multiple UI or style libraries. The benchmark implementations
+and the features of the style libraries are _only approximately equivalent in
+functionality_.
 
 ## Results
 
 Typical render timings*: mean ± two standard deviations.
 
-| Implementation                        | Deep tree (ms)    | Wide tree (ms)    | Tweets (ms)       |
+| Implementation                        | Deep tree (ms)    | Wide tree (ms)    | Triangle (ms)    |
 | :--- | ---: | ---: | ---: |
-| `css-modules`                         |  `80.47` `±25.13` | `144.87` `±32.70` | |
-| `[email protected]`              |  `88.68` `±28.78` | `178.17` `±39.90` | `13.78` `±2.90ms` |
+| `[email protected]`              |  `89.67` `±28.51` | `167.46` `±27.03` | `65.40` `±19.50` |
+| `css-modules`                         |  `77.42` `±45.50` | `141.44` `±33.96` | - |
+| `inline-styles`                       | `236.25` `±95.57` | `477.01` `±88.30` | `40.95` `±23.53` |
 
 Other libraries
 
-| Implementation                        | Deep tree (ms)    | Wide tree (ms)    |
+| Implementation                        | Deep tree (ms)    | Wide tree (ms)    | Triangle (ms)     |
 | :--- | ---: | ---: |
-| `[email protected]`                |  `79.41` `±27.49` | `152.95` `±29.46` |
-| `[email protected]`                     |  `85.13` `±25.39` | `162.87` `±25.91` |
-| `[email protected]`                      | `109.92` `±29.88` | `193.01` `±32.03` |
-| `react-jss@8.2.0`                     | `134.28` `±49.00` | `278.78` `±50.39` |
-| `emotion@8.0.12`                      | `139.08` `±46.18` | `253.45` `±52.69` |
-| `[email protected]`             | `194.43` `±46.28` | `404.86` `±56.59` |
-| `[email protected]`                      | `219.46` `±57.24` | `424.18` `±76.10` |
-| `[email protected]`                       | `359.32` `±90.27` | `795.91` `±88.93` |
+| `[email protected]`                |  `83.53` `±33.55` | `153.12` `±39.13` |  `56.47` `±24.22` |
+| `[email protected]`                     |  `88.23` `±31.22` | `164.03` `±34.70` | - |
+| `[email protected]`                      | `110.09` `±34.20` | `182.06` `±50.39` | ‡ |
+| `emotion@8.0.12`                      | `103.44` `±32.12` | `204.45` `±41.00` | `110.28` `±26.94` |
+| `react-jss@8.2.0`                     | `136.17` `±59.23` | `270.51` `±69.20` | - |
+| `[email protected]`             | `217.57` `±51.90` | `437.57` `±65.74` |  `76.99` `±41.79` |
+| `[email protected]`                      | `240.88` `±79.82` | `467.32` `±74.42` |  `70.95` `±32.90`|
+| `[email protected]`                       | `400.19` `±94.58` | `816.59` `±91.10` |  `71.13` `±27.22` |
 
 These results indicate that render times when using `react-native-web`,
 `css-modules`, `aphrodite`, and `styletron` are roughly equivalent and
 significantly faster than alternatives.
 
 *MacBook Pro (13-inch, Early 2011); 2.3 GHz Intel Core i5; 8 GB 1333 MHz DDR3. Google Chrome 62.
+‡Glamor essentially crashes
diff --git a/packages/benchmarks/index.js b/packages/benchmarks/index.js
diff --git a/packages/benchmarks/package.json b/packages/benchmarks/package.json
@@ -7,7 +7,9 @@
   },
   "dependencies": {
     "aphrodite": "^1.2.5",
+    "babel-polyfill": "^6.26.0",
     "classnames": "^2.2.5",
+    "d3-scale-chromatic": "^1.1.1",
     "emotion": "^8.0.12",
     "glamor": "^2.20.40",
     "marky": "^1.2.0",

diff --git a/packages/benchmarks/src/aphrodite.js b/packages/benchmarks/src/aphrodite.js
diff --git a/packages/benchmarks/benchmark.js → packages/benchmarks/src/benchmark.js b/packages/benchmarks/benchmark.js → packages/benchmarks/src/benchmark.js
@@ -38,30 +38,31 @@ const standardDeviation = values => {
   return Math.sqrt(meanSquareDiff);
 };
 
+export const log = (name, description, durations) => {
+  const stdDev = standardDeviation(durations);
+  const formattedMean = fmt(mean(durations));
+  const formattedMedian = fmt(median(durations));
+  const formattedStdDev = fmt(stdDev);
+
+  console.groupCollapsed(`${name}\n${formattedMean} ±${fmt(2 * stdDev)}`);
+  description && console.log(description);
+  console.log(`Median: ${formattedMedian}`);
+  console.log(`Mean: ${formattedMean}`);
+  console.log(`Standard deviation: ${formattedStdDev}`);
+  console.log(durations);
+  console.groupEnd();
+};
+
 const benchmark = ({ name, description, setup, teardown, task, runs }) => {
   return new Promise(resolve => {
     const durations = [];
     let i = 0;
 
     setup();
-    const first = measure('first', task);
     teardown();
 
     const done = () => {
-      const stdDev = standardDeviation(durations);
-      const formattedFirst = fmt(first);
-      const formattedMean = fmt(mean(durations));
-      const formattedMedian = fmt(median(durations));
-      const formattedStdDev = fmt(stdDev);
-
-      console.groupCollapsed(`${name}\n${formattedMean} ±${fmt(2 * stdDev)}`);
-      description && console.log(description);
-      console.log(`First: ${formattedFirst}`);
-      console.log(`Median: ${formattedMedian}`);
-      console.log(`Mean: ${formattedMean}`);
-      console.log(`Standard deviation: ${formattedStdDev}`);
-      console.log(durations);
-      console.groupEnd();
+      log(name, description, durations);
       resolve();
     };
 

diff --git a/...hmarks/src/components/NestedTree/index.js → packages/benchmarks/src/cases/NestedTree.js b/...hmarks/src/components/NestedTree/index.js → packages/benchmarks/src/cases/NestedTree.js
diff --git a/packages/benchmarks/src/cases/SierpinskiTriangle.js b/packages/benchmarks/src/cases/SierpinskiTriangle.js
@@ -0,0 +1,84 @@
+import PropTypes from 'prop-types';
+import React from 'react';
+import { interpolatePurples, interpolateBuPu, interpolateRdPu } from 'd3-scale-chromatic';
+
+const targetSize = 25;
+
+class SierpinskiTriangle extends React.Component {
+  static propTypes = {
+    Dot: PropTypes.node,
+    depth: PropTypes.number,
+    renderCount: PropTypes.number,
+    s: PropTypes.number,
+    x: PropTypes.number,
+    y: PropTypes.number
+  };
+
+  static defaultProps = {
+    depth: 0,
+    renderCount: 0
+  };
+
+  render() {
+    const { x, y, depth, renderCount, Dot } = this.props;
+    let { s } = this.props;
+
+    if (s <= targetSize) {
+      let fn;
+      switch (depth) {
+        case 1:
+          fn = interpolatePurples;
+          break;
+        case 2:
+          fn = interpolateBuPu;
+          break;
+        case 3:
+        default:
+          fn = interpolateRdPu;
+      }
+
+      return (
+        <Dot
+          color={fn(renderCount / 20)}
+          size={targetSize}
+          x={x - targetSize / 2}
+          y={y - targetSize / 2}
+        />
+      );
+    }
+
+    s /= 2;
+
+    return [
+      <SierpinskiTriangle
+        Dot={Dot}
+        depth={1}
+        key={1}
+        renderCount={renderCount}
+        s={s}
+        x={x}
+        y={y - s / 2}
+      />,
+      <SierpinskiTriangle
+        Dot={Dot}
+        depth={2}
+        key={2}
+        renderCount={renderCount}
+        s={s}
+        x={x - s}
+        y={y + s / 2}
+      />,
+      <SierpinskiTriangle
+        Dot={Dot}
+        depth={3}
+        key={3}
+        renderCount={renderCount}
+        s={s}
+        x={x + s}
+        y={y + s / 2}
+      />
+    ];
+  }
+}
+
+export default SierpinskiTriangle;
diff --git a/packages/benchmarks/tests/renderDeepTree.js → ...es/benchmarks/src/cases/renderDeepTree.js b/packages/benchmarks/tests/renderDeepTree.js → ...es/benchmarks/src/cases/renderDeepTree.js
@@ -1,10 +1,10 @@
 import createRenderBenchmark from '../createRenderBenchmark';
-import NestedTree from '../src/components/NestedTree';
+import NestedTree from './NestedTree';
 import React from 'react';
 
 const renderDeepTree = (label, components) =>
   createRenderBenchmark({
-    name: `Deep tree [${label}]`,
+    name: `[${label}] Deep tree`,
     runs: 20,
     getElement() {
       return <NestedTree breadth={3} components={components} depth={6} id={0} wrap={1} />;