Benchmarks and results?

[kozross]

In the package description on Hackage, you claim that:

In many benchmarks, the performance improvement is 2x-10x.

Which benchmarks are these? Are there some results we can look at?

[takano-akio]

You can find some benchmarks under the benchmarks directory: https://github.com/takano-akio/fast-builder/tree/master/benchmarks

On my machine they give results like this (bstr is the builder from the bytestring package, fast is this library):

Preprocessing benchmark 'aeson' for fast-builder-0.1.2.1..
Building benchmark 'aeson' for fast-builder-0.1.2.1..
Running 1 benchmarks...
Benchmark aeson: RUNNING...
benchmarking fast
time                 640.8 μs   (638.4 μs .. 642.8 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 644.0 μs   (641.8 μs .. 647.3 μs)
std dev              9.285 μs   (6.663 μs .. 14.98 μs)

benchmarking bstr
time                 979.1 μs   (974.4 μs .. 983.3 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 976.8 μs   (972.9 μs .. 983.0 μs)
std dev              15.82 μs   (11.40 μs .. 25.61 μs)

benchmarking aeson
time                 951.6 μs   (947.1 μs .. 956.4 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 955.0 μs   (950.3 μs .. 959.6 μs)
std dev              15.23 μs   (13.08 μs .. 17.89 μs)

Benchmark aeson: FINISH
Preprocessing benchmark 'map' for fast-builder-0.1.2.1..
Building benchmark 'map' for fast-builder-0.1.2.1..
Running 1 benchmarks...
Benchmark map: RUNNING...
benchmarking 10/lazy/fast
time                 340.3 ns   (335.0 ns .. 351.5 ns)
                     0.997 R²   (0.991 R² .. 1.000 R²)
mean                 337.5 ns   (335.5 ns .. 344.5 ns)
std dev              11.80 ns   (1.471 ns .. 24.91 ns)
variance introduced by outliers: 51% (severely inflated)

benchmarking 10/lazy/bstr
time                 1.040 μs   (1.035 μs .. 1.043 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.038 μs   (1.035 μs .. 1.041 μs)
std dev              9.497 ns   (7.459 ns .. 14.01 ns)

benchmarking 100/lazy/fast
time                 2.562 μs   (2.551 μs .. 2.574 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 2.562 μs   (2.548 μs .. 2.590 μs)
std dev              68.43 ns   (41.12 ns .. 118.4 ns)
variance introduced by outliers: 34% (moderately inflated)

benchmarking 100/lazy/bstr
time                 7.249 μs   (7.217 μs .. 7.284 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 7.258 μs   (7.234 μs .. 7.291 μs)
std dev              88.56 ns   (68.17 ns .. 114.4 ns)

benchmarking 1000/lazy/fast
time                 23.21 μs   (22.97 μs .. 23.47 μs)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 23.17 μs   (23.00 μs .. 23.46 μs)
std dev              773.8 ns   (490.4 ns .. 1.085 μs)
variance introduced by outliers: 37% (moderately inflated)

benchmarking 1000/lazy/bstr
time                 73.13 μs   (72.76 μs .. 73.52 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 73.35 μs   (72.85 μs .. 74.06 μs)
std dev              2.032 μs   (1.466 μs .. 2.753 μs)
variance introduced by outliers: 26% (moderately inflated)

benchmarking 10000/lazy/fast
time                 240.3 μs   (237.2 μs .. 243.2 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 237.6 μs   (236.5 μs .. 239.4 μs)
std dev              4.541 μs   (3.461 μs .. 5.861 μs)
variance introduced by outliers: 11% (moderately inflated)

benchmarking 10000/lazy/bstr
time                 746.6 μs   (740.0 μs .. 754.4 μs)
                     0.999 R²   (0.999 R² .. 0.999 R²)
mean                 752.5 μs   (745.7 μs .. 757.8 μs)
std dev              20.54 μs   (17.27 μs .. 24.64 μs)
variance introduced by outliers: 17% (moderately inflated)

Benchmark map: FINISH
Preprocessing benchmark 'vector' for fast-builder-0.1.2.1..
Building benchmark 'vector' for fast-builder-0.1.2.1..
Running 1 benchmarks...
Benchmark vector: RUNNING...
benchmarking 10/lazy/fast
time                 240.0 ns   (239.5 ns .. 240.7 ns)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 239.6 ns   (239.3 ns .. 240.5 ns)
std dev              1.610 ns   (687.3 ps .. 3.134 ns)

benchmarking 10/lazy/bstr
time                 1.411 μs   (1.410 μs .. 1.413 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.412 μs   (1.411 μs .. 1.414 μs)
std dev              5.155 ns   (3.017 ns .. 9.029 ns)

benchmarking 10/strict/fast
time                 188.1 ns   (187.6 ns .. 188.7 ns)
                     1.000 R²   (0.998 R² .. 1.000 R²)
mean                 188.8 ns   (187.6 ns .. 194.5 ns)
std dev              7.543 ns   (836.1 ps .. 17.25 ns)
variance introduced by outliers: 59% (severely inflated)

benchmarking 10/io/fast
time                 406.7 ns   (405.6 ns .. 408.0 ns)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 408.4 ns   (405.7 ns .. 421.1 ns)
std dev              17.18 ns   (1.748 ns .. 39.45 ns)
variance introduced by outliers: 60% (severely inflated)

benchmarking 10/io/bstr
time                 1.291 μs   (1.290 μs .. 1.293 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.295 μs   (1.290 μs .. 1.327 μs)
std dev              34.56 ns   (3.797 ns .. 84.23 ns)
variance introduced by outliers: 35% (moderately inflated)

benchmarking 100/lazy/fast
time                 1.072 μs   (1.067 μs .. 1.076 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.070 μs   (1.068 μs .. 1.073 μs)
std dev              8.967 ns   (7.523 ns .. 11.23 ns)

benchmarking 100/lazy/bstr
time                 9.631 μs   (9.621 μs .. 9.641 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 9.629 μs   (9.623 μs .. 9.637 μs)
std dev              22.91 ns   (17.17 ns .. 32.28 ns)

benchmarking 100/strict/fast
time                 996.5 ns   (995.1 ns .. 998.4 ns)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 996.2 ns   (995.6 ns .. 998.0 ns)
std dev              3.345 ns   (2.017 ns .. 5.791 ns)

benchmarking 100/io/fast
time                 1.572 μs   (1.569 μs .. 1.574 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.571 μs   (1.570 μs .. 1.573 μs)
std dev              4.480 ns   (3.588 ns .. 5.551 ns)

benchmarking 100/io/bstr
time                 9.832 μs   (9.825 μs .. 9.844 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 9.834 μs   (9.830 μs .. 9.842 μs)
std dev              18.98 ns   (10.99 ns .. 37.59 ns)

benchmarking 1000/lazy/fast
time                 7.485 μs   (7.482 μs .. 7.489 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 7.486 μs   (7.483 μs .. 7.491 μs)
std dev              12.69 ns   (8.605 ns .. 19.19 ns)

benchmarking 1000/lazy/bstr
time                 91.89 μs   (91.84 μs .. 91.95 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 91.78 μs   (91.72 μs .. 91.94 μs)
std dev              300.1 ns   (142.5 ns .. 604.3 ns)

benchmarking 1000/strict/fast
time                 7.462 μs   (7.456 μs .. 7.472 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 7.459 μs   (7.456 μs .. 7.464 μs)
std dev              12.27 ns   (8.676 ns .. 19.23 ns)

benchmarking 1000/io/fast
time                 8.353 μs   (8.341 μs .. 8.361 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 8.340 μs   (8.332 μs .. 8.351 μs)
std dev              31.41 ns   (23.18 ns .. 47.08 ns)

benchmarking 1000/io/bstr
time                 116.7 μs   (115.9 μs .. 117.6 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 117.0 μs   (116.6 μs .. 117.5 μs)
std dev              1.419 μs   (1.244 μs .. 1.591 μs)

benchmarking 10000/lazy/fast
time                 66.72 μs   (66.62 μs .. 66.80 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 66.62 μs   (66.56 μs .. 66.69 μs)
std dev              221.7 ns   (183.3 ns .. 282.3 ns)

benchmarking 10000/lazy/bstr
time                 953.1 μs   (951.4 μs .. 954.5 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 951.4 μs   (950.0 μs .. 952.7 μs)
std dev              4.620 μs   (3.387 μs .. 6.447 μs)

benchmarking 10000/strict/fast
time                 66.28 μs   (66.26 μs .. 66.30 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 66.28 μs   (66.27 μs .. 66.30 μs)
std dev              52.49 ns   (45.15 ns .. 65.26 ns)

benchmarking 10000/io/fast
time                 73.20 μs   (73.13 μs .. 73.27 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 73.16 μs   (73.14 μs .. 73.19 μs)
std dev              77.65 ns   (54.62 ns .. 118.6 ns)

benchmarking 10000/io/bstr
time                 1.394 ms   (1.391 ms .. 1.398 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.395 ms   (1.393 ms .. 1.398 ms)
std dev              9.151 μs   (7.170 μs .. 14.18 μs)

Benchmark vector: FINISH

[kozross]

Thank you for providing this! It would be good to add this to the README (along with the results).