Skip to content

Latest commit

 

History

History
192 lines (152 loc) · 6.9 KB

benchmarks.md

File metadata and controls

192 lines (152 loc) · 6.9 KB

Benchmarks

Running

  • Quick run: mix run bench/bench.exs
  • Full run: mix run bench/bench.exs full

Benchmarking Summary

Perf-wise, the sweet spot for PersistentVector is scenarios when vector needs to be built by repeatedly appending to the end AND random-access (get/set operations) are also used.

If random-access after building is not required, then building and reversing a List is more efficient.

If building speed is not important, but removal from the end happens often, then Erlang's :array shows better performance.

get/set operations perform similar to :array:

  • PersistentVector.get/2 is slightly faster for larger collections compared to :array.
  • PersistentVector.set/3 is slightly faster for smaller collections compared to :array.

Map is added only for a baseline. In a sense that if Map was to outperform PersistentVector then this library would not be needed. This comparison is not fair to Map as it has much richer capabilities. The fact that Map performs worse for bigger collections is not surprising and is not Map's fault :-).

Raw Benchmarking results for v0.1.4

Using 2 inputs

#
# Build
#

##### With input     1'000 #####
Name                   ips        average  deviation         median
List   Build       41.43 K       24.13 μs    ±32.24%       31.00 μs
Vector Build        9.25 K      108.15 μs    ±66.78%      150.00 μs
Map    Build        4.60 K      217.59 μs    ±35.13%      160.00 μs
Array  Build        4.42 K      226.07 μs    ±34.40%      160.00 μs

Comparison:
List   Build       41.43 K
Vector Build        9.25 K - 4.48x slower
Map    Build        4.60 K - 9.02x slower
Array  Build        4.42 K - 9.37x slower

##### With input 1'000'000 #####
Name                   ips        average  deviation         median
List   Build         14.06       71.15 ms    ±10.85%       78.00 ms
Vector Build          5.77      173.22 ms     ±5.30%      172.00 ms
Array  Build          1.66      602.94 ms     ±1.61%      609.00 ms
Map    Build          0.99     1007.80 ms     ±1.97%     1007.50 ms

Comparison:
List   Build         14.06
Vector Build          5.77 - 2.43x slower
Array  Build          1.66 - 8.47x slower
Map    Build          0.99 - 14.16x slower

#
# Shrink
#

##### With input     1'000 #####
Name                         ips        average  deviation         median
Array  resize            13.96 K       71.65 μs    ±11.36%       78.00 μs
Vector remove_last        6.91 K      144.74 μs    ±36.61%      160.00 μs
Map    resize             4.05 K      246.67 μs    ±31.47%      310.00 μs

Comparison:
Array  resize            13.96 K
Vector remove_last        6.91 K - 2.02x slower
Map    resize             4.05 K - 3.44x slower

##### With input 1'000'000 #####
Name                         ips        average  deviation         median
Array  resize              13.10       76.34 ms     ±7.94%       78.00 ms
Vector remove_last          6.14      162.81 ms     ±5.01%      157.00 ms
Map    resize               1.05      950.27 ms     ±3.57%      937.00 ms

Comparison:
Array  resize              13.10
Vector remove_last          6.14 - 2.13x slower
Map    resize               1.05 - 12.45x slower

#
# Get
#

##### With input     1'000 #####
Name                 ips        average  deviation         median
Map    Get       15.58 K       64.20 μs   ±119.83%         0.0 μs
Vector Get        7.68 K      130.25 μs    ±44.82%      160.00 μs
Array  Get        7.37 K      135.70 μs    ±39.04%      160.00 μs

Comparison:
Map    Get       15.58 K
Vector Get        7.68 K - 2.03x slower
Array  Get        7.37 K - 2.11x slower

##### With input 1'000'000 #####
Name                 ips        average  deviation         median
Vector Get          5.85      171.08 ms     ±2.05%      172.00 ms
Array  Get          4.14      241.81 ms     ±3.23%      235.00 ms
Map    Get          3.14      318.84 ms     ±6.58%      312.00 ms

Comparison:
Vector Get          5.85
Array  Get          4.14 - 1.41x slower
Map    Get          3.14 - 1.86x slower

#
# Set
#

##### With input     1'000 #####
Name                 ips        average  deviation         median
Map    Set        4.58 K      218.36 μs    ±35.04%      160.00 μs
Vector Set        4.36 K      229.54 μs    ±33.97%      160.00 μs
Array  Set        3.35 K      298.54 μs    ±15.06%      310.00 μs

Comparison:
Map    Set        4.58 K
Vector Set        4.36 K - 1.05x slower
Array  Set        3.35 K - 1.37x slower

##### With input 1'000'000 #####
Name                 ips        average  deviation         median
Array  Set          1.41      708.33 ms     ±3.22%      703.00 ms
Vector Set          1.18      849.00 ms     ±2.85%      844.00 ms
Map    Set          0.80     1253.88 ms     ±3.92%     1234.00 ms

Comparison:
Array  Set          1.41
Vector Set          1.18 - 1.20x slower
Map    Set          0.80 - 1.77x slower

#
# Enumerate
#

##### With input     1'000 #####
Name                       ips        average  deviation         median
Vector Enumerate       17.19 K       58.17 μs    ±12.08%       62.00 μs
Map    Enumerate       14.01 K       71.36 μs    ±10.89%       78.00 μs

Comparison:
Vector Enumerate       17.19 K
Map    Enumerate       14.01 K - 1.23x slower

##### With input 1'000'000 #####
Name                       ips        average  deviation         median
Vector Enumerate         15.60       64.10 ms     ±7.92%       63.00 ms
Map    Enumerate          8.07      123.84 ms    ±15.47%      125.00 ms

Comparison:
Vector Enumerate         15.60
Map    Enumerate          8.07 - 1.93x slower

#
# to_list
#

##### With input     1'000 #####
Name                     ips        average  deviation         median
Vector to_list      190.39 K        5.25 μs    ±14.27%        4.70 μs
Array  to_list       67.39 K       14.84 μs    ±23.25%       16.00 μs
Map    to_list       49.62 K       20.15 μs    ±35.27%       16.00 μs
Vector into          17.55 K       56.99 μs    ±13.19%       62.00 μs
Map    into          13.66 K       73.23 μs     ±9.91%       78.00 μs

Comparison:
Vector to_list      190.39 K
Array  to_list       67.39 K - 2.83x slower
Map    to_list       49.62 K - 3.84x slower
Vector into          17.55 K - 10.85x slower
Map    into          13.66 K - 13.94x slower

##### With input 1'000'000 #####
Name                     ips        average  deviation         median
Array  to_list         53.20       18.80 ms    ±36.19%       16.00 ms
Vector to_list         49.58       20.17 ms    ±38.37%       16.00 ms
Map    to_list         26.54       37.68 ms    ±55.39%       31.00 ms
Vector into            10.98       91.05 ms    ±15.50%       93.00 ms
Map    into             8.29      120.67 ms     ±7.02%      125.00 ms

Comparison:
Array  to_list         53.20
Vector to_list         49.58 - 1.07x slower
Map    to_list         26.54 - 2.00x slower
Vector into            10.98 - 4.84x slower
Map    into             8.29 - 6.42x slower