Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: remove optional crate #91

Merged
merged 5 commits into from
Nov 2, 2023
Merged

chore: remove optional crate #91

merged 5 commits into from
Nov 2, 2023

Conversation

RobWalt
Copy link
Contributor

@RobWalt RobWalt commented Oct 9, 2023 •
edited
Loading

This PR fixes some clippy warnings and removes every last bit of the optional crate

  • tests ran through successfully
  • here are the benchmarks after the change compared to master. It's so clear what the benchmarks indicate to me. I would say the performance is round about the same but I'm also biased since I want the optional crate to vanish from the dependencies. I'll leave the ultimate decision up to you.
Benchmarks 
   Compiling spade v2.2.0 (/home/aviac/repos/forks/spade)
    Finished bench [optimized] target(s) in 2.82s
     Running benches/benchmarks.rs (target/release/deps/benchmarks-f7c9b1b70afe2951)
Gnuplot not found, using plotters backend
locate benchmark (uniform)/locate (hierarchy<02>), f64
                        time:   [1.0776 µs 1.0781 µs 1.0786 µs]
                        change: [-0.8915% +0.1151% +0.8861%] (p = 0.81 > 0.05)
                        No change in performance detected.
Found 3 outliers among 50 measurements (6.00%)
  1 (2.00%) low mild
  1 (2.00%) high mild
  1 (2.00%) high severe
locate benchmark (uniform)/locate (hierarchy<03>), f64
                        time:   [791.11 ns 792.78 ns 795.33 ns]
                        change: [+0.8364% +1.4165% +1.8938%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 50 measurements (14.00%)
  2 (4.00%) high mild
  5 (10.00%) high severe
locate benchmark (uniform)/locate (hierarchy<04>), f64
                        time:   [708.76 ns 709.11 ns 709.51 ns]
                        change: [+0.1248% +0.5365% +0.9155%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 50 measurements (10.00%)
  2 (4.00%) high mild
  3 (6.00%) high severe
locate benchmark (uniform)/locate (hierarchy<05>), f64
                        time:   [663.35 ns 663.67 ns 664.10 ns]
                        change: [-2.4425% -2.3817% -2.3201%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 50 measurements (12.00%)
  2 (4.00%) high mild
  4 (8.00%) high severe
locate benchmark (uniform)/locate (hierarchy<08>), f64
                        time:   [610.67 ns 610.87 ns 611.10 ns]
                        change: [+0.0114% +0.3148% +0.6228%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 50 measurements (16.00%)
  2 (4.00%) low mild
  2 (4.00%) high mild
  4 (8.00%) high severe
locate benchmark (uniform)/locate (hierarchy<10>), f64
                        time:   [590.51 ns 591.46 ns 593.34 ns]
                        change: [-0.0429% +0.3038% +0.8236%] (p = 0.16 > 0.05)
                        No change in performance detected.
Found 5 outliers among 50 measurements (10.00%)
  1 (2.00%) high mild
  4 (8.00%) high severe
locate benchmark (uniform)/locate (hierarchy<14>), f64
                        time:   [600.20 ns 600.45 ns 600.72 ns]
                        change: [-1.4693% +1.0132% +2.5744%] (p = 0.45 > 0.05)
                        No change in performance detected.
Found 2 outliers among 50 measurements (4.00%)
  1 (2.00%) low mild
  1 (2.00%) high severe
locate benchmark (uniform)/locate (hierarchy<16>), f64
                        time:   [606.88 ns 607.07 ns 607.28 ns]
                        change: [+3.7095% +3.8302% +3.9300%] (p = 0.00 < 0.05)
                        Performance has regressed.
locate benchmark (uniform)/locate (hierarchy<18>), f64
                        time:   [564.86 ns 565.04 ns 565.23 ns]
                        change: [-5.7784% -5.4022% -4.9870%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 50 measurements (10.00%)
  1 (2.00%) low mild
  1 (2.00%) high mild
  3 (6.00%) high severe
locate benchmark (uniform)/locate (hierarchy<20>), f64
                        time:   [581.77 ns 581.90 ns 582.03 ns]
                        change: [-0.0216% +0.2327% +0.4526%] (p = 0.06 > 0.05)
                        No change in performance detected.
Found 4 outliers among 50 measurements (8.00%)
  1 (2.00%) low mild
  1 (2.00%) high mild
  2 (4.00%) high severe
locate benchmark (uniform)/locate (hierarchy<32>), f64
                        time:   [590.47 ns 593.23 ns 596.06 ns]
                        change: [-0.5171% +0.1379% +0.7671%] (p = 0.68 > 0.05)
                        No change in performance detected.
Found 8 outliers among 50 measurements (16.00%)
  6 (12.00%) low severe
  1 (2.00%) low mild
  1 (2.00%) high severe
locate benchmark (uniform)/locate (hierarchy<40>), f64
                        time:   [591.95 ns 592.32 ns 592.70 ns]
                        change: [-1.6252% -1.4870% -1.3275%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 50 measurements (8.00%)
  2 (4.00%) high mild
  2 (4.00%) high severe
locate benchmark (uniform)/locate (hierarchy<50>), f64
                        time:   [605.57 ns 606.11 ns 606.73 ns]
                        change: [-1.8608% -1.2462% -0.6681%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 50 measurements (10.00%)
  2 (4.00%) high mild
  3 (6.00%) high severe
locate benchmark (uniform)/locate (hierarchy<64>), f64
                        time:   [635.14 ns 635.73 ns 636.47 ns]
                        change: [-0.2880% -0.1351% +0.0098%] (p = 0.11 > 0.05)
                        No change in performance detected.
Found 8 outliers among 50 measurements (16.00%)
  2 (4.00%) low mild
  2 (4.00%) high mild
  4 (8.00%) high severe
locate benchmark (uniform)/locate (hierarchy<70>), f64
                        time:   [664.54 ns 664.88 ns 665.24 ns]
                        change: [+3.1813% +3.3283% +3.4434%] (p = 0.00 < 0.05)
                        Performance has regressed.
locate benchmark (uniform)/locate (hierarchy<80>), f64
                        time:   [670.57 ns 671.06 ns 671.66 ns]
                        change: [+0.9546% +1.1440% +1.3604%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 50 measurements (10.00%)
  5 (10.00%) high severe
locate benchmark (uniform)/locate (hierarchy<90>), f64
                        time:   [674.94 ns 675.36 ns 676.01 ns]
                        change: [-0.1205% -0.0126% +0.1144%] (p = 0.87 > 0.05)
                        No change in performance detected.
Found 2 outliers among 50 measurements (4.00%)
  1 (2.00%) high mild
  1 (2.00%) high severe

bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/1000
                        time:   [140.97 µs 140.99 µs 141.02 µs]
                        thrpt:  [7.0911 Melem/s 7.0928 Melem/s 7.0937 Melem/s]
                 change:
                        time:   [+0.2605% +0.3249% +0.4056%] (p = 0.00 < 0.05)
                        thrpt:  [-0.4039% -0.3239% -0.2598%]
                        Change within noise threshold.
Found 3 outliers among 50 measurements (6.00%)
  2 (4.00%) high mild
  1 (2.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/2000
                        time:   [287.01 µs 287.12 µs 287.23 µs]
                        thrpt:  [6.9631 Melem/s 6.9658 Melem/s 6.9684 Melem/s]
                 change:
                        time:   [-0.2557% -0.1042% +0.0546%] (p = 0.19 > 0.05)
                        thrpt:  [-0.0546% +0.1043% +0.2564%]
                        No change in performance detected.
Found 4 outliers among 50 measurements (8.00%)
  4 (8.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/3000
                        time:   [576.95 µs 578.04 µs 579.32 µs]
                        thrpt:  [5.1785 Melem/s 5.1899 Melem/s 5.1998 Melem/s]
                 change:
                        time:   [-0.4743% -0.2884% -0.1210%] (p = 0.00 < 0.05)
                        thrpt:  [+0.1211% +0.2892% +0.4765%]
                        Change within noise threshold.
Found 1 outliers among 50 measurements (2.00%)
  1 (2.00%) low mild
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/6000
                        time:   [1.3679 ms 1.3683 ms 1.3687 ms]
                        thrpt:  [4.3836 Melem/s 4.3850 Melem/s 4.3863 Melem/s]
                 change:
                        time:   [-0.4117% -0.3323% -0.2175%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2179% +0.3334% +0.4134%]
                        Change within noise threshold.
Found 5 outliers among 50 measurements (10.00%)
  5 (10.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/10000
                        time:   [2.2982 ms 2.2996 ms 2.3015 ms]
                        thrpt:  [4.3450 Melem/s 4.3485 Melem/s 4.3512 Melem/s]
                 change:
                        time:   [+1.4075% +1.5046% +1.5947%] (p = 0.00 < 0.05)
                        thrpt:  [-1.5697% -1.4823% -1.3879%]
                        Performance has regressed.
Found 4 outliers among 50 measurements (8.00%)
  2 (4.00%) low mild
  2 (4.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/15000
                        time:   [3.5055 ms 3.5065 ms 3.5077 ms]
                        thrpt:  [4.2764 Melem/s 4.2777 Melem/s 4.2791 Melem/s]
                 change:
                        time:   [+0.8815% +0.9371% +0.9993%] (p = 0.00 < 0.05)
                        thrpt:  [-0.9894% -0.9284% -0.8738%]
                        Change within noise threshold.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/1000
                        time:   [134.55 µs 134.59 µs 134.63 µs]
                        thrpt:  [7.4278 Melem/s 7.4301 Melem/s 7.4324 Melem/s]
                 change:
                        time:   [+0.1873% +0.2597% +0.3441%] (p = 0.00 < 0.05)
                        thrpt:  [-0.3429% -0.2590% -0.1869%]
                        Change within noise threshold.
Found 4 outliers among 50 measurements (8.00%)
  1 (2.00%) high mild
  3 (6.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/2000
                        time:   [280.66 µs 281.67 µs 282.91 µs]
                        thrpt:  [7.0694 Melem/s 7.1005 Melem/s 7.1260 Melem/s]
                 change:
                        time:   [+0.3664% +0.5597% +0.8336%] (p = 0.00 < 0.05)
                        thrpt:  [-0.8268% -0.5566% -0.3651%]
                        Change within noise threshold.
Found 6 outliers among 50 measurements (12.00%)
  1 (2.00%) high mild
  5 (10.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/3000
                        time:   [543.73 µs 544.22 µs 544.72 µs]
                        thrpt:  [5.5074 Melem/s 5.5125 Melem/s 5.5174 Melem/s]
                 change:
                        time:   [+1.0986% +1.2598% +1.4319%] (p = 0.00 < 0.05)
                        thrpt:  [-1.4117% -1.2442% -1.0867%]
                        Performance has regressed.
Found 4 outliers among 50 measurements (8.00%)
  2 (4.00%) low mild
  2 (4.00%) high mild
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/6000
                        time:   [1.2628 ms 1.2634 ms 1.2646 ms]
                        thrpt:  [4.7447 Melem/s 4.7490 Melem/s 4.7515 Melem/s]
                 change:
                        time:   [-0.5495% -0.4546% -0.2983%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2992% +0.4567% +0.5525%]
                        Change within noise threshold.
Found 2 outliers among 50 measurements (4.00%)
  1 (2.00%) high mild
  1 (2.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/10000
                        time:   [2.1906 ms 2.1910 ms 2.1915 ms]
                        thrpt:  [4.5632 Melem/s 4.5641 Melem/s 4.5649 Melem/s]
                 change:
                        time:   [+1.6857% +1.8024% +1.8957%] (p = 0.00 < 0.05)
                        thrpt:  [-1.8605% -1.7705% -1.6578%]
                        Performance has regressed.
Found 4 outliers among 50 measurements (8.00%)
  4 (8.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/15000
                        time:   [3.3630 ms 3.3636 ms 3.3643 ms]
                        thrpt:  [4.4586 Melem/s 4.4595 Melem/s 4.4603 Melem/s]
                 change:
                        time:   [+0.6781% +0.8368% +0.9687%] (p = 0.00 < 0.05)
                        thrpt:  [-0.9594% -0.8299% -0.6736%]
                        Change within noise threshold.
Found 6 outliers among 50 measurements (12.00%)
  1 (2.00%) low mild
  2 (4.00%) high mild
  3 (6.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/1000
                        time:   [147.07 µs 147.21 µs 147.52 µs]
                        thrpt:  [6.7788 Melem/s 6.7930 Melem/s 6.7994 Melem/s]
                 change:
                        time:   [+0.6544% +0.7312% +0.8456%] (p = 0.00 < 0.05)
                        thrpt:  [-0.8385% -0.7259% -0.6501%]
                        Change within noise threshold.
Found 3 outliers among 50 measurements (6.00%)
  3 (6.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/2000
                        time:   [304.63 µs 304.79 µs 305.05 µs]
                        thrpt:  [6.5563 Melem/s 6.5620 Melem/s 6.5653 Melem/s]
                 change:
                        time:   [+0.6993% +0.7611% +0.8346%] (p = 0.00 < 0.05)
                        thrpt:  [-0.8277% -0.7553% -0.6944%]
                        Change within noise threshold.
Found 5 outliers among 50 measurements (10.00%)
  2 (4.00%) high mild
  3 (6.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/3000
                        time:   [615.11 µs 615.55 µs 615.90 µs]
                        thrpt:  [4.8709 Melem/s 4.8737 Melem/s 4.8771 Melem/s]
                 change:
                        time:   [+0.6762% +0.7818% +0.8891%] (p = 0.00 < 0.05)
                        thrpt:  [-0.8812% -0.7757% -0.6716%]
                        Change within noise threshold.
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/6000
                        time:   [1.4419 ms 1.4424 ms 1.4429 ms]
                        thrpt:  [4.1583 Melem/s 4.1598 Melem/s 4.1613 Melem/s]
                 change:
                        time:   [+0.5666% +0.6197% +0.6655%] (p = 0.00 < 0.05)
                        thrpt:  [-0.6611% -0.6158% -0.5634%]
                        Change within noise threshold.
Found 1 outliers among 50 measurements (2.00%)
  1 (2.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/10000
                        time:   [2.5228 ms 2.5234 ms 2.5241 ms]
                        thrpt:  [3.9619 Melem/s 3.9629 Melem/s 3.9638 Melem/s]
                 change:
                        time:   [+0.8568% +0.9344% +1.0103%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0002% -0.9257% -0.8495%]
                        Change within noise threshold.
Found 3 outliers among 50 measurements (6.00%)
  3 (6.00%) high severe
Benchmarking bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/15000: Warming up for 1.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 5.0s, enable flat sampling, or reduce sample count to 30.
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/15000
                        time:   [3.9075 ms 3.9131 ms 3.9239 ms]
                        thrpt:  [3.8227 Melem/s 3.8333 Melem/s 3.8388 Melem/s]
                 change:
                        time:   [+1.1813% +1.5475% +2.0039%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9646% -1.5239% -1.1675%]
                        Performance has regressed.
Found 7 outliers among 50 measurements (14.00%)
  2 (4.00%) high mild
  5 (10.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/1000
                        time:   [149.05 µs 149.10 µs 149.15 µs]
                        thrpt:  [6.7046 Melem/s 6.7068 Melem/s 6.7091 Melem/s]
                 change:
                        time:   [-0.4690% -0.3785% -0.2514%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2520% +0.3799% +0.4712%]
                        Change within noise threshold.
Found 3 outliers among 50 measurements (6.00%)
  2 (4.00%) high mild
  1 (2.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/2000
                        time:   [309.88 µs 310.24 µs 310.80 µs]
                        thrpt:  [6.4350 Melem/s 6.4466 Melem/s 6.4540 Melem/s]
                 change:
                        time:   [-0.4185% -0.1821% +0.0763%] (p = 0.16 > 0.05)
                        thrpt:  [-0.0762% +0.1825% +0.4202%]
                        No change in performance detected.
Found 2 outliers among 50 measurements (4.00%)
  1 (2.00%) high mild
  1 (2.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/3000
                        time:   [640.36 µs 640.86 µs 641.39 µs]
                        thrpt:  [4.6773 Melem/s 4.6812 Melem/s 4.6849 Melem/s]
                 change:
                        time:   [+0.3142% +0.4697% +0.6091%] (p = 0.00 < 0.05)
                        thrpt:  [-0.6054% -0.4676% -0.3132%]
                        Change within noise threshold.
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/6000
                        time:   [1.5451 ms 1.5457 ms 1.5463 ms]
                        thrpt:  [3.8802 Melem/s 3.8818 Melem/s 3.8831 Melem/s]
                 change:
                        time:   [+0.7766% +0.9968% +1.2190%] (p = 0.00 < 0.05)
                        thrpt:  [-1.2043% -0.9869% -0.7706%]
                        Change within noise threshold.
Found 11 outliers among 50 measurements (22.00%)
  11 (22.00%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/10000
                        time:   [2.6331 ms 2.6335 ms 2.6338 ms]
                        thrpt:  [3.7968 Melem/s 3.7973 Melem/s 3.7978 Melem/s]
                 change:
                        time:   [+3.1900% +3.2295% +3.2687%] (p = 0.00 < 0.05)
                        thrpt:  [-3.1652% -3.1284% -3.0914%]
                        Performance has regressed.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high severe
Benchmarking bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/15000: Warming up for 1.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 30.
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/15000
                        time:   [3.9714 ms 3.9723 ms 3.9736 ms]
                        thrpt:  [3.7749 Melem/s 3.7761 Melem/s 3.7770 Melem/s]
                 change:
                        time:   [+1.2799% +1.3360% +1.3925%] (p = 0.00 < 0.05)
                        thrpt:  [-1.3734% -1.3184% -1.2637%]
                        Performance has regressed.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high mild

bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/100000
                        time:   [23.775 ms 23.786 ms 23.800 ms]
                        thrpt:  [4.2017 Melem/s 4.2041 Melem/s 4.2060 Melem/s]
                 change:
                        time:   [+1.0830% +1.1389% +1.1971%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1829% -1.1261% -1.0713%]
                        Performance has regressed.
Found 3 outliers among 50 measurements (6.00%)
  1 (2.00%) high mild
  2 (4.00%) high severe
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/200000
                        time:   [48.734 ms 48.750 ms 48.767 ms]
                        thrpt:  [4.1011 Melem/s 4.1026 Melem/s 4.1040 Melem/s]
                 change:
                        time:   [-0.1612% -0.1201% -0.0796%] (p = 0.00 < 0.05)
                        thrpt:  [+0.0797% +0.1203% +0.1614%]
                        Change within noise threshold.
Benchmarking bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/400000: Warming up for 1.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 5.6s, or reduce sample count to 40.
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/400000
                        time:   [112.63 ms 112.73 ms 112.84 ms]
                        thrpt:  [3.5449 Melem/s 3.5483 Melem/s 3.5516 Melem/s]
                 change:
                        time:   [+1.7805% +1.9116% +2.0385%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9978% -1.8758% -1.7494%]
                        Performance has regressed.
Benchmarking bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/800000: Warming up for 1.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 12.2s, or reduce sample count to 20.
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/800000
                        time:   [244.63 ms 245.04 ms 245.45 ms]
                        thrpt:  [3.2593 Melem/s 3.2647 Melem/s 3.2702 Melem/s]
                 change:
                        time:   [+5.0667% +5.3584% +5.6280%] (p = 0.00 < 0.05)
                        thrpt:  [-5.3281% -5.0859% -4.8223%]
                        Performance has regressed.

bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/1000
                        time:   [149.80 µs 149.82 µs 149.85 µs]
                        thrpt:  [6.6735 Melem/s 6.6746 Melem/s 6.6756 Melem/s]
                 change:
                        time:   [+2.6306% +2.7348% +2.8325%] (p = 0.00 < 0.05)
                        thrpt:  [-2.7545% -2.6620% -2.5632%]
                        Performance has regressed.
Found 6 outliers among 50 measurements (12.00%)
  6 (12.00%) high mild
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/2000
                        time:   [305.24 µs 305.68 µs 306.38 µs]
                        thrpt:  [6.5277 Melem/s 6.5429 Melem/s 6.5522 Melem/s]
                 change:
                        time:   [+0.7643% +0.9598% +1.1429%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1300% -0.9507% -0.7585%]
                        Change within noise threshold.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high severe
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/3000
                        time:   [626.51 µs 626.89 µs 627.26 µs]
                        thrpt:  [4.7827 Melem/s 4.7855 Melem/s 4.7885 Melem/s]
                 change:
                        time:   [-0.7665% +0.0799% +0.9129%] (p = 0.85 > 0.05)
                        thrpt:  [-0.9046% -0.0799% +0.7725%]
                        No change in performance detected.
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/6000
                        time:   [1.4657 ms 1.4660 ms 1.4665 ms]
                        thrpt:  [4.0915 Melem/s 4.0927 Melem/s 4.0937 Melem/s]
                 change:
                        time:   [-0.6563% +0.9208% +1.9710%] (p = 0.20 > 0.05)
                        thrpt:  [-1.9329% -0.9124% +0.6607%]
                        No change in performance detected.
Found 4 outliers among 50 measurements (8.00%)
  4 (8.00%) high mild
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/10000
                        time:   [2.5419 ms 2.5426 ms 2.5433 ms]
                        thrpt:  [3.9319 Melem/s 3.9330 Melem/s 3.9340 Melem/s]
                 change:
                        time:   [+0.4481% +0.5999% +0.7180%] (p = 0.00 < 0.05)
                        thrpt:  [-0.7129% -0.5963% -0.4461%]
                        Change within noise threshold.
Found 3 outliers among 50 measurements (6.00%)
  1 (2.00%) high mild
  2 (4.00%) high severe
Benchmarking bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/15000: Warming up for 1.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 30.
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/15000
                        time:   [3.9256 ms 3.9319 ms 3.9406 ms]
                        thrpt:  [3.8065 Melem/s 3.8150 Melem/s 3.8211 Melem/s]
                 change:
                        time:   [+1.2419% +1.4583% +1.6823%] (p = 0.00 < 0.05)
                        thrpt:  [-1.6545% -1.4373% -1.2267%]
                        Performance has regressed.
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/1000
                        time:   [219.13 µs 219.45 µs 219.88 µs]
                        thrpt:  [4.5480 Melem/s 4.5568 Melem/s 4.5635 Melem/s]
                 change:
                        time:   [-3.5594% -3.3019% -3.1040%] (p = 0.00 < 0.05)
                        thrpt:  [+3.2034% +3.4147% +3.6908%]
                        Performance has improved.
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/2000
                        time:   [716.33 µs 716.57 µs 716.81 µs]
                        thrpt:  [2.7901 Melem/s 2.7911 Melem/s 2.7920 Melem/s]
                 change:
                        time:   [+6.4502% +6.6478% +6.8383%] (p = 0.00 < 0.05)
                        thrpt:  [-6.4006% -6.2334% -6.0594%]
                        Performance has regressed.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) low mild
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/3000
                        time:   [1.2828 ms 1.2831 ms 1.2834 ms]
                        thrpt:  [2.3375 Melem/s 2.3380 Melem/s 2.3386 Melem/s]
                 change:
                        time:   [+2.4230% +2.4612% +2.5005%] (p = 0.00 < 0.05)
                        thrpt:  [-2.4395% -2.4021% -2.3657%]
                        Performance has regressed.
Found 6 outliers among 50 measurements (12.00%)
  1 (2.00%) low severe
  2 (4.00%) low mild
  3 (6.00%) high mild
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/6000
                        time:   [2.8289 ms 2.8291 ms 2.8293 ms]
                        thrpt:  [2.1206 Melem/s 2.1208 Melem/s 2.1210 Melem/s]
                 change:
                        time:   [+0.6152% +0.7204% +0.8197%] (p = 0.00 < 0.05)
                        thrpt:  [-0.8130% -0.7153% -0.6114%]
                        Change within noise threshold.
Found 3 outliers among 50 measurements (6.00%)
  1 (2.00%) low mild
  2 (4.00%) high severe
Benchmarking bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/10000: Warming up for 1.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 6.5s, enable flat sampling, or reduce sample count to 30.
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/10000
                        time:   [5.0637 ms 5.0646 ms 5.0654 ms]
                        thrpt:  [1.9742 Melem/s 1.9745 Melem/s 1.9748 Melem/s]
                 change:
                        time:   [+0.8493% +0.9221% +0.9932%] (p = 0.00 < 0.05)
                        thrpt:  [-0.9835% -0.9137% -0.8422%]
                        Change within noise threshold.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high mild
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/15000
                        time:   [8.0719 ms 8.0782 ms 8.0854 ms]
                        thrpt:  [1.8552 Melem/s 1.8568 Melem/s 1.8583 Melem/s]
                 change:
                        time:   [+0.0807% +0.1634% +0.2605%] (p = 0.00 < 0.05)
                        thrpt:  [-0.2598% -0.1632% -0.0807%]
                        Change within noise threshold.
Found 5 outliers among 50 measurements (10.00%)
  3 (6.00%) high mild
  2 (4.00%) high severe

Feel free to just close the PR if you don't want the change.

Related to #89

@RobWalt RobWalt changed the title Chore/remove optional crate chore: remove optional crate Oct 9, 2023
@RobWalt
Copy link
Contributor Author

RobWalt commented Oct 9, 2023

I ran another set of benchmarks but with a sample size of 1000 and warm up time of 5 seconds

Benchmarks 
Compiling spade v2.2.0 (/home/aviac/repos/forks/spade)
    Finished bench [optimized] target(s) in 2.86s
     Running benches/benchmarks.rs (target/release/deps/benchmarks-f7c9b1b70afe2951)
Gnuplot not found, using plotters backend
locate benchmark (uniform)/locate (hierarchy<02>), f64
                        time:   [1.0853 µs 1.0882 µs 1.0913 µs]
                        change: [-8.0256% -7.5028% -6.9932%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 91 outliers among 1000 measurements (9.10%)
  1 (0.10%) low mild
  59 (5.90%) high mild
  31 (3.10%) high severe
locate benchmark (uniform)/locate (hierarchy<03>), f64
                        time:   [803.27 ns 805.54 ns 808.63 ns]
                        change: [+0.0358% +0.1793% +0.3684%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 87 outliers among 1000 measurements (8.70%)
  5 (0.50%) low severe
  29 (2.90%) low mild
  26 (2.60%) high mild
  27 (2.70%) high severe
locate benchmark (uniform)/locate (hierarchy<04>), f64
                        time:   [709.19 ns 711.05 ns 713.16 ns]
                        change: [+1.4442% +1.7525% +2.0695%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 19 outliers among 1000 measurements (1.90%)
  12 (1.20%) high mild
  7 (0.70%) high severe
locate benchmark (uniform)/locate (hierarchy<05>), f64
                        time:   [669.68 ns 671.59 ns 673.78 ns]
                        change: [+0.0153% +0.3867% +0.7239%] (p = 0.03 < 0.05)
                        Change within noise threshold.
Found 221 outliers among 1000 measurements (22.10%)
  167 (16.70%) low mild
  36 (3.60%) high mild
  18 (1.80%) high severe
locate benchmark (uniform)/locate (hierarchy<08>), f64
                        time:   [607.10 ns 607.66 ns 608.25 ns]
                        change: [-0.2253% -0.0854% +0.0427%] (p = 0.20 > 0.05)
                        No change in performance detected.
Found 65 outliers among 1000 measurements (6.50%)
  2 (0.20%) low mild
  53 (5.30%) high mild
  10 (1.00%) high severe
locate benchmark (uniform)/locate (hierarchy<10>), f64
                        time:   [605.38 ns 605.90 ns 606.45 ns]
                        change: [-0.2843% -0.0954% +0.1164%] (p = 0.37 > 0.05)
                        No change in performance detected.
Found 86 outliers among 1000 measurements (8.60%)
  3 (0.30%) low mild
  46 (4.60%) high mild
  37 (3.70%) high severe
locate benchmark (uniform)/locate (hierarchy<14>), f64
                        time:   [584.29 ns 585.50 ns 586.97 ns]
                        change: [-0.5920% -0.4653% -0.3385%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 42 outliers among 1000 measurements (4.20%)
  31 (3.10%) high mild
  11 (1.10%) high severe
locate benchmark (uniform)/locate (hierarchy<16>), f64
                        time:   [597.92 ns 599.34 ns 600.69 ns]
                        change: [-0.7428% -0.5387% -0.3469%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 1000 measurements (0.10%)
  1 (0.10%) high mild
locate benchmark (uniform)/locate (hierarchy<18>), f64
                        time:   [589.52 ns 589.70 ns 589.87 ns]
                        change: [-1.5479% -1.4592% -1.3725%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 43 outliers among 1000 measurements (4.30%)
  5 (0.50%) low mild
  28 (2.80%) high mild
  10 (1.00%) high severe
locate benchmark (uniform)/locate (hierarchy<20>), f64
                        time:   [579.23 ns 579.72 ns 580.31 ns]
                        change: [-2.9742% -2.3631% -1.8623%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 29 outliers among 1000 measurements (2.90%)
  2 (0.20%) low mild
  17 (1.70%) high mild
  10 (1.00%) high severe
locate benchmark (uniform)/locate (hierarchy<32>), f64
                        time:   [592.42 ns 592.60 ns 592.78 ns]
                        change: [-1.9456% -1.8635% -1.7853%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 43 outliers among 1000 measurements (4.30%)
  4 (0.40%) low severe
  11 (1.10%) low mild
  15 (1.50%) high mild
  13 (1.30%) high severe
locate benchmark (uniform)/locate (hierarchy<40>), f64
                        time:   [609.11 ns 609.30 ns 609.49 ns]
                        change: [-2.0177% -1.5364% -1.0927%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 48 outliers among 1000 measurements (4.80%)
  3 (0.30%) low severe
  12 (1.20%) low mild
  22 (2.20%) high mild
  11 (1.10%) high severe
locate benchmark (uniform)/locate (hierarchy<50>), f64
                        time:   [620.09 ns 620.36 ns 620.64 ns]
                        change: [-0.0641% +0.1053% +0.2620%] (p = 0.21 > 0.05)
                        No change in performance detected.
Found 40 outliers among 1000 measurements (4.00%)
  3 (0.30%) low severe
  5 (0.50%) low mild
  24 (2.40%) high mild
  8 (0.80%) high severe
locate benchmark (uniform)/locate (hierarchy<64>), f64
                        time:   [657.87 ns 658.25 ns 658.67 ns]
                        change: [+2.1785% +2.3865% +2.5742%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 69 outliers among 1000 measurements (6.90%)
  1 (0.10%) low severe
  11 (1.10%) low mild
  33 (3.30%) high mild
  24 (2.40%) high severe
locate benchmark (uniform)/locate (hierarchy<70>), f64
                        time:   [661.95 ns 663.55 ns 665.42 ns]
                        change: [-0.8400% -0.3191% +0.1724%] (p = 0.22 > 0.05)
                        No change in performance detected.
Found 10 outliers among 1000 measurements (1.00%)
  5 (0.50%) high mild
  5 (0.50%) high severe
locate benchmark (uniform)/locate (hierarchy<80>), f64
                        time:   [661.54 ns 661.90 ns 662.25 ns]
                        change: [-0.8056% -0.7064% -0.6116%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 40 outliers among 1000 measurements (4.00%)
  6 (0.60%) low mild
  26 (2.60%) high mild
  8 (0.80%) high severe
locate benchmark (uniform)/locate (hierarchy<90>), f64
                        time:   [684.42 ns 685.30 ns 686.24 ns]
                        change: [+0.2961% +0.5403% +0.7887%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 44 outliers among 1000 measurements (4.40%)
  5 (0.50%) low severe
  24 (2.40%) low mild
  5 (0.50%) high mild
  10 (1.00%) high severe

bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/1000
                        time:   [142.14 µs 142.17 µs 142.21 µs]
                        thrpt:  [7.0321 Melem/s 7.0337 Melem/s 7.0353 Melem/s]
                 change:
                        time:   [+1.0075% +1.0320% +1.0596%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0485% -1.0215% -0.9975%]
                        Performance has regressed.
Found 5 outliers among 1000 measurements (0.50%)
  4 (0.40%) high mild
  1 (0.10%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/2000
                        time:   [288.98 µs 289.04 µs 289.11 µs]
                        thrpt:  [6.9179 Melem/s 6.9194 Melem/s 6.9209 Melem/s]
                 change:
                        time:   [+1.0755% +1.1498% +1.2118%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1973% -1.1367% -1.0640%]
                        Performance has regressed.
Found 13 outliers among 1000 measurements (1.30%)
  13 (1.30%) high mild
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/3000
                        time:   [581.95 µs 582.04 µs 582.12 µs]
                        thrpt:  [5.1536 Melem/s 5.1543 Melem/s 5.1550 Melem/s]
                 change:
                        time:   [+3.2310% +3.2668% +3.3010%] (p = 0.00 < 0.05)
                        thrpt:  [-3.1955% -3.1635% -3.1299%]
                        Performance has regressed.
Found 40 outliers among 1000 measurements (4.00%)
  36 (3.60%) high mild
  4 (0.40%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/6000
                        time:   [1.3806 ms 1.3807 ms 1.3808 ms]
                        thrpt:  [4.3452 Melem/s 4.3456 Melem/s 4.3460 Melem/s]
                 change:
                        time:   [-0.6479% -0.2068% +0.2137%] (p = 0.35 > 0.05)
                        thrpt:  [-0.2132% +0.2072% +0.6521%]
                        No change in performance detected.
Found 6 outliers among 1000 measurements (0.60%)
  1 (0.10%) low mild
  4 (0.40%) high mild
  1 (0.10%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/10000
                        time:   [2.3021 ms 2.3023 ms 2.3026 ms]
                        thrpt:  [4.3429 Melem/s 4.3434 Melem/s 4.3438 Melem/s]
                 change:
                        time:   [+1.8249% +1.8927% +1.9481%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9108% -1.8575% -1.7922%]
                        Performance has regressed.
Found 35 outliers among 1000 measurements (3.50%)
  17 (1.70%) low mild
  12 (1.20%) high mild
  6 (0.60%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, locally clustered/15000
                        time:   [3.5719 ms 3.5723 ms 3.5728 ms]
                        thrpt:  [4.1984 Melem/s 4.1990 Melem/s 4.1994 Melem/s]
                 change:
                        time:   [+3.0774% +3.1456% +3.2038%] (p = 0.00 < 0.05)
                        thrpt:  [-3.1043% -3.0497% -2.9855%]
                        Performance has regressed.
Found 25 outliers among 1000 measurements (2.50%)
  15 (1.50%) high mild
  10 (1.00%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/1000
                        time:   [135.41 µs 135.49 µs 135.58 µs]
                        thrpt:  [7.3758 Melem/s 7.3808 Melem/s 7.3853 Melem/s]
                 change:
                        time:   [+1.0144% +1.0812% +1.1560%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1428% -1.0697% -1.0042%]
                        Performance has regressed.
Found 9 outliers among 1000 measurements (0.90%)
  4 (0.40%) high mild
  5 (0.50%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/2000
                        time:   [281.22 µs 281.49 µs 281.82 µs]
                        thrpt:  [7.0967 Melem/s 7.1051 Melem/s 7.1118 Melem/s]
                 change:
                        time:   [-0.5382% -0.4102% -0.2693%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2700% +0.4119% +0.5411%]
                        Change within noise threshold.
Found 13 outliers among 1000 measurements (1.30%)
  5 (0.50%) high mild
  8 (0.80%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/3000
                        time:   [538.95 µs 539.05 µs 539.16 µs]
                        thrpt:  [5.5643 Melem/s 5.5653 Melem/s 5.5664 Melem/s]
                 change:
                        time:   [-0.4095% -0.3875% -0.3642%] (p = 0.00 < 0.05)
                        thrpt:  [+0.3656% +0.3890% +0.4112%]
                        Change within noise threshold.
Found 16 outliers among 1000 measurements (1.60%)
  14 (1.40%) high mild
  2 (0.20%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/6000
                        time:   [1.2618 ms 1.2627 ms 1.2636 ms]
                        thrpt:  [4.7483 Melem/s 4.7519 Melem/s 4.7551 Melem/s]
                 change:
                        time:   [-0.3868% -0.3220% -0.2484%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2490% +0.3231% +0.3883%]
                        Change within noise threshold.
Found 100 outliers among 1000 measurements (10.00%)
  65 (6.50%) low mild
  22 (2.20%) high mild
  13 (1.30%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/10000
                        time:   [2.1856 ms 2.1870 ms 2.1885 ms]
                        thrpt:  [4.5693 Melem/s 4.5725 Melem/s 4.5753 Melem/s]
                 change:
                        time:   [+0.9397% +1.0352% +1.1291%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1165% -1.0246% -0.9310%]
                        Change within noise threshold.
Found 12 outliers among 1000 measurements (1.20%)
  6 (0.60%) high mild
  6 (0.60%) high severe
bulk load benchmark (small)/bulk loading on last used vertex heuristic, uniformly distributed/15000
                        time:   [3.3778 ms 3.3815 ms 3.3856 ms]
                        thrpt:  [4.4305 Melem/s 4.4359 Melem/s 4.4407 Melem/s]
                 change:
                        time:   [+0.8692% +0.9817% +1.1230%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1105% -0.9722% -0.8617%]
                        Change within noise threshold.
Found 18 outliers among 1000 measurements (1.80%)
  2 (0.20%) high mild
  16 (1.60%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/1000
                        time:   [148.19 µs 148.69 µs 149.25 µs]
                        thrpt:  [6.7003 Melem/s 6.7254 Melem/s 6.7483 Melem/s]
                 change:
                        time:   [+0.3412% +0.6682% +1.0473%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0364% -0.6638% -0.3400%]
                        Change within noise threshold.
Found 36 outliers among 1000 measurements (3.60%)
  1 (0.10%) high mild
  35 (3.50%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/2000
                        time:   [366.18 µs 366.29 µs 366.40 µs]
                        thrpt:  [5.4585 Melem/s 5.4601 Melem/s 5.4618 Melem/s]
                 change:
                        time:   [+22.263% +22.331% +22.397%] (p = 0.00 < 0.05)
                        thrpt:  [-18.299% -18.254% -18.209%]
                        Performance has regressed.
Found 41 outliers among 1000 measurements (4.10%)
  3 (0.30%) low mild
  32 (3.20%) high mild
  6 (0.60%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/3000
                        time:   [613.15 µs 613.50 µs 613.87 µs]
                        thrpt:  [4.8870 Melem/s 4.8900 Melem/s 4.8928 Melem/s]
                 change:
                        time:   [+1.7121% +1.7913% +1.8733%] (p = 0.00 < 0.05)
                        thrpt:  [-1.8389% -1.7598% -1.6833%]
                        Performance has regressed.
Found 21 outliers among 1000 measurements (2.10%)
  10 (1.00%) low mild
  5 (0.50%) high mild
  6 (0.60%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/6000
                        time:   [1.4404 ms 1.4406 ms 1.4409 ms]
                        thrpt:  [4.1641 Melem/s 4.1648 Melem/s 4.1655 Melem/s]
                 change:
                        time:   [+1.5040% +1.6010% +1.6864%] (p = 0.00 < 0.05)
                        thrpt:  [-1.6584% -1.5758% -1.4817%]
                        Performance has regressed.
Found 5 outliers among 1000 measurements (0.50%)
  3 (0.30%) high mild
  2 (0.20%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/10000
                        time:   [2.5118 ms 2.5125 ms 2.5133 ms]
                        thrpt:  [3.9788 Melem/s 3.9800 Melem/s 3.9813 Melem/s]
                 change:
                        time:   [+0.7223% +0.7572% +0.7932%] (p = 0.00 < 0.05)
                        thrpt:  [-0.7869% -0.7515% -0.7171%]
                        Change within noise threshold.
Found 7 outliers among 1000 measurements (0.70%)
  7 (0.70%) high mild
bulk load benchmark (small)/bulk loading on hierarchy lookup, uniformly distributed/15000
                        time:   [3.8925 ms 3.8933 ms 3.8942 ms]
                        thrpt:  [3.8518 Melem/s 3.8527 Melem/s 3.8536 Melem/s]
                 change:
                        time:   [+0.4194% +0.4990% +0.5617%] (p = 0.00 < 0.05)
                        thrpt:  [-0.5585% -0.4965% -0.4176%]
                        Change within noise threshold.
Found 4 outliers among 1000 measurements (0.40%)
  4 (0.40%) high mild
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/1000
                        time:   [150.47 µs 150.51 µs 150.56 µs]
                        thrpt:  [6.6421 Melem/s 6.6441 Melem/s 6.6458 Melem/s]
                 change:
                        time:   [-1.3743% -1.0877% -0.8435%] (p = 0.00 < 0.05)
                        thrpt:  [+0.8507% +1.0997% +1.3934%]
                        Change within noise threshold.
Found 24 outliers among 1000 measurements (2.40%)
  19 (1.90%) low mild
  2 (0.20%) high mild
  3 (0.30%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/2000
                        time:   [319.50 µs 319.61 µs 319.75 µs]
                        thrpt:  [6.2548 Melem/s 6.2576 Melem/s 6.2598 Melem/s]
                 change:
                        time:   [+2.9741% +3.0180% +3.0680%] (p = 0.00 < 0.05)
                        thrpt:  [-2.9767% -2.9296% -2.8882%]
                        Performance has regressed.
Found 16 outliers among 1000 measurements (1.60%)
  4 (0.40%) high mild
  12 (1.20%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/3000
                        time:   [637.82 µs 638.47 µs 639.20 µs]
                        thrpt:  [4.6934 Melem/s 4.6988 Melem/s 4.7036 Melem/s]
                 change:
                        time:   [+0.2495% +0.3470% +0.4673%] (p = 0.00 < 0.05)
                        thrpt:  [-0.4652% -0.3458% -0.2489%]
                        Change within noise threshold.
Found 28 outliers among 1000 measurements (2.80%)
  14 (1.40%) high mild
  14 (1.40%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/6000
                        time:   [1.5548 ms 1.5552 ms 1.5555 ms]
                        thrpt:  [3.8572 Melem/s 3.8581 Melem/s 3.8590 Melem/s]
                 change:
                        time:   [+1.5458% +1.7040% +1.8422%] (p = 0.00 < 0.05)
                        thrpt:  [-1.8089% -1.6754% -1.5223%]
                        Performance has regressed.
Found 42 outliers among 1000 measurements (4.20%)
  1 (0.10%) low mild
  39 (3.90%) high mild
  2 (0.20%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/10000
                        time:   [2.5757 ms 2.5760 ms 2.5763 ms]
                        thrpt:  [3.8815 Melem/s 3.8820 Melem/s 3.8825 Melem/s]
                 change:
                        time:   [+1.7513% +1.7822% +1.8127%] (p = 0.00 < 0.05)
                        thrpt:  [-1.7804% -1.7510% -1.7212%]
                        Performance has regressed.
Found 38 outliers among 1000 measurements (3.80%)
  30 (3.00%) low mild
  6 (0.60%) high mild
  2 (0.20%) high severe
bulk load benchmark (small)/bulk loading on hierarchy lookup, locally clustered/15000
                        time:   [3.9479 ms 3.9495 ms 3.9516 ms]
                        thrpt:  [3.7959 Melem/s 3.7980 Melem/s 3.7995 Melem/s]
                 change:
                        time:   [+1.1950% +1.2771% +1.3568%] (p = 0.00 < 0.05)
                        thrpt:  [-1.3386% -1.2610% -1.1809%]
                        Performance has regressed.
Found 26 outliers among 1000 measurements (2.60%)
  6 (0.60%) low severe
  14 (1.40%) low mild
  2 (0.20%) high mild
  4 (0.40%) high severe

Benchmarking bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/100000: Warming up for 5.0000 s
Warning: Unable to complete 1000 samples in 5.0s. You may wish to increase target time to 24.4s, or reduce sample count to 200.
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/100000
                        time:   [24.333 ms 24.334 ms 24.336 ms]
                        thrpt:  [4.1091 Melem/s 4.1094 Melem/s 4.1097 Melem/s]
                 change:
                        time:   [+2.0719% +2.1608% +2.2279%] (p = 0.00 < 0.05)
                        thrpt:  [-2.1794% -2.1151% -2.0298%]
                        Performance has regressed.
Found 8 outliers among 1000 measurements (0.80%)
  2 (0.20%) low mild
  3 (0.30%) high mild
  3 (0.30%) high severe
Benchmarking bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/200000: Warming up for 5.0000 s
Warning: Unable to complete 1000 samples in 5.0s. You may wish to increase target time to 48.8s, or reduce sample count to 100.
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/200000
                        time:   [48.595 ms 48.677 ms 48.766 ms]
                        thrpt:  [4.1012 Melem/s 4.1087 Melem/s 4.1157 Melem/s]
                 change:
                        time:   [+0.5542% +0.8009% +1.0487%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0379% -0.7946% -0.5512%]
                        Change within noise threshold.
Found 88 outliers among 1000 measurements (8.80%)
  1 (0.10%) low mild
  9 (0.90%) high mild
  78 (7.80%) high severe
Benchmarking bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/400000: Warming up for 5.0000 s
Warning: Unable to complete 1000 samples in 5.0s. You may wish to increase target time to 112.3s, or reduce sample count to 40.
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/400000
                        time:   [110.58 ms 110.71 ms 110.83 ms]
                        thrpt:  [3.6091 Melem/s 3.6132 Melem/s 3.6172 Melem/s]
                 change:
                        time:   [+3.0420% +3.1697% +3.3006%] (p = 0.00 < 0.05)
                        thrpt:  [-3.1951% -3.0723% -2.9522%]
                        Performance has regressed.
Found 9 outliers among 1000 measurements (0.90%)
  9 (0.90%) high mild
Benchmarking bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/800000: Warming up for 5.0000 s
Warning: Unable to complete 1000 samples in 5.0s. You may wish to increase target time to 239.5s, or reduce sample count to 20.
bulk load benchmark (big)/bulk loading on last used vertex heuristic, uniformly distributed/800000
                        time:   [239.19 ms 239.28 ms 239.38 ms]
                        thrpt:  [3.3419 Melem/s 3.3433 Melem/s 3.3446 Melem/s]
                 change:
                        time:   [+1.7946% +1.8580% +1.9276%] (p = 0.00 < 0.05)
                        thrpt:  [-1.8912% -1.8241% -1.7630%]
                        Performance has regressed.
Found 57 outliers among 1000 measurements (5.70%)
  7 (0.70%) low mild
  19 (1.90%) high mild
  31 (3.10%) high severe

bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/1000
                        time:   [145.60 µs 145.67 µs 145.74 µs]
                        thrpt:  [6.8613 Melem/s 6.8648 Melem/s 6.8680 Melem/s]
                 change:
                        time:   [-0.8811% -0.8178% -0.7540%] (p = 0.00 < 0.05)
                        thrpt:  [+0.7598% +0.8246% +0.8890%]
                        Change within noise threshold.
Found 7 outliers among 1000 measurements (0.70%)
  3 (0.30%) high mild
  4 (0.40%) high severe
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/2000
                        time:   [303.11 µs 303.38 µs 303.72 µs]
                        thrpt:  [6.5850 Melem/s 6.5924 Melem/s 6.5982 Melem/s]
                 change:
                        time:   [-0.0066% +0.0845% +0.1970%] (p = 0.10 > 0.05)
                        thrpt:  [-0.1966% -0.0845% +0.0066%]
                        No change in performance detected.
Found 15 outliers among 1000 measurements (1.50%)
  2 (0.20%) high mild
  13 (1.30%) high severe
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/3000
                        time:   [608.27 µs 608.44 µs 608.60 µs]
                        thrpt:  [4.9293 Melem/s 4.9307 Melem/s 4.9320 Melem/s]
                 change:
                        time:   [-0.9841% -0.9525% -0.9204%] (p = 0.00 < 0.05)
                        thrpt:  [+0.9290% +0.9617% +0.9938%]
                        Change within noise threshold.
Found 10 outliers among 1000 measurements (1.00%)
  10 (1.00%) high mild
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/6000
                        time:   [1.4394 ms 1.4402 ms 1.4411 ms]
                        thrpt:  [4.1635 Melem/s 4.1660 Melem/s 4.1683 Melem/s]
                 change:
                        time:   [+2.3970% +2.4666% +2.5467%] (p = 0.00 < 0.05)
                        thrpt:  [-2.4835% -2.4073% -2.3409%]
                        Performance has regressed.
Found 7 outliers among 1000 measurements (0.70%)
  4 (0.40%) low mild
  3 (0.30%) high severe
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/10000
                        time:   [2.5589 ms 2.5592 ms 2.5598 ms]
                        thrpt:  [3.9065 Melem/s 3.9074 Melem/s 3.9080 Melem/s]
                 change:
                        time:   [+2.6531% +2.7279% +2.7907%] (p = 0.00 < 0.05)
                        thrpt:  [-2.7149% -2.6555% -2.5846%]
                        Performance has regressed.
Found 35 outliers among 1000 measurements (3.50%)
  1 (0.10%) low severe
  15 (1.50%) low mild
  15 (1.50%) high mild
  4 (0.40%) high severe
bulk vs incremental loading/bulk loading on hierarchy lookup, uniformly distributed/15000
                        time:   [4.0072 ms 4.0095 ms 4.0120 ms]
                        thrpt:  [3.7388 Melem/s 3.7411 Melem/s 3.7433 Melem/s]
                 change:
                        time:   [+3.6352% +3.7570% +3.8717%] (p = 0.00 < 0.05)
                        thrpt:  [-3.7274% -3.6209% -3.5077%]
                        Performance has regressed.
Found 88 outliers among 1000 measurements (8.80%)
  5 (0.50%) high mild
  83 (8.30%) high severe
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/1000
                        time:   [220.85 µs 220.87 µs 220.88 µs]
                        thrpt:  [4.5273 Melem/s 4.5276 Melem/s 4.5279 Melem/s]
                 change:
                        time:   [-4.8989% -4.8011% -4.7104%] (p = 0.00 < 0.05)
                        thrpt:  [+4.9432% +5.0433% +5.1513%]
                        Performance has improved.
Found 54 outliers among 1000 measurements (5.40%)
  35 (3.50%) high mild
  19 (1.90%) high severe
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/2000
                        time:   [705.30 µs 705.79 µs 706.31 µs]
                        thrpt:  [2.8316 Melem/s 2.8337 Melem/s 2.8357 Melem/s]
                 change:
                        time:   [+2.1815% +2.6657% +3.1154%] (p = 0.00 < 0.05)
                        thrpt:  [-3.0213% -2.5964% -2.1349%]
                        Performance has regressed.
Found 28 outliers among 1000 measurements (2.80%)
  16 (1.60%) high mild
  12 (1.20%) high severe
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/3000
                        time:   [1.2615 ms 1.2621 ms 1.2627 ms]
                        thrpt:  [2.3758 Melem/s 2.3770 Melem/s 2.3781 Melem/s]
                 change:
                        time:   [+1.4957% +1.5513% +1.6061%] (p = 0.00 < 0.05)
                        thrpt:  [-1.5807% -1.5276% -1.4737%]
                        Performance has regressed.
Found 11 outliers among 1000 measurements (1.10%)
  8 (0.80%) high mild
  3 (0.30%) high severe
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/6000
                        time:   [2.8215 ms 2.8223 ms 2.8232 ms]
                        thrpt:  [2.1253 Melem/s 2.1259 Melem/s 2.1265 Melem/s]
                 change:
                        time:   [+1.3212% +1.4146% +1.5003%] (p = 0.00 < 0.05)
                        thrpt:  [-1.4781% -1.3948% -1.3039%]
                        Performance has regressed.
Found 2 outliers among 1000 measurements (0.20%)
  2 (0.20%) high mild
Benchmarking bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/10000: Warming up for 5.0000 s
Warning: Unable to complete 1000 samples in 5.0s. You may wish to increase target time to 5.1s, or reduce sample count to 980.
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/10000
                        time:   [5.0489 ms 5.0505 ms 5.0520 ms]
                        thrpt:  [1.9794 Melem/s 1.9800 Melem/s 1.9806 Melem/s]
                 change:
                        time:   [+0.9294% +0.9647% +1.0006%] (p = 0.00 < 0.05)
                        thrpt:  [-0.9907% -0.9555% -0.9209%]
                        Change within noise threshold.
Found 23 outliers among 1000 measurements (2.30%)
  22 (2.20%) high mild
  1 (0.10%) high severe
Benchmarking bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/15000: Warming up for 5.0000 s
Warning: Unable to complete 1000 samples in 5.0s. You may wish to increase target time to 8.3s, or reduce sample count to 600.
bulk vs incremental loading/incremental loading on hierarchy lookup, uniformly distributed/15000
                        time:   [8.2734 ms 8.2740 ms 8.2747 ms]
                        thrpt:  [1.8128 Melem/s 1.8129 Melem/s 1.8130 Melem/s]
                 change:
                        time:   [+2.9500% +2.9681% +2.9853%] (p = 0.00 < 0.05)
                        thrpt:  [-2.8988% -2.8825% -2.8655%]
                        Performance has regressed.
Found 6 outliers among 1000 measurements (0.60%)
  6 (0.60%) high mild

michaelkirk
michaelkirk approved these changes Nov 2, 2023
View reviewed changes
Copy link
Contributor

@michaelkirk michaelkirk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a user, not a maintainer, of this crate, so my review means nothing, but LGTM.

Cargo.toml Show resolved Hide resolved
src/lib.rs Outdated Show resolved Hide resolved
@Stoeoef
Copy link
Owner

Stoeoef commented Nov 2, 2023

This PR fixes some clippy warnings and removes every last bit of the optional crate

* tests ran through successfully

* here are the benchmarks after the change compared to master. It's so clear what the benchmarks indicate to me. I would say the performance is round about the same but I'm also biased since I want the `optional` crate to vanish from the dependencies. I'll leave the ultimate decision up to you.

Benchmarks

Feel free to just close the PR if you don't want the change.

Related to #89

I've rerun the benchmarks locally and could also not see a clear difference.
I recall that introducing Optional definitely did make a difference, but maybe something has changed in the meantime and makes this obsolete. Let's get rid of optional for now.

@Stoeoef
Copy link
Owner

Stoeoef commented Nov 2, 2023 •
edited
Loading

Thanks a lot for the contribution (and sorry for my late reply :( )!
All looks good to me, let's remove Optional for now.

Edit: Clippy seems to be acting up since the last commit (though the fix is easy)

@Stoeoef Stoeoef mentioned this pull request Nov 2, 2023
Merged
@Stoeoef Stoeoef merged commit e3bc71d into Stoeoef:master Nov 2, 2023
5 checks passed
@RobWalt RobWalt deleted the chore/remove-optional-crate branch November 2, 2023 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Stoeoef Stoeoef Stoeoef left review comments

@michaelkirk michaelkirk michaelkirk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@RobWalt @Stoeoef @michaelkirk