Docs for Benchee.report/1

README.md

@@ -530,12 +530,13 @@ Enum."-map/2-lists^map/1-0-"/2                  10001 26.38 2282    0.23
 
 ### Saving, loading and comparing previous runs
 
-Benchee can store the results of previous runs in a file and then load them again to compare them. For example this is useful to compare what was recorded on the main branch against a branch with performance improvements.
+Benchee can store the results of previous runs in a file and then load them again to compare them. For example this is useful to compare what was recorded on the main branch against a branch with performance improvements. You may also use this to benchmark across different exlixir/erlang versions.
 
-**Saving** is done through the `save` configuration option. You can specify a `path` where results are saved, or you can use the default option of`"benchmark.benchee"` if you don't pass a `path`. You can also pass a `tag` option which annotates these results (for instance with a branch name). The default option for the `tag` is a timestamp of when the benchmark was run.
+**Saving** is done through the `:save` configuration option. You can specify a `:path` where results are saved, or you can use the default option of`"benchmark.benchee"` if you don't pass a `:path`. You can also pass a `:tag` option which annotates these results (for instance with a branch name or elixir version number). The default option for the `:tag` is a timestamp of when the benchmark was run.
 
-**Loading** is done through the `load` option specifying a path to the file to
-load (for instance `"benchmark.benchee"`). You can also specify multiple files to load through a list of paths (`["my.benchee", "main_save.benchee"]`) - each one of those can also be a glob expression to match even more files glob (`"save_number*.benchee"`).
+**Loading** is done through the `:load` option specifying a path to the file to load (for instance `"benchmark.benchee"`). You can also specify multiple files to load through a list of paths (`["my.benchee", "main_save.benchee"]`) - each one of those can also be a glob expression to match even more files glob (`"save_number*.benchee"`).
+If all you want to do is to use `:load` without running any benchmarks then you can use `Benchee.report/1` which will take a normal configuration with a `:load` key and just format the loaded saved results with the given formatters.
+In the more verbose API loading is triggered via `Benchee.load/1`.
 
 ```elixir
 Benchee.run(
@@ -545,11 +546,9 @@ Benchee.run(
   save: [path: "save.benchee", tag: "first-try"]
 )
 
-Benchee.run(%{}, load: "save.benchee")
+Benchee.report(load: "save.benchee")
 ```
 
-In the more verbose API this is triggered via `Benchee.load/1`.
-
 ### Hooks (Setup, Teardown etc.)
 
 Most of the time, it's best to keep your benchmarks as simple as possible: plain old immutable functions work best. But sometimes you need other things to happen. When you want to add before or after hooks to your benchmarks, we've got you covered! Before you dig into this section though remember one thing: **you usually don't need hooks!**

samples/save_and_report.exs

@@ -0,0 +1,59 @@
+list = Enum.to_list(1..10_000)
+map_fun = fn i -> [i, i * i] end
+
+Benchee.run(
+  %{
+    "flat_map" => fn -> Enum.flat_map(list, map_fun) end,
+    "map.flatten" => fn -> list |> Enum.map(map_fun) |> List.flatten() end
+  },
+  # report will give us the output
+  formatters: [],
+  time: 5,
+  save: [path: "save.benchee", tag: "save-me"]
+)
+
+IO.puts("----------------------------------------------")
+
+Benchee.report(
+  load: "save.benchee",
+  formatters: [{Benchee.Formatters.Console, extended_statistics: true}]
+)
+
+# Operating System: Linux
+# CPU Information: AMD Ryzen 9 5900X 12-Core Processor
+# Number of Available Cores: 24
+# Available memory: 31.25 GB
+# Elixir 1.16.0-rc.0
+# Erlang 26.1.2
+# JIT enabled: true
+
+# Benchmark suite executing with the following configuration:
+# warmup: 2 s
+# time: 5 s
+# memory time: 0 ns
+# reduction time: 0 ns
+# parallel: 1
+# inputs: none specified
+# Estimated total run time: 14 s
+
+# Benchmarking flat_map ...
+# Benchmarking map.flatten ...
+# Calculating statistics...
+# Formatting results...
+# Suite saved in external term format at save.benchee
+# ----------------------------------------------
+# Formatting results...
+
+# Name                            ips        average  deviation         median         99th %
+# flat_map (save-me)           3.69 K      270.69 μs    ±21.13%      259.71 μs      703.98 μs
+# map.flatten (save-me)        1.88 K      530.50 μs    ±45.74%      410.19 μs     1227.46 μs
+
+# Comparison:
+# flat_map (save-me)           3.69 K
+# map.flatten (save-me)        1.88 K - 1.96x slower +259.82 μs
+
+# Extended statistics:
+
+# Name                          minimum        maximum    sample size                     mode
+# flat_map (save-me)          215.30 μs     1066.77 μs        18.44 K                257.25 μs
+# map.flatten (save-me)       203.19 μs     1522.54 μs         9.41 K406.65 μs, 406.35 μs, 377