Skip to content

Commit

Permalink
docs: updated README files for reproducing results and plotting bench…
Browse files Browse the repository at this point in the history
…marks
  • Loading branch information
nishaq503 committed Jan 18, 2025
1 parent b3dcb91 commit 5c97ace
Show file tree
Hide file tree
Showing 2 changed files with 50 additions and 7 deletions.
40 changes: 40 additions & 0 deletions benches/cakes/src/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Benchmarks for CAKES Search Algorithms

This is crate provides a CLI to run benchmarks for the CAKES search algorithms and reproduce the results from our paper.

## Reproducing the Results

Let's say you have data from the [ANN-Benchmarks suite](https://github.com/erikbern/ann-benchmarks?tab=readme-ov-file#data-sets) in a directory `../data/input` and you want to run the benchmarks for the CAKES search algorithms on the `sift` dataset.
You can run the following command:

```bash
cargo run --release --package bench-cakes -- \
--inp-dir ../data/input/ \
--dataset sift \
--out-dir ../data/output/ \
--seed 42 \
--num-queries 10000 \
--max-power 7 \
--max-time 300 \
--balanced-data \
--permuted-trees
```

This will run the CAKES search algorithms on the `sift` dataset with 10000 search queries.
The results will be saved in the directory `../data/output/`.
The dataset will be augmented by powers of 2 up to 2^7.
Each algorithm will be run for at least 300 seconds.
The `--balanced` flag will build trees with balanced partitions.
The `--permuted` flag will permute the dataset into depth-first order after building the tree.

There are several other available options.
Running the following command will provide documentation on how to use the CLI:

```bash
cargo run --release --package bench-cakes -- --help
```

## Plotting the Results

The outputs from the benchmarks can be plotted using the python package we provide at `../py-cakes`.
See the associated README for more information.
17 changes: 10 additions & 7 deletions benches/py-cakes/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# Benchmarks for the CAKES search algorithms
# Plotting results of the CAKES benchmarks

This package provides a CLI for plotting the results of the CAKES benchmarks.

You mush first run any benchmarks you want to plot using the `bench-cakes` crate we provide at `../cakes`.
See the associated README for more information.

## Usage

Expand All @@ -15,12 +20,10 @@ Install the package with the following command:
python -m pip install -e benches/py-cakes
```

Then, see the help message for the available commands:
Let's say you ran benchmarks for CAKES and saved results in a directory `../data/output`.
You now want to generate the plots and save them in a directory `../data/summary`.
You can do this with the following command:

```bash
python -m py_cakes --help
python -m py_cakes summarize-rust --inp-dir ../data/output --out-dir ../data/summary
```

## Citation

TODO...

0 comments on commit 5c97ace

Please sign in to comment.