diff --git a/content/docs/start/experiments.md b/content/docs/start/experiments.md
index 84831ccc64..2bda91cc25 100644
--- a/content/docs/start/experiments.md
+++ b/content/docs/start/experiments.md
@@ -19,31 +19,82 @@ once they're no longer needed.
## Running experiments
In the previous page, we learned how to tune
-[ML pipelines](/doc/start/data-pipelines) and compare the changes. Let's further
-increase the number of features in the `featurize` stage to see how it compares.
+[ML pipelines](/doc/start/data-pipelines) and compare the changes. In this
+section we'll use a [new Get Started project][dvcgs] to illustrate
+experimentation features in DVC 2.0
`dvc exp run` makes it easy to change hyperparameters and run a new
-experiment:
+experiment. We'll use it to find parameters that results in better
+classification performance for [Fashion-MNIST][fmnist] dataset.
+
+[fmnist]: https://github.com/zalandoresearch/fashion-mnist
+
+> You can run these commands in the container we built for this tutorial. It has
+> all the code and data required to run these examples.
+> `docker run -it dvcorg/doc-start:experiments`
+
+In order to run a baseline experiment with the default parameters defined in
+`params.yaml`:
+
+```dvc
+dvc exp run
+```
+
+This resembles `dvc repro` without any command-line arguments. However, when
+using `dvc repro` we need to update `params.yaml` manually, run the pipeline, if
+the results are worth it commit them to DVC and Git. `dvc exp` automates this
+process through its subcommands.
+
+Let's see some metrics produced by this baseline experiment:
+
+```dvc
+$ dvc exp show --include-metrics categorical_accuracy,precision,recall \
+ --include-params model.name,model.cnn.conv_units
+┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
+┃ Experiment ┃ Created ┃ categorical_accuracy ┃ precision ┃ recall ┃ model.name ┃ model.cnn.conv_units ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
+│ workspace │ - │ 0.8498 │ 0.88219 │ 0.8147 │ cnn │ 16 │
+│ experiments │ Apr 26, 2021 │ - │ - │ - │ cnn │ 16 │
+│ └── 7a738cc [exp-c621e] │ 05:34 PM │ 0.8498 │ 0.88219 │ 0.8147 │ cnn │ 16 │
+└─────────────────────────┴──────────────┴──────────────────────┴───────────┴────────┴────────────┴──────────────────────┘
+```
+
+Note that the experiment results are identical with the values checked-out from
+Git. By default each experiment is given a name automatically. We can set the
+name by the `--name/-n` argument.
+
+Let's change the number of units in CNN and make another experiment:
```dvc
-$ dvc exp run --set-param featurize.max_features=3000
+$ dvc exp run -n cnn-32 --set-param model.cnn.conv_units=32
+'data/fashion-mnist/raw.dvc' didn't change, skipping
+Stage 'prepare' didn't change, skipping
+Stage 'preprocess' didn't change, skipping
+Running stage 'train':
+> python3 src/train.py
+...
```
+`--set-param/-S` argument of `dvc exp run` is used to set parameters in
+`params.yaml`. DVC runs _only_ the stages that depend on these parameter values.
+Parameter dependencies are defined via `dvc stage add` command and stored in
+`dvc.yaml`.
+
### 💡 Expand to see what this command does.
-`dvc exp run` is similar to `dvc repro` but with some added conveniences for
-running experiments. The `--set-param` (or `-S`) flag sets the values for
+The `--set-param` (or `-S`) flag sets the values for
[parameters](/doc/command-reference/params) as a shortcut to editing
`params.yaml`.
-Check that the `featurize.max_features` value has been updated in `params.yaml`:
+Check that the `model.cnn.units` value has been updated in `params.yaml`:
```git
- featurize:
-- max_features: 1500
-+ max_features: 3000
+ model:
+ cnn:
+- conv_units: 16
++ conv_units: 32
```
Any edits to dependencies (parameters or source code) will be
@@ -55,44 +106,56 @@ reflected in the experiment run.
```dvc
$ dvc exp diff
-Path Metric Value Change
-scores.json avg_prec 0.56191 0.009322
-scores.json roc_auc 0.93345 0.018087
-
-Path Param Value Change
-params.yaml featurize.max_features 3000 1500
+Path Metric Value Change
+metrics.json PR 0.92535 0.0048383
+metrics.json ROC 0.98807 0.0014254
+metrics.json categorical_accuracy 0.8693 0.0041
+metrics.json false_negatives 1532 -98
+metrics.json false_positives 990 -12
+metrics.json loss 0.36403 -0.01709
+metrics.json precision 0.89533 0.002241
+metrics.json recall 0.8468 0.0098
+metrics.json true_negatives 89010 12
+metrics.json true_positives 8468 98
+
+Path Param Value Change
+params.yaml model.cnn.conv_units 32 16
```
## Queueing experiments
-So far, we have been tuning the `featurize` stage, but there are also parameters
-for the `train` stage, which trains a
-[random forest classifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html).
-
-These are the `train` parameters in `params.yaml`:
-
-```yaml
-train:
- seed: 20170428
- n_est: 50
- min_split: 2
-```
+Instead of running the experiments one-by-one, we can define them without
+executing. This is especially handy when you have long running experiments to
+try.
-Let's setup experiments with different hyperparameters. We can define all the
-combinations we want to try without executing anything, by using the `--queue`
-flag:
+We add experiments to the queue using `--queue` option of `dvc exp run`. Here,
+we also set the names of experiments to observe the results clearly:
```dvc
-$ dvc exp run --queue -S train.min_split=8
-Queued experiment 'd3f6d1e' for future execution.
-$ dvc exp run --queue -S train.min_split=64
-Queued experiment 'f1810e0' for future execution.
-$ dvc exp run --queue -S train.min_split=2 -S train.n_est=100
-Queued experiment '7323ea2' for future execution.
-$ dvc exp run --queue -S train.min_split=8 -S train.n_est=100
-Queued experiment 'c605382' for future execution.
-$ dvc exp run --queue -S train.min_split=64 -S train.n_est=100
-Queued experiment '0cdee86' for future execution.
+$ dvc exp run --queue -n cnn-16-drop-0.1 -S model.cnn.conv_units=16 -S model.cnn.dropout=0.1
+Queued experiment 'c5fe01f' for future execution.
+$ dvc exp run --queue -n cnn-16-drop-0.5 -S model.cnn.conv_units=16 -S model.cnn.dropout=0.5
+Queued experiment 'e51a8a9' for future execution.
+$ dvc exp run --queue -n cnn-16-drop-0.9 -S model.cnn.conv_units=16 -S model.cnn.dropout=0.9
+Queued experiment 'b266b70' for future execution.
+$ dvc exp run --queue -n cnn-32-drop-0.1 -S model.cnn.conv_units=32 -S model.cnn.dropout=0.1
+Queued experiment 'ca07a75' for future execution.
+$ dvc exp run --queue -n cnn-32-drop-0.5 -S model.cnn.conv_units=32 -S model.cnn.dropout=0.5
+Queued experiment '747ca93' for future execution.
+$ dvc exp run --queue -n cnn-32-drop-0.9 -S model.cnn.conv_units=32 -S model.cnn.dropout=0.9
+Queued experiment 'a1fa8b1' for future execution.
+$ dvc exp run --queue -n cnn-64-drop-0.1 -S model.cnn.conv_units=64 -S model.cnn.dropout=0.1
+Queued experiment 'dd39ab6' for future execution.
+$ dvc exp run --queue -n cnn-64-drop-0.5 -S model.cnn.conv_units=64 -S model.cnn.dropout=0.5
+Queued experiment '2e4bcbe' for future execution.
+$ dvc exp run --queue -n cnn-64-drop-0.9 -S model.cnn.conv_units=64 -S model.cnn.dropout=0.9
+Queued experiment 'f807740' for future execution.
+$ dvc exp run --queue -n cnn-128-drop-0.1 -S model.cnn.conv_units=128 -S model.cnn.dropout=0.1
+Queued experiment '496a6b9' for future execution.
+$ dvc exp run --queue -n cnn-128-drop-0.5 -S model.cnn.conv_units=128 -S model.cnn.dropout=0.5
+Queued experiment '86307af' for future execution.
+$ dvc exp run --queue -n cnn-128-drop-0.9 -S model.cnn.conv_units=128 -S model.cnn.dropout=0.9
+Queued experiment 'bf224a9' for future execution.
```
Next, run all queued experiments using `--run-all` (and in parallel with
@@ -102,46 +165,56 @@ Next, run all queued experiments using `--run-all` (and in parallel with
$ dvc exp run --run-all --jobs 2
```
+The command will run all queued experiments in a random order.
+
## Comparing many experiments
To compare all of these experiments, we need more than `diff`. `dvc exp show`
compares any number of experiments in one table:
```dvc
-$ dvc exp show --no-timestamp
- --include-params train.n_est,train.min_split
-┏━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
-┃ Experiment ┃ avg_prec ┃ roc_auc ┃ train.n_est┃ train.min_split ┃
-┡━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
-│ workspace │ 0.56191 │ 0.93345 │ 50 │ 2 │
-│ master │ 0.55259 │ 0.91536 │ 50 │ 2 │
-│ ├── exp-bfe64 │ 0.57833 │ 0.95555 │ 50 │ 8 │
-│ ├── exp-b8082 │ 0.59806 │ 0.95287 │ 50 │ 64 │
-│ ├── exp-c7250 │ 0.58876 │ 0.94524 │ 100 │ 2 │
-│ ├── exp-b9cd4 │ 0.57953 │ 0.95732 │ 100 │ 8 │
-│ ├── exp-98a96 │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ └── exp-ad5b1 │ 0.56191 │ 0.93345 │ 50 │ 2 │
-└───────────────┴──────────┴─────────┴────────────┴─────────────────┘
+$ dvc exp show --no-timestamp \
+ --include-params model.cnn.conv_units,model.cnn.dropout \
+ --include-metrics categorical_accuracy,precision,recall
+┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
+┃ Experiment ┃ categorical_accuracy ┃ precision ┃ recall ┃ model.cnn.conv_units ┃ model.cnn.dropout ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
+│ workspace │ 0.8693 │ 0.89533 │ 0.8468 │ 32 │ 0.1 │
+│ experiments │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+│ ├── e51a8a9 [cnn-16-drop-0.5] │ 0.8556 │ 0.89122 │ 0.8242 │ 16 │ 0.5 │
+│ ├── c5fe01f [cnn-16-drop-0.1] │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+│ ├── b266b70 [cnn-16-drop-0.9] │ 0.8224 │ 0.88641 │ 0.7476 │ 16 │ 0.9 │
+│ ├── 747ca93 [cnn-32-drop-0.5] │ 0.8641 │ 0.8958 │ 0.8365 │ 32 │ 0.5 │
+│ ├── ca07a75 [cnn-32-drop-0.1] │ 0.8693 │ 0.89533 │ 0.8468 │ 32 │ 0.1 │
+│ ├── a1fa8b1 [cnn-32-drop-0.9] │ 0.8409 │ 0.89877 │ 0.7662 │ 32 │ 0.9 │
+│ ├── dd39ab6 [cnn-64-drop-0.1] │ 0.8785 │ 0.8999 │ 0.8594 │ 64 │ 0.1 │
+│ ├── 2e4bcbe [cnn-64-drop-0.5] │ 0.8759 │ 0.90428 │ 0.8493 │ 64 │ 0.5 │
+│ ├── f807740 [cnn-128-drop-0.1] │ 0.8787 │ 0.90401 │ 0.8523 │ 128 │ 0.1 │
+│ ├── 496a6b9 [cnn-64-drop-0.9] │ 0.8503 │ 0.896 │ 0.8081 │ 64 │ 0.9 │
+│ ├── 86307af [cnn-128-drop-0.9] │ 0.8556 │ 0.88936 │ 0.8199 │ 128 │ 0.9 │
+│ ├── bf224a9 [cnn-128-drop-0.5] │ 0.8796 │ 0.90273 │ 0.8575 │ 128 │ 0.5 │
+│ ├── cd59a21 [cnn-32] │ 0.8693 │ 0.89533 │ 0.8468 │ 32 │ 0.1 │
+│ └── b39380d [exp-44136] │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+└────────────────────────────────┴──────────────────────┴───────────┴────────┴──────────────────────┴───────────────────┘
```
-Each experiment is given an arbitrary name by default (although we can specify
-one with `dvc exp run -n`.) We can see that `exp-98a96` performed best among
-both of our metrics, with 100 estimators and a minimum of 64 samples to split a
-node.
+We can see that `cnn-128-drop-0.5` performed best in `categorical_accuracy`,
+`cnn-64-drop-0.5` in `precision` and `cnn-64-drop-0.1` in `recall`.
> See `dvc exp show --help` for more info on its options.
## Persisting experiments
-Now that we know the best parameters, let's keep that experiment and ignore the
-rest.
+The metrics to optimize are usually domain or problem dependent. In these
+experiments we are trying to optimize for `categorical_accuracy`. Let's keep
+experiment with the highest `categorical_accuracy` score and ignore the rest.
`dvc exp apply` rolls back the workspace to the specified
experiment:
```dvc
-$ dvc exp apply exp-98a96
-Changes for experiment 'exp-98a96' have been applied to your workspace.
+$ dvc exp apply cnn-128-drop-0.1
+Changes for experiment 'cnn-128-drop-0.1' have been applied to your workspace.
```
@@ -152,10 +225,22 @@ Changes for experiment 'exp-98a96' have been applied to your workspace.
tracks everything in the pipeline for each experiment (parameters, metrics,
dependencies, and outputs) and can later retrieve it as needed.
-Check that `scores.json` reflects the metrics in the table above:
+Check that `metrics.json` reflects all the metrics produced by the experiment
+now.
```json
-{ "avg_prec": 0.6040544652105823, "roc_auc": 0.9608017142900953 }
+{
+ "loss": 0.3368302583694458,
+ "categorical_accuracy": 0.8795999884605408,
+ "precision": 0.9027265906333923,
+ "recall": 0.8575000166893005,
+ "ROC": 0.9894749522209167,
+ "PR": 0.9345906376838684,
+ "true_positives": 8575.0,
+ "true_negatives": 89076.0,
+ "false_positives": 924.0,
+ "false_negatives": 1425.0
+}
```
@@ -165,8 +250,8 @@ reproducing the result without `dvc exp run`. Let's make it persistent in our
regular pipeline by committing it in our Git branch:
```dvc
-$ git add dvc.lock params.yaml prc.json roc.json scores.json
-$ git commit -a -m "Preserve best random forest experiment"
+$ git add dvc.lock params.yaml logs.csv metrics.json
+$ git commit -m "Preserve the experiment with 128 Conv Units and Dropout = 0.5"
```
## Sharing experiments
@@ -194,32 +279,29 @@ Storage, HTTP, HDFS, etc.). The Git remote is often a central Git server
-Experiments that have not been made persistent will not be stored or shared
-remotely through `dvc push` or `git push`.
-
`dvc exp push` enables storing and sharing any experiment remotely.
```dvc
-$ dvc exp push gitremote exp-bfe64
-Pushed experiment 'exp-bfe64' to Git remote 'gitremote'.
+$ dvc exp push gitremote cnn-128-drop-0.5
+Pushed experiment 'cnn-128-drop-0.5' to Git remote 'gitremote'.
```
`dvc exp list` shows all experiments that have been saved.
```dvc
$ dvc exp list gitremote --all
-72ed9cd:
- exp-bfe64
+experiments:
+ cnn-128-drop-0.5
```
`dvc exp pull` retrieves the experiment from a Git remote.
```dvc
-$ dvc exp pull gitremote exp-bfe64
-Pulled experiment 'exp-bfe64' from Git remote 'gitremote'.
+$ dvc exp pull gitremote cnn-128-drop-0.5
+Pulled experiment 'cnn-128-drop-0.5' from Git remote 'gitremote'.
```
-> All these commands take a Git remote as an argument. A default DVC remote is
+> All these commands take a _Git remote_ as an argument. A default DVC remote is
> also required to share the experiment data.
## Cleaning up
@@ -227,14 +309,15 @@ Pulled experiment 'exp-bfe64' from Git remote 'gitremote'.
Let's take another look at the experiments table:
```dvc
-$ dvc exp show --no-timestamp
- --include-params train.n_est,train.min_split
-┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
-┃ Experiment ┃ avg_prec ┃ roc_auc ┃ train.n_est┃ train.min_split ┃
-┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
-│ workspace │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ master │ 0.60405 │ 0.9608 │ 100 │ 64 │
-└────────────┴──────────┴─────────┴────────────┴─────────────────┘
+$ dvc exp show --no-timestamp \
+ --include-params model.cnn.conv_units,model.cnn.dropout \
+ --include-metrics categorical_accuracy,precision,recall
+┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
+┃ Experiment ┃ categorical_accuracy ┃ precision ┃ recall ┃ model.cnn.conv_units ┃ model.cnn.dropout ┃
+┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
+│ workspace │ 0.8796 │ 0.90273 │ 0.8575 │ 128 │ 0.5 │
+│ experiments │ 0.8796 │ 0.90273 │ 0.8575 │ 128 │ 0.5 │
+└─────────────┴──────────────────────┴───────────┴────────┴──────────────────────┴───────────────────┘
```
Where did all the experiments go? By default, `dvc exp show` only shows
@@ -243,39 +326,59 @@ experiments since the last commit, but don't worry. The experiments remain
experiments from the previous _n_ commits:
```dvc
-$ dvc exp show -n 2 --no-timestamp
- --include-params train.n_est,train.min_split
-┏━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
-┃ Experiment ┃ avg_prec ┃ roc_auc ┃ train.n_est┃ train.min_split ┃
-┡━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
-│ workspace │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ master │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ 64d74b2 │ 0.55259 │ 0.91536 │ 50 │ 2 │
-│ ├── exp-bfe64 │ 0.57833 │ 0.95555 │ 50 │ 8 │
-│ ├── exp-b8082 │ 0.59806 │ 0.95287 │ 50 │ 64 │
-│ ├── exp-c7250 │ 0.58876 │ 0.94524 │ 100 │ 2 │
-│ ├── exp-98a96 │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ ├── exp-b9cd4 │ 0.57953 │ 0.95732 │ 100 │ 8 │
-│ └── exp-ad5b1 │ 0.56191 │ 0.93345 │ 50 │ 2 │
-└───────────────┴──────────┴─────────┴────────────┴─────────────────┘
+$ dvc exp show -n 2 --no-timestamp \
+ --include-params model.cnn.conv_units,model.cnn.dropout \
+ --include-metrics categorical_accuracy,precision,recall
+┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
+┃ Experiment ┃ categorical_accuracy ┃ precision ┃ recall ┃ model.cnn.conv_units ┃ model.cnn.dropout ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
+│ workspace │ 0.8693 │ 0.89533 │ 0.8468 │ 32 │ 0.1 │
+│ experiments │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+│ ├── e51a8a9 [cnn-16-drop-0.5] │ 0.8556 │ 0.89122 │ 0.8242 │ 16 │ 0.5 │
+│ ├── c5fe01f [cnn-16-drop-0.1] │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+│ ├── b266b70 [cnn-16-drop-0.9] │ 0.8224 │ 0.88641 │ 0.7476 │ 16 │ 0.9 │
+│ ├── 747ca93 [cnn-32-drop-0.5] │ 0.8641 │ 0.8958 │ 0.8365 │ 32 │ 0.5 │
+│ ├── ca07a75 [cnn-32-drop-0.1] │ 0.8693 │ 0.89533 │ 0.8468 │ 32 │ 0.1 │
+│ ├── a1fa8b1 [cnn-32-drop-0.9] │ 0.8409 │ 0.89877 │ 0.7662 │ 32 │ 0.9 │
+│ ├── dd39ab6 [cnn-64-drop-0.1] │ 0.8785 │ 0.8999 │ 0.8594 │ 64 │ 0.1 │
+│ ├── 2e4bcbe [cnn-64-drop-0.5] │ 0.8759 │ 0.90428 │ 0.8493 │ 64 │ 0.5 │
+│ ├── f807740 [cnn-128-drop-0.1] │ 0.8787 │ 0.90401 │ 0.8523 │ 128 │ 0.1 │
+│ ├── 496a6b9 [cnn-64-drop-0.9] │ 0.8503 │ 0.896 │ 0.8081 │ 64 │ 0.9 │
+│ ├── 86307af [cnn-128-drop-0.9] │ 0.8556 │ 0.88936 │ 0.8199 │ 128 │ 0.9 │
+│ ├── bf224a9 [cnn-128-drop-0.5] │ 0.8796 │ 0.90273 │ 0.8575 │ 128 │ 0.5 │
+│ ├── cd59a21 [cnn-32] │ 0.8693 │ 0.89533 │ 0.8468 │ 32 │ 0.1 │
+│ └── b39380d [exp-44136] │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+└────────────────────────────────┴──────────────────────┴───────────┴────────┴──────────────────────┴───────────────────┘
```
-Eventually, old experiments may clutter the experiments table.
+After `dvc exp apply` the useful experiments and `dvc exp push` them to share or
+backup them, you may need to clean up the experiments.
`dvc exp gc` removes all references to old experiments:
```dvc
$ dvc exp gc --workspace
-$ dvc exp show -n 2 --no-timestamp
- --include-params train.n_est,train.min_split
-┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
-┃ Experiment ┃ avg_prec ┃ roc_auc ┃ train.n_est┃ train.min_split ┃
-┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
-│ workspace │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ master │ 0.60405 │ 0.9608 │ 100 │ 64 │
-│ 64d74b2 │ 0.55259 │ 0.91536 │ 50 │ 2 │
-└────────────┴──────────┴─────────┴────────────┴─────────────────┘
+...
+Removed 14 experiments. To remove unused cache files use 'dvc gc'.
+$ dvc exp show -n 2 --no-timestamp \
+ --include-params model.cnn.conv_units,model.cnn.dropout \
+ --include-metrics categorical_accuracy,precision,recall
+┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
+┃ Experiment ┃ categorical_accuracy ┃ precision ┃ recall ┃ model.cnn.conv_units ┃ model.cnn.dropout ┃
+┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
+│ workspace │ 0.8796 │ 0.90273 │ 0.8575 │ 128 │ 0.5 │
+│ experiments │ 0.8796 │ 0.90273 │ 0.8575 │ 128 │ 0.5 │
+│ a28bf0d │ 0.8652 │ 0.89309 │ 0.837 │ 16 │ 0.1 │
+└─────────────┴──────────────────────┴───────────┴────────┴──────────────────────┴───────────────────┘
```
> `dvc exp gc` only removes references to the experiments, not the cached
> objects associated to them. To clean up the cache, use `dvc gc`.
+
+## Going Further
+
+You can continue to experiment with [the project][gsexp], there are many
+parameters available to change, e.g., `noise` or `units` in Dense layer of the
+network.
+
+[gsexp]: https://github.com/iterative/get-started-experiments/