diff --git a/content/docs/command-reference/exp/apply.md b/content/docs/command-reference/exp/apply.md index 9167047bd0..db7ef7f482 100644 --- a/content/docs/command-reference/exp/apply.md +++ b/content/docs/command-reference/exp/apply.md @@ -38,3 +38,88 @@ the current Git commit. - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Example: Make an experiment persistent + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have run 3 experiments in our project: + +```dvc +$ dvc exp show --include-params=featurize +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-e6c97 │ Oct 21, 2020 │ 0.69830 │ 2000 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 1200 │ 2 │ +│ └── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +Since `exp-e6c97` has the best `auc`, we may want to commit it into our project +(this is what we call to "make it persistent"): + +```dvc +$ dvc exp apply exp-e6c97 +Changes for experiment 'exp-e6c97' have been applied... +``` + +We can inspect what changed in the workspace with Git, + +```dvc +$ git status +On branch master +Changes not staged for commit: + modified: dvc.lock + modified: params.yaml + modified: scores.json +$ git diff params.yaml +``` + +```git +@@ -3,7 +3,7 @@ prepare: + featurize: +- max_features: 1500 ++ max_features: 2000 + ngrams: 2 +``` + +and with DVC: + +```dvc +$ dvc status +Data and pipelines are up to date. +$ dvc diff +Modified: + data/features/ + data/features/test.pkl + data/features/train.pkl + model.pkl +files summary: 0 added, 0 deleted, 3 modified, 0 not in cache +``` + +To finish making this experiment persistent, we commit the changes to the repo: + +```dvc +$ git add . +$ git commit -m "persist exp-e6c97" +``` + +We can now see that the experiment is the new tip of our master branch: + +```dvc +$ dvc exp show --include-params=featurize +┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.69830 │ 2000 │ 2 │ +│ master │ 04:31 PM │ 0.69830 │ 2000 │ 2 │ +└────────────┴──────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +Note that all the other experiments are based on a previous commit, so +`dvc exp show` won't display them by default (but they're still saved). diff --git a/content/docs/command-reference/exp/branch.md b/content/docs/command-reference/exp/branch.md index 8af16b5e64..a2649181aa 100644 --- a/content/docs/command-reference/exp/branch.md +++ b/content/docs/command-reference/exp/branch.md @@ -48,3 +48,45 @@ To switch into the new branch, use `git checkout branch` and `dvc checkout`. - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Example: Make a persistent branch from an experiment + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have run 3 experiments in our project: + +```dvc +$ dvc exp show --include-params=featurize +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ +│ └── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +We may want to branch-off `exp-1dad0` for a separate experimentation process +(based on 2000 `max_features`). + +```dvc +$ dvc exp branch exp-1dad0 maxf-2000 +Git branch 'maxf-2000' has been created from experiment 'exp-1dad0'. +To switch to the new branch run: + git checkout my-branch +``` + +We can inspect the result with Git: + +```dvc +$ git branch +* master + maxf-2000 +``` + +`maxf-2000` can now be checked out, merged, rebased, pushed, etc. like any other +Git branch. diff --git a/content/docs/command-reference/exp/diff.md b/content/docs/command-reference/exp/diff.md index 69226b98eb..8d4610d5d7 100644 --- a/content/docs/command-reference/exp/diff.md +++ b/content/docs/command-reference/exp/diff.md @@ -75,3 +75,59 @@ all the current experiments (without comparisons). problems arise, otherwise 1. - `-v`, `--verbose` - displays detailed tracing information. + +## Examples + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have run 3 experiments in our project: + +```dvc +$ dvc exp show --include-params=featurize +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ +│ └── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +Since we haven't made any changes to the workspace, we can compare `exp-1dad0` +to its baseline (`10-bigrams-experiment`, current `HEAD`) like this: + +```dvc +$ dvc exp diff exp-1dad0 +Path Metric Value Change +scores.json auc 0.61314 0.035575 +Path Param Value Change +params.yaml featurize.max_features 1500 -500 +``` + +To compare two specific experiments (values are shown for the second one by +default): + +```dvc +$ dvc exp diff exp-1dad0 exp-1df77 +Path Metric Value Change +scores.json auc 0.51676 -0.060799 +Path Param Value Change +params.yaml featurize.max_features 500 -1500 +``` + +To compare an experiment to the +[`7-ml-pipeline`](https://github.com/iterative/example-get-started/releases/tag/7-ml-pipeline) +tag (or any other [revision](https://git-scm.com/docs/revisions)): + +```dvc +$ dvc exp diff exp-1dad0 7-ml-pipeline +Path Metric Value Change +scores.json auc None diff not supported +Path Param Value Change +params.yaml featurize.max_features 500 -1500 +params.yaml featurize.ngrams 1 -1 +``` diff --git a/content/docs/command-reference/exp/gc.md b/content/docs/command-reference/exp/gc.md index 8816f9f457..01d74213d8 100644 --- a/content/docs/command-reference/exp/gc.md +++ b/content/docs/command-reference/exp/gc.md @@ -58,3 +58,73 @@ separately to delete it. - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Examples + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have the following project, and have just +[applied](/docs/command-reference/exp/apply) and committed `exp-1dad0` (current +`HEAD` of `master`): + +```dvc +$ dvc exp show --all-commits --include-params=featurize +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.57756 │ 2000 │ 2 │ +│ master │ 05:39 PM │ 0.57756 │ 2000 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ +│ └── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +│ 9-bigrams-model │ Jun 20, 2020 │ 0.54175 │ 1500 │ 2 │ +│ └── exp-069d9 │ Sep 24, 2020 │ 0.51076 │ 2500 │ 2 │ +│ 8-evaluation │ Jun 20, 2020 │ 0.54175 │ 500 │ 1 │ +│ 7-ml-pipeline │ Jun 20, 2020 │ - │ 500 │ 1 │ + ... +│ 0-git-init │ Jun 20, 2020 │ - │ 1500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +If we consider all the other experiments unnecessary, we can delete them like +this: + +```dvc +$ dvc exp gc -w +WARNING: This will remove all experiments except ... +Are you sure you want to proceed? [y/n] y +Removed 4 experiments. To remove unused cache files use 'dvc gc'. +``` + +We can confirm that all the previous experiments are gone: + +```dvc +$ dvc exp show --all-commits --include-params=featurize +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.57756 │ 2000 │ 2 │ +│ master │ 05:39 PM │ 0.57756 │ 2000 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ 9-bigrams-model │ Jun 20, 2020 │ 0.54175 │ 1500 │ 2 │ + ... +│ 0-git-init │ Jun 20, 2020 │ - │ 2000 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +To remove any cached data associated to the deleted experiments and +which are no longer needed in the project, we can use regular `dvc gc` (with the +appropriate options): + +```dvc +$ dvc dvc gc --all-commits +WARNING: This will remove all cache except ... +Are you sure you want to proceed? [y/n] y +... +``` + +> Note the use of `--all-commits` to ensure that we do not garbage collect files +> or directories referenced in remaining commits in the repo. diff --git a/content/docs/command-reference/exp/list.md b/content/docs/command-reference/exp/list.md index e0bbf086b2..e7889404e0 100644 --- a/content/docs/command-reference/exp/list.md +++ b/content/docs/command-reference/exp/list.md @@ -20,7 +20,8 @@ limited to experiment names and with very simple formatting. See also `dvc exp run`. If a working `git_remote` name (e.g. `origin`) or valid Git repo's URL is -provided, lists experiments in that repository instead (if any). +provided, lists experiments in that repository instead (if any, +based on the `dvc remote default`). > Note that this utility doesn't require an existing DVC project to > run from when a `git_remote` URL is given. @@ -45,3 +46,51 @@ options below). - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Examples + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have run 3 experiments in our project. You can quickly list the +available experiments with this command: + +```dvc +$ dvc exp list --all +10-bigrams-experiment: + exp-e6c97 + exp-1dad0 + exp-1df77 +``` + +> Contrast this with the full table +> [displayed by `dvc exp show`](/doc/command-reference/exp/show#examples). + +You can also list experiments in any DVC repo with `dvc exp list`: + +```dvc +$ dvc exp list --all git@github.com:iterative/example-get-started.git +10-bigrams-experiment: + exp-e6c97 + exp-86dd6 +``` + +We can see that two experiments are available in +([the DVC repo](https://github.com/iterative/example-get-started)). + +If we're currently in a local clone of the repo, we can also use +[Git remote](https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes) +name instead: + +```dvc +$ git remote -v +origin git@github.com:iterative/example-get-started.git +$ dvc exp list --all origin +10-bigrams-experiment: + exp-e6c97 + exp-86dd6 +``` + +And in this context, `dvc exp pull` can download the experiments if needed, as +`dvc exp push` can upload any local ones we wish to share. diff --git a/content/docs/command-reference/exp/pull.md b/content/docs/command-reference/exp/pull.md index af00cfc2d1..2539ac2707 100644 --- a/content/docs/command-reference/exp/pull.md +++ b/content/docs/command-reference/exp/pull.md @@ -70,3 +70,28 @@ given with `--remote`. - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Examples + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have cloned a DVC repository, and would like to fetch an experiment +that someone else shared (see also `dvc exp list`). + +```dvc +$ dvc exp list --all origin +master: + exp-e6c97 +$ dvc exp pull origin exp-e6c97 +Pulled experiment 'exp-e6c97' from Git remote 'origin'. +``` + +We can now see that the experiment exists in the local repo: + +```dvc +$ dvc exp list --all +master: + exp-e6c97 +``` diff --git a/content/docs/command-reference/exp/push.md b/content/docs/command-reference/exp/push.md index 16fc4e27f3..bce2090c3b 100644 --- a/content/docs/command-reference/exp/push.md +++ b/content/docs/command-reference/exp/push.md @@ -66,3 +66,34 @@ given with `--remote`. - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Examples + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have run 3 experiments in our project: + +```dvc +$ dvc exp list --all +11-bigrams-experiment: + exp-e6c97 + exp-1dad0 + exp-1df77 +``` + +We would now like to share one of them with others via the Git remote: + +```dvc +$ dvc exp push origin exp-e6c97 +Pushed experiment 'exp-e6c97' to Git remote 'origin'. +``` + +We can now see that the experiment exists in the remote repo: + +```dvc +$ dvc exp list --all origin +master: + exp-e6c97 +``` diff --git a/content/docs/command-reference/exp/remove.md b/content/docs/command-reference/exp/remove.md index d196f9c21c..6363d9d662 100644 --- a/content/docs/command-reference/exp/remove.md +++ b/content/docs/command-reference/exp/remove.md @@ -32,3 +32,25 @@ With `--queue`, the queue of experiments is cleared. - `-v`, `--verbose` - displays detailed tracing information from executing the `dvc pull` command. + +## Examples + +Let's say we have `dvc exp run` 3 experiments in our project: + +```dvc +$ dvc exp list +master: + exp-e6c97 + exp-1dad0 + exp-1df77 +``` + +To remove any of them, just give their names to `dvc exp remove`: + +```dvc +$ dvc exp remove exp-1dad0 exp-1df77 + +$ dvc exp list +master: + exp-e6c97 +``` diff --git a/content/docs/command-reference/exp/run.md b/content/docs/command-reference/exp/run.md index 8dcf1292a9..28bc2cba13 100644 --- a/content/docs/command-reference/exp/run.md +++ b/content/docs/command-reference/exp/run.md @@ -170,3 +170,91 @@ CPU cores). regardless of this flag. - `-v`, `--verbose` - displays detailed tracing information. + +## Examples + +> These examples are based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +
+ +### Expand to prepare the example ML project + +Clone the DVC repo and download the data it depends on: + +```dvc +$ git clone git@github.com:iterative/example-get-started.git +$ cd example-get-started +$ dvc pull +``` + +Let's also install the Python requirements: + +> We **strongly** recommend creating a +> [virtual environment](https://python.readthedocs.io/en/stable/library/venv.html) +> first. + +```dvc +$ pip install -r src/requirements.txt +``` + +
+ +Let's check the latest metrics of the project: + +```dvc +$ dvc metrics show +Path avg_prec roc_auc +scores.json 0.60405 0.9608 +``` + +For this experiment, we want to see the results for a smaller dataset input, so +let's limit the data to 20 MB and reproduce the pipeline with `dvc exp run`: + +```dvc +$ truncate --size=20M data/data.xml +$ dvc exp run +... +Reproduced experiment(s): exp-44136 +Experiment results have been applied to your workspace. + +$ dvc metrics diff +Path Metric Old New Change +scores.json avg_prec 0.60405 0.56103 -0.04302 +scores.json roc_auc 0.9608 0.94003 -0.02077 +``` + +The `dvc metrics diff` command shows the difference in performance for the +experiment we just ran (`exp-44136`). + +## Example: Modify parameters on-the-fly + +You could modify a params file just like any other dependency and +run an experiment on that basis. Since this is a common need, `dvc exp run` +comes with the `--set-param` (`-S`) option built-in. This saves you the need to +manually edit the params file: + +```dvc +$ dvc exp run -S prepare.split=0.25 -S featurize.max_features=2000 +... +Reproduced experiment(s): exp-18bf6 +Experiment results have been applied to your workspace. +``` + +To see the results, we can use `dvc exp diff` which compares both params and +metrics to the previous project version: + +```dvc +$ dvc exp diff +Path Metric Value Change +scores.json avg_prec 0.58187 -0.022184 +scores.json roc_auc 0.93634 -0.024464 + +Path Param Value Change +params.yaml featurize.max_features 2000 -1000 +params.yaml prepare.split 0.25 0.05 +``` + +> Notice that experiments run as a series don't build up on each other. They are +> all based on `HEAD`. diff --git a/content/docs/command-reference/exp/show.md b/content/docs/command-reference/exp/show.md index 1a49b974fa..ffad01934f 100644 --- a/content/docs/command-reference/exp/show.md +++ b/content/docs/command-reference/exp/show.md @@ -109,3 +109,91 @@ metric or param. problems arise, otherwise 1. - `-v`, `--verbose` - displays detailed tracing information. + +## Examples + +> This example is based on our +> [Get Started](/doc/tutorials/get-started/experiments), where you can find the +> actual source code. + +Let's say we have run 3 experiments in our project. The basic usage shows the +workspace (Git working tree) and experiments derived from `HEAD` +(`10-bigrams-experiment` branch in this case), and all of their metrics and +params (scroll right to see all): + +```dvc +$ dvc exp show +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_fea… ┃ featurize.ngrams ┃ prepare.seed ┃ prepare.split ┃ train.n_estimators ┃ train.seed ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ 20170428 │ 0.2 │ 50 │ 20170428 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ 20170428 │ 0.2 │ 50 │ 20170428 │ +│ ├── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ 20170428 │ 0.2 │ 50 │ 20170428 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ 20170428 │ 0.2 │ 50 │ 20170428 │ +│ └── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ 20170428 │ 0.2 │ 50 │ 20170428 │ +└───────────────────────┴──────────────┴─────────┴────────────────────┴──────────────────┴──────────────┴───────────────┴────────────────────┴────────────┘ +``` + +> You can exit this screen with `Q`, typically. + +Let's limit the param columns to only include the `featurize` group: + +```dvc +$ dvc exp show --include-params=featurize +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ +│ └── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +Sort experiments by the `auc` metric, in ascending order: + +```dvc +$ dvc exp show --include-params=featurize --sort-by=auc --sort-order=asc +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ +│ └── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +To see all experiments throughout the Git history: + +```dvc +$ dvc exp show --all-commits --include-params=featurize --sort-by=auc --sort-order=asc +┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ +┃ Experiment ┃ Created ┃ auc ┃ featurize.max_features ┃ featurize.ngrams ┃ +┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ +│ workspace │ - │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-experiment │ Jun 20, 2020 │ 0.61314 │ 1500 │ 2 │ +│ ├── exp-1df77 │ Oct 09, 2020 │ 0.51676 │ 500 │ 2 │ +│ ├── exp-1dad0 │ Oct 09, 2020 │ 0.57756 │ 2000 │ 2 │ +│ └── exp-e6c97 │ Oct 21, 2020 │ 0.61314 │ 1500 │ 2 │ +│ 10-bigrams-model │ Jun 20, 2020 │ 0.54175 │ 1500 │ 2 │ +│ └── exp-069d9 │ Sep 24, 2020 │ 0.51076 │ 2500 │ 2 │ +│ 9-evaluation │ Jun 20, 2020 │ 0.54175 │ 500 │ 1 │ +│ 8-ml-pipeline │ Jun 20, 2020 │ - │ 500 │ 1 │ +│ 6-prep-stage │ Jun 20, 2020 │ - │ 500 │ 1 │ +│ 5-source-code │ Jun 20, 2020 │ - │ 500 │ 1 │ +│ 4-import-data │ Jun 20, 2020 │ - │ 1500 │ 2 │ +│ 2-track-data │ Jun 20, 2020 │ - │ 1500 │ 2 │ +│ 3-config-remote │ Jun 20, 2020 │ - │ 1500 │ 2 │ +│ 1-dvc-init │ Jun 20, 2020 │ - │ 1500 │ 2 │ +│ 0-git-init │ Jun 20, 2020 │ - │ 1500 │ 2 │ +└───────────────────────┴──────────────┴─────────┴────────────────────────┴──────────────────┘ +``` + +Note that in the final example, Git commits remain in chronological order. The +sorting only applies to experiment groups (sharing a parent commit). + +📖 See [Metrics, Parameters, and Plots](/doc/start/metrics-parameters-plots) for +an introduction to parameters, metrics, plots. diff --git a/content/docs/user-guide/basic-concepts/dependency.md b/content/docs/user-guide/basic-concepts/dependency.md index 2af6b23b8e..256e77f183 100644 --- a/content/docs/user-guide/basic-concepts/dependency.md +++ b/content/docs/user-guide/basic-concepts/dependency.md @@ -1,6 +1,6 @@ --- name: Dependency -match: [dependency, dependencies] +match: [dependency, dependencies, depends] tooltip: >- A file or directory (possibly tracked by DVC) recorded in the `deps` section of a stage (in `dvc.yaml`) or `.dvc` file file. See `dvc run`. Stages are