diff --git a/content/docs/command-reference/metrics/diff.md b/content/docs/command-reference/metrics/diff.md index dd381aecc3..cfbecae900 100644 --- a/content/docs/command-reference/metrics/diff.md +++ b/content/docs/command-reference/metrics/diff.md @@ -1,6 +1,6 @@ # metrics diff -Show changes in [metrics](/doc/command-reference/metrics) between commits in the +Compare [metrics](/doc/command-reference/metrics) between two commits in the DVC repository, or between a commit and the workspace. ## Synopsis @@ -21,28 +21,23 @@ positional arguments: ## Description This command provides a quick way to compare metrics among experiments in the -repository history. It requires that Git is being used to version the metrics. +repository history. All metrics defined in `dvc.yaml` are used by default. The +comparison shown by this command includes the new value, and the numeric +difference (delta) with the previous value (rounded to 5 digits precision). -> This kind of metrics can be defined with the `-m` (`--metrics`) and `-M` -> (`--metrics-no-cache`) options of `dvc run`. - -Run without arguments, this command compares metrics currently present in the -workspace uncommitted changes) with the latest committed version. - -The differences shown by this command include the new value, and numeric -difference (delta) from the previous value of metrics (rounded to 5 digits -precision). They're calculated between two commits (hash, branch, tag, or any -[Git revision](https://git-scm.com/docs/revisions)) for all metrics in the -project, found by examining all of the `dvc.yaml` and `.dvc` files -in both versions. +`a_rev` and `b_rev` are Git commit hashes, tag, or branch names. If none are +specified, `dvc metrics diff` compares metrics currently present in the +workspace (uncommitted changes) with the latest committed versions +(required). A single specified revision results in comparing the workspace and +that version. Another way to display metrics is the `dvc metrics show` command, which just -lists all the current metrics without comparisons. +lists all the current metrics, without comparisons. ## Options -- `--targets ` - limit command scope to these metric files. Using `-R`, - directories to search metric files in can also be given. When specifying +- `--targets ` - limit command scope to these metrics files. Using `-R`, + directories to search metrics files in can also be given. When specifying arguments for `--targets` before `revisions`, you should use `--` after this option's arguments, e.g.: @@ -56,7 +51,7 @@ lists all the current metrics without comparisons. $ dvc metrics diff HEAD v1 --targets t1.json t2.json ``` -- `-R`, `--recursive` - determines the metric files to use by searching each +- `-R`, `--recursive` - determines the metrics files to use by searching each target directory and its subdirectories for DVC-files to inspect. If there are no directories among the `targets`, this option is ignored. @@ -124,7 +119,7 @@ metrics.json TP 531 4 ## Example: compare metrics among specific versions -Metric files committed with Git can be compared by referencing the commits (any +Metrics files committed with Git can be compared by referencing the commits (any two [revisions](https://git-scm.com/docs/revisions)): ```dvc diff --git a/content/docs/command-reference/metrics/index.md b/content/docs/command-reference/metrics/index.md index 9a60c89f37..8da2126e2f 100644 --- a/content/docs/command-reference/metrics/index.md +++ b/content/docs/command-reference/metrics/index.md @@ -32,8 +32,9 @@ ability to mark a certain stage outputs as metrics. These metrics are project-specific floating-point or integer values e.g. AUC, ROC, false positives, etc. -This kind of metrics can be defined with the `-m` (`--metrics`) and `-M` -(`--metrics-no-cache`) options of `dvc run`. +This type of metrics files are typically generated by user data processing code, +and are tracked using the `-m` (`--metrics`) and `-M` (`--metrics-no-cache`) +options of `dvc run`. In contrast to `dvc plots`, these metrics should be stored in hierarchical files. Unlike its `dvc plots` counterpart, `dvc metrics diff` can report the @@ -46,7 +47,7 @@ $ dvc metrics diff summary.json AUC 0.801807 0.037826 ``` -`dvc metrics` subcommands by default use the metric files specified in +`dvc metrics` subcommands by default use the metrics files specified in `dvc.yaml` (if any), for example `summary.json` below: ```yaml @@ -63,7 +64,7 @@ stages: ``` > `cache: false` above specifies that `summary.json` is not tracked or -> cached by DVC (`-M` option of `dvc run`). These metric files are +> cached by DVC (`-M` option of `dvc run`). These metrics files are > normally committed with Git instead. See `dvc.yaml` for more information on > the file format above. @@ -113,12 +114,12 @@ $ dvc run -n evaluate -d code/evaluate.py -M eval.json \ python code/evaluate.py ``` -> `-M` (`--metrics-no-cache`) tells DVC to mark `eval.json` as a metric file, +> `-M` (`--metrics-no-cache`) tells DVC to mark `eval.json` as a metrics file, > without tracking it directly (You can track it with Git). See `dvc run` for > more info. -Now let's print metric values that we are tracking in this project, -using `dvc metrics show`: +Now let's print metrics values that we are tracking in this +project, using `dvc metrics show`: ```dvc $ dvc metrics show @@ -128,7 +129,7 @@ $ dvc metrics show TP: 516 ``` -When there are metric file changes (before committing them with Git), the +When there are metrics file changes (before committing them with Git), the `dvc metrics diff` command shows the difference between metrics values: ```dvc diff --git a/content/docs/command-reference/metrics/show.md b/content/docs/command-reference/metrics/show.md index a58d16770f..60cd9f67e3 100644 --- a/content/docs/command-reference/metrics/show.md +++ b/content/docs/command-reference/metrics/show.md @@ -9,8 +9,8 @@ usage: dvc metrics show [-h] [-q | -v] [-a] [-T] [--all-commits] [-R] [--show-json] [targets [targets ...]] positional arguments: - targets Limit command scope to these metric files. - Using -R, directories to search metric files + targets Limit command scope to these metrics files. + Using -R, directories to search metrics files in can also be given. ``` @@ -19,13 +19,10 @@ positional arguments: Finds and prints all metrics in the project by examining all of its [DVC-files](/doc/user-guide/dvc-files-and-directories). -> This kind of metrics can be defined with the `-m` (`--metrics`) and `-M` -> (`--metrics-no-cache`) options of `dvc run`. - -If `targets` are provided, it will show those specific metric files instead. +If `targets` are provided, it will show those specific metrics files instead. With the `-a` or `-T` options, this command shows the different metrics values across all Git branches or tags, respectively. With the `-R` option, some of the -target can even be directories, so that DVC recursively shows all metric files +target can even be directories, so that DVC recursively shows all metrics files inside. An alternative way to display metrics is the `dvc metrics diff` command, which @@ -33,7 +30,7 @@ compares them with a previous version. ## Options -- `-a`, `--all-branches` - print metric file contents in all Git branches +- `-a`, `--all-branches` - print metrics file contents in all Git branches instead of just those present in the current workspace. It can be used to compare different experiments. Note that this can be combined with `-T` below, for example using the `-aT` flag. @@ -49,7 +46,7 @@ compares them with a previous version. - `--show-json` - prints the command's output in easily parsable JSON format, instead of a human-readable table. -- `-R`, `--recursive` - determines the metric files to show by searching each +- `-R`, `--recursive` - determines the metrics files to show by searching each target directory and its subdirectories for DVC-files to inspect. If there are no directories among the `targets`, this option is ignored. @@ -122,4 +119,4 @@ increase_bow: The [Compare Experiments](/doc/tutorials/get-started/experiments#compare-experiments) chapter of our _Get Started_ covers the `-a` option to collect and print a -metric file value across all Git branches. +metrics file value across all Git branches. diff --git a/content/docs/command-reference/plots/diff.md b/content/docs/command-reference/plots/diff.md index 5ebef31601..413d54e5f6 100644 --- a/content/docs/command-reference/plots/diff.md +++ b/content/docs/command-reference/plots/diff.md @@ -1,7 +1,7 @@ # plots diff Show multiple versions of [plot metrics](/doc/command-reference/plots) by -plotting them in a single image. This allows to easily compare them. +overlaying them in a single plot. This allows to compare them easily. ## Synopsis @@ -24,7 +24,7 @@ experiments in the repository history, by plotting multiple versions of the metrics. All plots defined in `dvc.yaml` are used by default. > Note that unlike `dvc metrics diff`, this command does not calculate numeric -> differences between metric file values. +> differences between metrics file values. `revisions` are Git commit hashes, tag, or branch names. If none are specified, `dvc plots diff` compares targets currently present in the @@ -33,19 +33,22 @@ versions (required). A single specified revision results in comparing the workspace and that version. Note that any number of `revisions` can be provided, and the resulting plot -shows all of them in a single output. +shows all of them in a single image. The plot style can be customized with [plot templates](/doc/command-reference/plots#plot-templates), using the -`--template` option. To learn more about metric file formats and templates +`--template` option. To learn more about metrics file formats and templates please see `dvc plots`. > Note that the default behavior of this command can be modified per metrics > file with `dvc plots modify`. +Another way to display plots is the `dvc plots show` command, which just lists +all the current plots, without comparisons. + ## Options -- `--targets ` - specific metric files to visualize. These must be listed +- `--targets ` - specific metrics files to visualize. These must be listed in a [`dvc.yaml`](/doc/user-guide/dvc-files-and-directories#dvcyaml-file) file (see the `--plots` option of `dvc run`). When specifying arguments for `--targets` before `revisions`, you should use `--` after this option's @@ -74,11 +77,11 @@ please see `dvc plots`. auto-generated `index` field is used by default. See [Custom templates](/doc/command-reference/plots#custom-templates) for more information on this `index` field. Column names or numbers are expected for - tabular metric files. + tabular metrics files. - `-y ` - field name from which the Y axis data comes from. The last field found in the `--targets` is used by default. Column names or numbers are - expected for tabular metric files. + expected for tabular metrics files. - `--x-label ` - X axis label. The X field name is the default. @@ -143,7 +146,7 @@ cat,turtle The predefined confusion matrix [template](/doc/command-reference/plots#plot-templates) (in -`.dvc/plots/confusion.json`) shows how metric comparisons can be faceted by +`.dvc/plots/confusion.json`) shows how metrics comparisons can be faceted by separate plots. It can be enabled with `-t` (`--template`): ```dvc diff --git a/content/docs/command-reference/plots/index.md b/content/docs/command-reference/plots/index.md index d206f24bf4..a0f35d74a7 100644 --- a/content/docs/command-reference/plots/index.md +++ b/content/docs/command-reference/plots/index.md @@ -33,9 +33,11 @@ DVC provides a set of commands to visualize metrics of machine learning experiments. Usual plot examples are AUC curves, loss functions, confusion matrices, among others. -This kind of metric files are created by users, or generated by user data -processing code. `dvc plots` subcommands can work with metric files committed to -a Git repo history, data files controlled by DVC, or any other file in system. +This type of metrics files are created by users, or generated by user data +processing code, and get defined with the `-p` (`--plots`) and +`--plots-no-cache`) options of `dvc run`. `dvc plots` subcommands can work with +plots files committed to a Git repo history, data files controlled by DVC, or +any other file in system. DVC generates plots as HTML files that can be open with a web browser. These HTML files use [Vega-Lite](https://vega.github.io/vega-lite/). Vega is a @@ -118,7 +120,7 @@ You can create a custom template from scratch, or modify an existing one from 💡 Note that custom templates can be safely added to the template directory. -All metric files given to `dvc plots show` and `dvc plots diff` as input are +All metrics files given to `dvc plots show` and `dvc plots diff` as input are combined together into a single data array for injection into a template file. There are two important fields that DVC adds to the plot data: @@ -129,12 +131,12 @@ There are two important fields that DVC adds to the plot data: distinguish between different versions when using the `dvc plots diff` command. -Note that in the case of CSV/TSV metric files, column names from the table +Note that in the case of CSV/TSV metrics files, column names from the table header (first row) are equivalent to field names. ### DVC template anchors -- `` (**required**) - the plot data from any kind of metric +- `` (**required**) - the plot data from any kind of metrics files is converted to a single JSON array internally, and injected instead of this anchor. Two additional fields will be added: `index` and `rev` (explained above). diff --git a/content/docs/command-reference/plots/modify.md b/content/docs/command-reference/plots/modify.md index 30625a91e1..84a62d7b90 100644 --- a/content/docs/command-reference/plots/modify.md +++ b/content/docs/command-reference/plots/modify.md @@ -12,7 +12,7 @@ usage: dvc plots modify [-h] [-q | -v] [-t ] [-x ] target positional arguments: - target Metric file to set properties to + target Metrics file to set properties to ``` ## Description diff --git a/content/docs/command-reference/plots/show.md b/content/docs/command-reference/plots/show.md index fe3bcb41e5..3d8d398f3e 100644 --- a/content/docs/command-reference/plots/show.md +++ b/content/docs/command-reference/plots/show.md @@ -12,7 +12,7 @@ usage: dvc plots show [-h] [-q | -v] [-t ] [-x ] [targets [targets ...]] positional arguments: - targets Metric files to visualize. + targets Metrics files to visualize. Shows all plots by default. ``` @@ -22,12 +22,12 @@ This command provides a quick way to visualize metrics such as loss functions, AUC curves, confusion matrices, etc. All plots defined in `dvc.yaml` are used by default. -Optionally, specific metric file `targets` to show are accepted. These must be +Optionally, specific metrics file `targets` to show are accepted. These must be listed in a `dvc.yaml` file (see the `--plots` option of `dvc run`). The plot style can be customized with [plot templates](/doc/command-reference/plots#plot-templates), using the -`--template` option. To learn more about metric file formats and templates +`--template` option. To learn more about metrics file formats and templates please see `dvc plots`. > Note that the default behavior of this command can be modified per metrics @@ -48,11 +48,11 @@ please see `dvc plots`. auto-generated `index` field is used by default. See [Custom templates](/doc/command-reference/plots#custom-templates) for more information on this `index` field. Column names or numbers are expected for - tabular metric files. + tabular metrics files. - `-y ` - field name from which the Y axis data comes from. The last field found in the `targets` is used by default. Column names or numbers are - expected for tabular metric files. + expected for tabular metrics files. - `--x-label ` - X axis label. The X field name is the default. diff --git a/content/docs/command-reference/repro.md b/content/docs/command-reference/repro.md index 5a0cbe046f..a299cfc747 100644 --- a/content/docs/command-reference/repro.md +++ b/content/docs/command-reference/repro.md @@ -119,7 +119,7 @@ up-to-date and only execute the final stage. data repeatedly when running multiple experiments. - `-m`, `--metrics` - show metrics after reproduction. The target pipelines must - have at least one metric file defined either with the `dvc metrics` command, + have at least one metrics file defined either with the `dvc metrics` command, or by the `-M` or `-m` options of the `dvc run` command. - `--dry` - only print the commands that would be executed without actually diff --git a/content/docs/use-cases/versioning-data-and-model-files/tutorial.md b/content/docs/use-cases/versioning-data-and-model-files/tutorial.md index 20d6766ca3..44588ad577 100644 --- a/content/docs/use-cases/versioning-data-and-model-files/tutorial.md +++ b/content/docs/use-cases/versioning-data-and-model-files/tutorial.md @@ -366,7 +366,7 @@ pipelines, and try to apply it here. Don't hesitate to join our [community](/chat) and ask any questions! Another detail we only brushed upon here is the way we captured the -`metrics.csv` metric file with the `-M` option of `dvc run`. Marking this +`metrics.csv` metrics file with the `-M` option of `dvc run`. Marking this output as a metric enables us to compare its values across Git tags or branches (for example, representing different experiments). See `dvc metrics` and