Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add docs for dvc metrics diff #933

Merged
merged 65 commits into from
Feb 13, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
5f0da08
add docs for dvc metrics diff
efiop Jan 17, 2020
9da0661
nav: add `metrics diff` to sidebar
jorgeorpinel Jan 24, 2020
4a1b775
cmd ref: typos in `metrics diff`
jorgeorpinel Jan 27, 2020
8ae2e6d
cmd ref: rewrite `metrics diff` ref and
jorgeorpinel Jan 27, 2020
041fd22
Merge branch 'master' into 921
jorgeorpinel Jan 27, 2020
c056834
cmd ref: update descs, review options, link all metrics subcmds
jorgeorpinel Jan 27, 2020
344839a
cmd ref: update cmd argument descriptions for `diff` and `metics diff`
jorgeorpinel Jan 27, 2020
5a08d37
metrics diff: big terminology review around the intro of this new com…
jorgeorpinel Jan 29, 2020
f10cf2b
term: review usage of "hash", "commit hash", "SHA", and "MD5"
jorgeorpinel Jan 29, 2020
1d14086
term: rewrite definition of "workspace"
jorgeorpinel Jan 29, 2020
e55c362
cmd ref: change link from `metrics diff` options to `metrics show`
jorgeorpinel Jan 30, 2020
f989586
cmd ref: update example in `dvc metrics diff` and similar ones
jorgeorpinel Jan 30, 2020
734994a
cmd ref: simplify dvc gc -a option
jorgeorpinel Jan 30, 2020
961b513
cmd ref: use "reference" more than "revision" in diff
jorgeorpinel Jan 30, 2020
6b259ba
cmd ref: link term "revision" in diff and `metrics diff`
jorgeorpinel Jan 30, 2020
c006d18
term: put Git ref exapmles before term and link
jorgeorpinel Jan 30, 2020
e76329a
cmd ref: friendlier explanation of "tip of default branch"
jorgeorpinel Jan 30, 2020
d02ccd2
cmd ref: use tag name instead of term "the revision"
jorgeorpinel Jan 30, 2020
14d4c23
term: revert some "revision"->"reference" changes, and related simpli…
jorgeorpinel Jan 30, 2020
e7e0b97
cmd ref: review desc. of `-a` options throughout refs
jorgeorpinel Jan 30, 2020
c5dbb96
cmd ref: update diff params
jorgeorpinel Jan 30, 2020
b30df29
cmd ref: update notes around moving/static Git refs in import and update
jorgeorpinel Jan 30, 2020
1e9f3ae
revert workspace glossary entry
jorgeorpinel Jan 30, 2020
af6fc63
tutorial: use full name of Deep Dive Tutorial in title and links
jorgeorpinel Jan 30, 2020
bd0c9bd
user-guide: undo change on "binary" literal for analytics example
jorgeorpinel Jan 30, 2020
a0c51ff
use-cases: avoid term "revision" in data-registries
jorgeorpinel Jan 30, 2020
a8a7c1d
term: revert "hash"->"checksum" in this PR
jorgeorpinel Jan 31, 2020
a1c782a
cmd ref: "revision"->"commit" in get ref
jorgeorpinel Jan 31, 2020
fc83207
cmd ref: use correct tag names in checkout examples
jorgeorpinel Jan 31, 2020
cc82aca
diff: remove backquotes adound "HEAD" same as in core repo
jorgeorpinel Feb 1, 2020
f611b4c
Merge branch 'master' into 921
jorgeorpinel Feb 9, 2020
4d7c9c0
cmd ref: don't use link to git reference doc
jorgeorpinel Feb 9, 2020
43120ad
cmd ref: don't use term "revision" in diff, prefer "commit"
jorgeorpinel Feb 9, 2020
541c8d3
cmd ref: no need for word "specific" (or "SHA") in get/import
jorgeorpinel Feb 9, 2020
296c2f0
cmd ref: update "project"->"workspace" term and example intros in `dv…
jorgeorpinel Feb 9, 2020
d158a57
docs: 2 misc updates
jorgeorpinel Feb 10, 2020
cbecbb3
tutorials: update model->"data or model"
jorgeorpinel Feb 10, 2020
455e793
cmd ref: fixed link to `metrics diff` and updated mention of it in `m…
jorgeorpinel Feb 10, 2020
88fbb09
get-started: typo in pipelines chapter
jorgeorpinel Feb 10, 2020
57ff736
cmd ref: rewrite paragraph about fixed revision import stages in `upd…
jorgeorpinel Feb 10, 2020
cb134fe
cmd ref: rewrite p about `repro` rewriting artifacts in cache
jorgeorpinel Feb 10, 2020
b50d6a9
cmd ref: rephrase and split p about what to compare and about targets…
jorgeorpinel Feb 10, 2020
5fbec22
cmd ref: reorg last part of repro desc
jorgeorpinel Feb 11, 2020
76cb691
cmd ref: restore `git tag` sample output in checkout examples
jorgeorpinel Feb 11, 2020
c64e70e
cmd ref: small rewording around tag names
jorgeorpinel Feb 11, 2020
1eb88fe
cmd ref: `metrics diff` and `diff` intro updates
jorgeorpinel Feb 11, 2020
fc25128
term: use "version" instead of "revision" in `import` cmd ref
jorgeorpinel Feb 12, 2020
1400f4a
cmd ref: updates to `metrics diff` (and `diff`) descriptions
jorgeorpinel Feb 12, 2020
d52dc5a
cmd ref: change "ref" -> "rev" per iterative/dvc/pull/3299
jorgeorpinel Feb 12, 2020
9e2b173
cmd ref: "revision"->"version" in a couple more docs
jorgeorpinel Feb 12, 2020
630875f
cmd ref: simplify note about `metrics diff` in `metrics show`
jorgeorpinel Feb 12, 2020
70f9ad5
cmd ref: use descriptive exampe-get-started repo tags in `get` examples
jorgeorpinel Feb 12, 2020
51d5605
term: "commit hash"->"commit SHA hash" to match #962 but
jorgeorpinel Feb 12, 2020
6ed1d44
cmd ref: improve -a adn -c option descs
jorgeorpinel Feb 13, 2020
f0f8c2d
cmd ref: remove p about --targets option in metrics diff
jorgeorpinel Feb 13, 2020
da227d6
cmd ref: rewrite pa bout fixed revisions/re-importing in `update`
jorgeorpinel Feb 13, 2020
0ead25e
tutorials: use and instead of data "or" models in versioning tut
jorgeorpinel Feb 13, 2020
4f49733
user-guide: restore bullet about `git` in analytics
jorgeorpinel Feb 13, 2020
bbf573f
you don't usually merge tags
jorgeorpinel Feb 13, 2020
6215bc2
term: don't use "version of repo/project" when referring to commits
jorgeorpinel Feb 13, 2020
fb459f9
cmd ref: simlpify note about metrics diff in metrics show
jorgeorpinel Feb 13, 2020
e12e3d7
cmd ref: and->and/or in checkout sample of versioning tut
jorgeorpinel Feb 13, 2020
2b2bfca
user-guide: updated analytics details
jorgeorpinel Feb 13, 2020
fc6d3e9
cmd ref: restore simpler wording about status -aT desc
jorgeorpinel Feb 13, 2020
1473ebe
cmd ref: correct (again) the short desc for diff and metrics diff
jorgeorpinel Feb 13, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions public/static/docs/command-reference/metrics/diff.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# metrics diff

Find and print [project metrics](/doc/command-reference/metrics#description)
changes between commits, commit and a working tree, etc.

## Synopsis

```usage
usage: dvc metrics diff [-h] [-q | -v] [--targets [TARGETS [TARGETS ...]]]
[-t TYPE] [-x XPATH] [-R] [--show-json]
[a_ref] [b_ref]

positional arguments:
a_ref Git reference from which diff is calculated. If
omitted `HEAD`(latest commit) is used.
b_ref Git reference to which diff is calculated. If omitted
current working tree is used.
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
```

## Description
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

Finds and prints changes between commits for all metrics in the
<abbr>project</abbr> by examining all of its
[DVC-files](/doc/user-guide/dvc-file-format). If `--targets` are provided, it
will show changes for those specific metric files instead.

The optional `--targets` argument can contain several metric files. With the
`-R` option, a target can even be a directory, so that DVC recursively shows
changes for all metric files in it.

Providing a `type` (`-t` option) overwrites the full metric specification (both
`type` and `xpath` fields) defined in the
[DVC-file](/doc/user-guide/dvc-file-format) (usually set originally with the
`dvc metrics modify` command).

If `type` (via `-t`) is not specified and only `xpath` (`-x` option) is, only
the `xpath` field is overwritten in its DVC-file. (DVC will first try to read
`type` from the DVC-file, but it can be automatically detected by the file
extension.)

> Alternatively, see `dvc metrics modify` command to learn how to apply `-t` and
> `-x` permanently.

## Options

- `--targets` - metric files or directories (see -R) to show changes for. If not
specified, will show changes for all metric files, if not specified.

- `-t`, `--type` - specify a type of the metric file. Accepted values are:
`raw`, `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be used to determine
appropriate parsing and displaying format for this metric file and will
override `type` defined in the corresponding DVC-file See `dvc metrics modify`
for a full description of acceptable types. If no `type` is specified either
as a CLI `-t|--type` nor in the corresponding DVC-file itself,
`dvc metrics diff` will try to detect it on-the-fly.

- `-x`, `--xpath` - specify a path within a metric file to get a specific metric
value to show changes for. If ommited, will show changes for all possible
paths. It will override `xpath` defined in the correspodning DVC-file. See
`dvc metrics modify` for a full description of `xpath` when applied to
specific metric types.

If multiple metric files exist in the <abbr>project</abbr>, the same parser
and path will be applied to all of them. If `xpath` for a particular metric
has been set using `dvc metrics modify`, the path passed with this option will
overwrite it for the current command run only – It may fail to produce any
results or parse files that are not in a corresponding format in this case.

- `-R`, `--recursive` - `path` is expected to be a directory for this option to
have effect. Determines the metric files to show changes for by searching each
target directory and its subdirectories for DVC-files to inspect.

- `--show-json` - prints diff in easilly parsable JSON format instead of
human-readable table.

- `-h`, `--help` - prints the usage/help message, and exit.

- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no
problems arise, otherwise 1.

- `-v`, `--verbose` - displays detailed tracing information.

## Examples

Let's create a metrics file using a dummy command and commit it to git:

```
$ dvc run -M metrics.json 'echo "{\"AUC\": 0.5}" > metrics.json'
$ git commit -a -m "add metrics"
```

Now let's say we've adjusted our scripts and our AUC has changed:

```
$ dvc run -M metrics.json 'echo "{\"AUC\": 0.6}" > metrics.json'
```

To see the change, let's run `dvc metrics diff` without arguments, that would
compare our current metrics to what we've had in the last commit (similar to
`git diff`):

```
$ dvc metrics diff
Path Metric Value Change
metrics.json AUC 0.600 0.100
```
4 changes: 4 additions & 0 deletions public/static/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,10 @@
"label": "metrics modify",
"slug": "modify"
},
{
"label": "metrics diff",
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
"slug": "diff"
},
{
"label": "metrics remove",
"slug": "remove"
Expand Down