Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regular updates (mid Mar) #1049

Merged
merged 6 commits into from
Mar 15, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion public/static/docs/command-reference/fetch.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Get tracked files or directories from
## Synopsis

```usage
usage: dvc fetch [-h] [-q | -v] [-j JOBS]
usage: dvc fetch [-h] [-q | -v] [-j <number>]
[-r <name>] [-a] [-T] [-d] [-R]
[targets [targets ...]]

Expand Down
33 changes: 11 additions & 22 deletions public/static/docs/command-reference/list.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,35 +88,26 @@ train.dvc
```

If you open the
[example-get-started project's page](https://github.com/iterative/example-get-started)
you will see something like:
[example-get-started project's page](https://github.com/iterative/example-get-started),
you will see a similar list, except that `model.pkl` will be missing. That's
because its tracked by DVC and not visible to Git. You can find it specified as
an output if you open
[`train.dvc`](https://github.com/iterative/example-get-started/blob/master/train.dvc).

```dvc
.gitignore
README.md
auc.metric
data
evaluate.dvc
featurize.dvc
prepare.dvc
src
train.dvc
```

The difference is the `model.pkl` binary file that is DVC-tracked and is not
visible in second listing (which is actually controlled by the `train.dvc`).

We can now, for example, run:
We can now, for example, run

```dvc
$ dvc get https://github.com/iterative/example-get-started model.pkl
```

to download the model file.
to download the model file (see `dvc get`).

## Example: List all files and directories in a data registry

We can do this recursively, using `-R` option:
Let's imagine a DVC repo used as a
[data registry](/doc/use-cases/data-registries#using-registries), structured
with different datasets in separate directories. We can do this recursively,
using `-R` option:

```dvc
$ dvc list -R https://github.com/iterative/dataset-registry
Expand All @@ -129,7 +120,5 @@ images/.gitignore
images/dvc-logo-outlines.png
images/dvc-logo-outlines.png.dvc
images/owl_sticker.png
images/owl_sticker.png.dvc
images/owl_sticker.svg
...
```
32 changes: 16 additions & 16 deletions public/static/docs/command-reference/metrics/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Mark <abbr>output</abbr> file as a
## Synopsis

```usage
usage: dvc metrics add [-h] [-q | -v] [-t TYPE] [-x XPATH] path
usage: dvc metrics add [-h] [-q | -v] [-t <type>] [-x <path>] path

positional arguments:
path Path to a metric file.
Expand All @@ -31,27 +31,27 @@ specific value, if the file contains multiple metrics. See the

## Options

- `-t`, `--type` - specify a type for the metric file. Accepted values are:
`raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be saved into
the corresponding DVC-file, and used by `dvc metrics show` to determine how to
handle displaying metrics.
- `-t <type>`, `--type <type>` - specify a type for the metric file. Accepted
values are: `raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be
saved into the corresponding DVC-file, and used by `dvc metrics show` to
determine how to handle displaying metrics.

`raw` means that no additional parsing is applied, and `--xpath` is ignored.
`htsv`/`hcsv` are the same as `tsv`/`csv`, but the values in the first row of
the file will be used as the field names and should be used to address columns
in the `--xpath` option.

- `-x`, `--xpath` - specify a path within a metric file to get a specific metric
value. Should be used if the metric file contains multiple numbers and you
want to use only one of them. Only a single path is allowed. It will be saved
into the corresponding DVC-file, and used by `dvc metrics show` to determine
how to display metrics. The accepted value depends on the metric file type
(`--type` option):

- For `json` - see [JSONPath spec](https://goessner.net/articles/JsonPath/) or
[jsonpath-ng](https://github.com/h2non/jsonpath-ng) for available options.
For example, `"AUC"` extracts the value from the following JSON-formatted
metric file: `{"AUC": "0.624652"}`.
- `-x <path>`, `--xpath <path>` - specify a path within a metric file to get a
specific metric value. Should be used if the metric file contains multiple
numbers and you want to use only one of them. Only a single path is allowed.
It will be saved into the corresponding DVC-file, and used by
`dvc metrics show` to determine how to display metrics. The accepted value
depends on the metric file type (`--type` option):

- For `json` - see [JSONPath](https://goessner.net/articles/JsonPath/) or
[jsonpath-ng](https://github.com/h2non/jsonpath-ng) to know the syntax. For
example, `"AUC"` extracts the value from the following JSON-formatted metric
file: `{"AUC": "0.624652"}`.
- For `tsv`/`csv` - `row,column` e.g. `1,2`. Indices are 0-based.
- For `htsv`/`hcsv` - `row,column name` e.g. `0,Name`. Row index is 0-based.
First row is used to specify column names and is not included into index.
Expand Down
24 changes: 12 additions & 12 deletions public/static/docs/command-reference/metrics/diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ commits in the <abbr>DVC repository</abbr>, or between a commit and the

```usage
usage: dvc metrics diff [-h] [-q | -v]
[--targets [TARGETS [TARGETS ...]]]
[-t TYPE] [-x XPATH] [-R] [--show-json]
[--targets [<path> [<path> ...]]]
[-t <type>] [-x <path>] [-R] [--show-json]
[a_ref] [b_ref]

positional arguments:
Expand All @@ -35,26 +35,26 @@ They're calculated between two commits (hash, branch, tag, or any

## Options

- `--targets` - limit the comparison to these specific metric files.
- `--targets <paths>` - limit the comparison to these specific metric files.

- `-R`, `--recursive` - determines the metric files to use by searching each
target directory and its subdirectories for DVC-files to inspect. If there are
no directories among the `targets`, this option is ignored.

- `-t`, `--type` - specify a type of the metric file. Accepted values are: `raw`
(default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be used to determine
how to parse and format metics for display. See `dvc metrics show` for more
details.
- `-t <type>`, `--type <type>` - specify a type of the metric file. Accepted
values are: `raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be
used to determine how to parse and format metics for display. See
`dvc metrics show` for more details.

This option will override `type` and `xpath` defined in the corresponding
DVC-file. If no `type` is provided or found in the DVC-file, DVC will try to
detect it based on file extension.

- `-x`, `--xpath` - specify a path within a metric file to show changes for a
specific metric value only. Should be used if the metric file contains
multiple numbers and you want to use only one of them. Only a single path is
allowed. It will override `xpath` defined in the corresponding DVC-file. See
`dvc metrics show` for more details.
- `-x <path>`, `--xpath <path>` - specify a path within a metric file to show
changes for a specific metric value only. Should be used if the metric file
contains multiple numbers and you want to use only one of them. Only a single
path is allowed. It will override `xpath` defined in the corresponding
DVC-file. See `dvc metrics show` for more details.

- `--show-json` - prints the command's output in easily parsable JSON format,
instead of a human-readable table.
Expand Down
32 changes: 16 additions & 16 deletions public/static/docs/command-reference/metrics/modify.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ options such as `type` or `xpath`. See full [options](#options) info below.
## Synopsis

```usage
usage: dvc metrics modify [-h] [-q | -v] [-t TYPE] [-x XPATH] path
usage: dvc metrics modify [-h] [-q | -v] [-t <type>] [-x <path>] path

positional arguments:
path Path to a metric file.
Expand All @@ -32,27 +32,27 @@ ERROR: failed to modify metric file settings -

## Options

- `-t`, `--type` - specify a type for the metric file. Accepted values are:
`raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be saved into
the corresponding DVC-file, and used by `dvc metrics show` to determine how to
handle displaying metrics.
- `-t <type>`, `--type <type>` - specify a type for the metric file. Accepted
values are: `raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be
saved into the corresponding DVC-file, and used by `dvc metrics show` to
determine how to handle displaying metrics.

`raw` means that no additional parsing is applied, and `--xpath` is ignored.
`htsv`/`hcsv` are the same as `tsv`/`csv`, but the values in the first row of
the file will be used as the field names and should be used to address columns
in the `--xpath` option.

- `-x`, `--xpath` - specify a path within a metric file to get a specific metric
value. Should be used if the metric file contains multiple numbers and you
want to use only one of them. Only a single path is allowed. It will be saved
into the corresponding DVC-file, and used by `dvc metrics show` to determine
how to display metrics. The accepted value depends on the metric file type
(`--type` option):

- For `json` - see [JSONPath spec](https://goessner.net/articles/JsonPath/) or
[jsonpath-ng](https://github.com/h2non/jsonpath-ng) for available options.
For example, `"AUC"` extracts the value from the following JSON-formatted
metric file: `{"AUC": "0.624652"}`.
- `-x <path>`, `--xpath <path>` - specify a path within a metric file to get a
specific metric value. Should be used if the metric file contains multiple
numbers and you want to use only one of them. Only a single path is allowed.
It will be saved into the corresponding DVC-file, and used by
`dvc metrics show` to determine how to display metrics. The accepted value
depends on the metric file type (`--type` option):

- For `json` - see [JSONPath](https://goessner.net/articles/JsonPath/) or
[jsonpath-ng](https://github.com/h2non/jsonpath-ng) to know the syntax. For
example, `"AUC"` extracts the value from the following JSON-formatted metric
file: `{"AUC": "0.624652"}`.
- For `tsv`/`csv` - `row,column` e.g. `1,2`. Indices are 0-based.
- For `htsv`/`hcsv` - `row,column name` e.g. `0,Name`. Row index is 0-based.
First row is used to specify column names and is not included into index.
Expand Down
37 changes: 18 additions & 19 deletions public/static/docs/command-reference/metrics/show.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@ formatting.
## Synopsis

```usage
usage: dvc metrics show [-h] [-q | -v]
[-t TYPE] [-x XPATH] [-a] [-T] [-R]
[targets [targets ...]]
usage: dvc metrics show [-h] [-q | -v] [-t <type>] [-x <path>]
[-a] [-T] [-R] [targets [targets ...]]

positional arguments:
targets Metric files or directories (see -R) to show
Expand Down Expand Up @@ -43,9 +42,9 @@ compares them with a previous version.

## Options

- `-t`, `--type` - specify a type for the metric file. Accepted values are:
`raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be used to
determine how to parse and format metics for display.
- `-t <type>`, `--type <type>` - specify a type for the metric file. Accepted
values are: `raw` (default), `json`, `tsv`, `htsv`, `csv`, `hcsv`. It will be
used to determine how to parse and format metics for display.

`raw` means that no additional parsing is applied, and `--xpath` is ignored.
`htsv`/`hcsv` are the same as `tsv`/`csv`, but the values in the first row of
Expand All @@ -56,19 +55,19 @@ compares them with a previous version.
DVC-file. If no `type` is provided or found in the DVC-file, DVC will try to
detect it based on file extension.

- `-x`, `--xpath` - specify a path within a metric file to get a specific metric
value. Should be used if the metric file contains multiple numbers and you
want to use only one of them. Only a single path is allowed. It will override
`xpath` defined in the corresponding DVC-file. The accepted value depends on
the metric file type (`--type` option):

- For `json` - see [JSONPath spec](https://goessner.net/articles/JsonPath/) or
[jsonpath-ng](https://github.com/h2non/jsonpath-ng) for available options.
For example, `"AUC"` extracts the value from the following JSON-formatted
metric file: `{"AUC": "0.624652"}`. You can also filter on certain values,
for example `"$.metrics[?(@.deviation_mse<0.30) & (@.value_mse>0.4)]"`
extracts only the values for model versions if they meet the given
conditions from the metric file:
- `-x <path>`, `--xpath <path>` - specify a path within a metric file to get a
specific metric value. Should be used if the metric file contains multiple
numbers and you want to use only one of them. Only a single path is allowed.
It will override `xpath` defined in the corresponding DVC-file. The accepted
value depends on the metric file type (`--type` option):

- For `json` - see [JSONPath](https://goessner.net/articles/JsonPath/) or
[jsonpath-ng](https://github.com/h2non/jsonpath-ng) to know the syntax. For
example, `"AUC"` extracts the value from the following JSON-formatted metric
file: `{"AUC": "0.624652"}`. You can also filter on certain values, for
example `"$.metrics[?(@.deviation_mse<0.30) & (@.value_mse>0.4)]"` extracts
only the values for model versions if they meet the given conditions from
the metric file:
`{"metrics": [{"dataset": "train", "deviation_mse": 0.173461, "value_mse": 0.421601}]}`
- For `tsv`/`csv` - `row,column` e.g. `1,2`. Indices are 0-based.
- For `htsv`/`hcsv` - `row,column name` e.g. `0,Name`. Row index is 0-based.
Expand Down
5 changes: 2 additions & 3 deletions public/static/docs/command-reference/pipeline/list.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@ usage: dvc pipeline list [-h] [-q | -v]

## Description

`dvc list` displays a list of all existing stages in the <abbr>project</abbr>,
grouped in their corresponding [pipeline](/doc/command-reference/pipeline), when
connected.
Displays a list of all existing stages in the <abbr>project</abbr>, grouped in
their corresponding [pipeline](/doc/command-reference/pipeline), when connected.

> Note that the stages in these lists are in ascending order, that is, from last
> to first.
Expand Down
8 changes: 4 additions & 4 deletions public/static/docs/command-reference/pipeline/show.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ instead of stages.

## Options

- `-c`, `--commands` - show pipeline as a list (graph, if `--ascii` or `--dot`
option is specified) of commands instead of paths to DVC-files.
- `-c`, `--commands` - show pipeline as a list (diagram if `--ascii` or `--dot`
is used) of commands instead of paths to DVC-files.

- `-o`, `--outs` - show pipeline as a list (graph, if `--ascii` or `--dot`
option is specified) of stage outputs instead of paths to DVC-files.
- `-o`, `--outs` - show pipeline as a list (diagram if `--ascii` or `--dot` is
used) of stage outputs instead of paths to DVC-files.

- `--ascii` - visualize pipeline. It will print a graph (ASCII) instead of a
list of path to DVC-files. (`less` pager may be used, see
Expand Down
2 changes: 1 addition & 1 deletion public/static/docs/command-reference/pull.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The default remote is used (see `dvc config core.remote`) unless the `--remote`
option is used. See `dvc remote` for more information on how to configure a
remote.

With no arguments, just `dvc pull` or `dvc pull --remote REMOTE`, it downloads
With no arguments, just `dvc pull` or `dvc pull --remote <name>`, it downloads
only the files (or directories) missing from the workspace by searching all
[DVC-files](/doc/user-guide/dvc-file-format) currently in the
<abbr>project</abbr>. It will not download files associated with earlier commits
Expand Down
4 changes: 2 additions & 2 deletions public/static/docs/command-reference/push.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Upload tracked files or directories to
## Synopsis

```usage
usage: dvc push [-h] [-q | -v] [-j JOBS]
[-r REMOTE] [-a] [-T] [-d] [-R]
usage: dvc push [-h] [-q | -v] [-j <number>]
[-r <name>] [-a] [-T] [-d] [-R]
[targets [targets ...]]

positional arguments:
Expand Down
14 changes: 7 additions & 7 deletions public/static/docs/command-reference/remote/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,7 @@ $ dvc remote add myremote "azure://"
The connection string can be found in the "Access Keys" pane of your Storage
Account resource in the Azure portal.

> 💡 Make sure the value is quoted to prevent shell from misprocessing the
> command.
> 💡 Make sure the value is quoted so its processed correctly by the console.

- `container name` - this is the top-level container in your Azure Storage
Account under which all the files for this remote will be uploaded. If the
Expand All @@ -213,8 +212,9 @@ $ dvc remote modify myremote gdrive_client_secret <client secret>

Note that GDrive remotes are not "trusted" by default. This means that the
[`verify`](/doc/command-reference/remote/modify#available-settings-for-all-remotes)
option is enabled on this type of storage, so DVC recalculates the file hashes
upon download (e.g. `dvc pull`), to make sure that these haven't been modified.
parameter is enabled on this type of storage, so DVC recalculates the file
hashes upon download (e.g. `dvc pull`), to make sure that these haven't been
modified.

> Please note our [Privacy Policy (Google APIs)](/doc/user-guide/privacy).

Expand Down Expand Up @@ -298,8 +298,8 @@ check that you are able to connect both ways to the remote location, with tools
like `ssh` and `sftp` (GNU/Linux).

> Note that your server's SFTP root might differ from its physical root (`/`).
> (On Linux, see the `ChrootDirectory` config option in `/etc/ssh/sshd_config`.)
> In these cases, the path component in the SSH URL (e.g. `/path/to/dir` above)
> (On Linux, see the `ChrootDirectory` setting in `/etc/ssh/sshd_config`.) In
> these cases, the path component in the SSH URL (e.g. `/path/to/dir` above)
> should be specified relative to the SFTP root instead. For example, on some
> Sinology NAS drives, the SFTP root might be in directory `/volume1`, in which
> case you should use path `/path/to/dir` instead of `/volume1/path/to/dir`.
Expand Down Expand Up @@ -372,7 +372,7 @@ $ cat .dvc/config

## Example: Customize an S3 remote

Add an Amazon S3 remote as the _default_ (via `-d` option), and modify its
Add an Amazon S3 remote as the _default_ (via the `-d` option), and modify its
region.

> 💡 Before adding an S3 remote, be sure to
Expand Down
6 changes: 3 additions & 3 deletions public/static/docs/command-reference/remote/default.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ positional arguments:

## Description

You can query/set/replace/unset default remote using options of this command. If
the `name` of the remote is not provided and `--unset` is not specified, this
command returns the name of the default remote.
You can query/set/replace/unset default remote using the options of this
command. If the `name` of the remote is not provided and `--unset` is not
specified, this command returns the name of the default remote.

```dvc
$ dvc remote default myremote
Expand Down
Loading