Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get-started: initial refactoring #1051

Merged
merged 16 commits into from
Mar 20, 2020
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions content/docs/changelog/0.18.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,10 @@ really excited to share the progress with you:

- Commands startup latency reduced 3x

- 📙 **Documentation got better** - a whole new [Get Started](/doc/get-started)
section, new [Use Cases](/doc/use-cases), [User Guide](/doc/user-guide), and
lot of other great stuff you can find here.
- 📙 **Documentation got better** - a whole ne
[Get Started](/doc/tutorials/get-started) section, new
[Use Cases](/doc/use-cases), [User Guide](/doc/user-guide), and lot of other
great stuff you can find here.

- 🙂 **Usability improvements** - DVC interface got more informative and easier
to use:
Expand Down
5 changes: 3 additions & 2 deletions content/docs/changelog/0.35.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ improvements) we have done in the last few months:
all DVC commands that also have `-a` or `--all-branches` (use all Git
branches).

- 📖 The [Get Started](/doc/get-started/agenda) section has been simplified
(e.g. to use tags instead of branches) and extended. We have also prepared a
- 📖 The [Get Started](/doc/tutorials/get-started/agenda) section has been
simplified (e.g. to use tags instead of branches) and extended. We have also
prepared a
[DVC project on GitHub](https://github.com/iterative/example-get-started) that
reflects the sequence of chapters in the “get started” section. You can now
download the whole project and reproduce all the models.
Expand Down
4 changes: 2 additions & 2 deletions content/docs/command-reference/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,8 @@ $ dvc run -f train.dvc \
python train.py
```

> To follow the full example, see
> [Tutorial: Versioning](/doc/tutorials/versioning).
> To follow the full example, see the [Versioning](/doc/tutorials/versioning)
> tutorial.

If instead we use the `--recursive` (`-R`) option, the output looks like this:

Expand Down
4 changes: 2 additions & 2 deletions content/docs/command-reference/checkout.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ be pulled from remote storage using `dvc pull`.

Let's employ a simple <abbr>workspace</abbr> with some data, code, ML models,
pipeline stages, such as the <abbr>DVC project</abbr> created in our
[Get Started](/doc/get-started) section. Then we can see what happens with
`git checkout` and `dvc checkout` as we switch from tag to tag.
[Get Started](/doc/tutorials/get-started) section. Then we can see what happens
with `git checkout` and `dvc checkout` as we switch from tag to tag.

<details>

Expand Down
4 changes: 2 additions & 2 deletions content/docs/command-reference/commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ reproducibility in those cases.

Let's employ a simple <abbr>workspace</abbr> with some data, code, ML models,
pipeline stages, such as the <abbr>DVC project</abbr> created in our
[Get Started](/doc/get-started) section. Then we can see what happens with
`git commit` and `dvc commit` in different situations.
[Get Started](/doc/tutorials/get-started) section. Then we can see what happens
with `git commit` and `dvc commit` in different situations.

<details>

Expand Down
7 changes: 4 additions & 3 deletions content/docs/command-reference/diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ for example when `dvc init` was used with the `--no-scm` option.

## Examples

For these examples we can use the [Get Started](/doc/get-started) project.
For these examples we can use the [Get Started](/doc/tutorials/get-started)
project.

<details>

Expand Down Expand Up @@ -103,8 +104,8 @@ $ dvc diff

Let's checkout the
[3-add-file](https://github.com/iterative/example-get-started/releases/tag/3-add-file)
tag, corresponding to the [Add Files](/doc/get-started/add-files) _Get Started_
chapter, right after we added `data.xml` file with DVC:
tag, corresponding to the [Add Files](/doc/tutorials/get-started/add-files) _Get
Started_ chapter, right after we added `data.xml` file with DVC:

```dvc
$ git checkout 3-add-file
Expand Down
28 changes: 14 additions & 14 deletions content/docs/command-reference/fetch.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,27 +46,27 @@ since files tracked by DVC should already exist in remote storage, but won't be
in the project's <abbr>cache</abbr>. (Refer to `dvc remote` for more information
on DVC remotes.) These necessary data or model files are listed as dependencies
or outputs in a DVC-file (target [stage](/doc/command-reference/run)) so they
are required to [reproduce](/doc/get-started/reproduce) the corresponding
[pipeline](/doc/command-reference/pipeline). (See
are required to [reproduce](/doc/tutorials/get-started/reproduce) the
corresponding [pipeline](/doc/command-reference/pipeline). (See
[DVC-File Format](/doc/user-guide/dvc-file-format) for more information on
dependencies and outputs.)

`dvc fetch` ensures that the files needed for a DVC-file to be
[reproduced](/doc/get-started/reproduce) exist in cache. If no `targets` are
specified, the set of data files to fetch is determined by analyzing all
DVC-files in the current branch, unless `--all-branches` or `--all-tags` is
specified.
[reproduced](/doc/tutorials/get-started/reproduce) exist in cache. If no
`targets` are specified, the set of data files to fetch is determined by
analyzing all DVC-files in the current branch, unless `--all-branches` or
`--all-tags` is specified.

The default remote is used (see `dvc config core.remote`) unless the `--remote`
option is used.

`dvc fetch`, `dvc pull`, and `dvc push` are related in that these 3 commands
perform data synchronization among local and remote storage. The specific way in
which the set of files to push/fetch/pull is determined begins with calculating
file hashes when these are [added](/doc/get-started/add-files) with DVC. File
hashes are stored in the corresponding DVC-files (typically versioned with Git).
Only the hashes specified in DVC-files currently in the workspace are considered
by `dvc fetch` (unless the `-a` or `-T` options are used).
file hashes when these are [added](/doc/tutorials/get-started/add-files) with
DVC. File hashes are stored in the corresponding DVC-files (typically versioned
with Git). Only the hashes specified in DVC-files currently in the workspace are
considered by `dvc fetch` (unless the `-a` or `-T` options are used).

## Options

Expand All @@ -93,8 +93,8 @@ by `dvc fetch` (unless the `-a` or `-T` options are used).
- `-a`, `--all-branches` - fetch cache for all Git branches instead of just the
current workspace. This means DVC may download files needed to reproduce
different versions of a DVC-file
([experiments](/doc/get-started/experiments)), not just the ones currently in
the workspace.
([experiments](/doc/tutorials/get-started/experiments)), not just the ones
currently in the workspace.

- `-T`, `--all-tags` - fetch cache for all Git tags. Similar to `-a` above. Note
that both options can be combined, for example using the `-aT` flag.
Expand All @@ -110,8 +110,8 @@ by `dvc fetch` (unless the `-a` or `-T` options are used).

Let's employ a simple <abbr>workspace</abbr> with some data, code, ML models,
pipeline stages, such as the <abbr>DVC project</abbr> created in our
[Get Started](/doc/get-started) section. Then we can see what happens with
`dvc fetch` as we switch from tag to tag.
[Get Started](/doc/tutorials/get-started) section. Then we can see what happens
with `dvc fetch` as we switch from tag to tag.

<details>

Expand Down
16 changes: 8 additions & 8 deletions content/docs/command-reference/get.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@ https://remote.dvc.org/get-started/66/2eb7f64216d9c2c1088d0a5e2c6951
location to place the artifact within the workspace. Combining these two options
allows us to do something we can't achieve with the regular `git checkout` +
`dvc checkout` process – see for example the
[Get Older Data Version](/doc/get-started/older-versions) chapter of our _Get
Started_.
[Get Older Data Version](/doc/tutorials/get-started/older-versions) chapter of
our _Get Started_.

Let's use the
[get started example repo](https://github.com/iterative/example-get-started)
Expand All @@ -161,12 +161,12 @@ $ git clone https://github.com/iterative/example-get-started
$ cd example-get-started
```

If you are familiar with our [Get Started](/doc/get-started) project (used in
these examples), you may remember that the chapter where we train a first
version of the model corresponds to the the `baseline-experiment` tag in the
repo. Similarly `bigrams-experiment` points to an improved model (trained using
bigrams). What if we wanted to have both versions of the model "checked out" at
the same time? `dvc get` provides an easy way to do this:
If you are familiar with our [Get Started](/doc/tutorials/get-started) project
(used in these examples), you may remember that the chapter where we train a
first version of the model corresponds to the the `baseline-experiment` tag in
the repo. Similarly `bigrams-experiment` points to an improved model (trained
using bigrams). What if we wanted to have both versions of the model "checked
out" at the same time? `dvc get` provides an easy way to do this:

```dvc
$ dvc get . model.pkl --rev baseline-experiment
Expand Down
22 changes: 11 additions & 11 deletions content/docs/command-reference/import-url.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ up to date from the external data source.
## Examples

To illustrate these examples we will be using the <abbr>project</abbr> explained
in the [Get Started](/doc/get-started) section.
in the [Get Started](/doc/tutorials/get-started) section.

<details>

Expand All @@ -132,8 +132,8 @@ in the [Get Started](/doc/get-started) section.
Start by cloning our example repo if you don't already have it. Then move into
the repo and checkout the
[2-remote](https://github.com/iterative/example-get-started/releases/tag/2-remote)
tag, corresponding to the [Configure](/doc/get-started/configure) _Get Started_
chapter:
tag, corresponding to the [Configure](/doc/tutorials/get-started/configure) _Get
Started_ chapter:

```dvc
$ git clone https://github.com/iterative/example-get-started
Expand All @@ -143,14 +143,14 @@ $ mkdir data
```

You should now have a blank workspace, just before the
[Add Files](/doc/get-started/add-files) chapter.
[Add Files](/doc/tutorials/get-started/add-files) chapter.

</details>

## Example: Tracking a remote file

An advanced alternate to [Add Files](/doc/get-started/add-files) chapter of the
_Get Started_ section is to use `dvc import-url`:
An advanced alternate to [Add Files](/doc/tutorials/get-started/add-files)
chapter of the _Get Started_ section is to use `dvc import-url`:

```dvc
$ dvc import-url https://data.dvc.org/get-started/data.xml \
Expand Down Expand Up @@ -196,9 +196,9 @@ regenerating a <abbr>data artifact</abbr> based on the updated data source.
[Pipeline](/doc/command-reference/pipeline) reproduction can be triggered based
on a changed external dependency.

Let's use the [Get Started](/doc/get-started) project again, simulating an
updated external data source. (Remember to prepare the <abbr>workspace</abbr>,
as explained in [Examples](#examples))
Let's use the [Get Started](/doc/tutorials/get-started) project again,
simulating an updated external data source. (Remember to prepare the
<abbr>workspace</abbr>, as explained in [Examples](#examples))

To illustrate this scenario, let's use a local file system directory (external
to the workspace) to simulate a remote data source location. (In real life, the
Expand Down Expand Up @@ -243,8 +243,8 @@ And instead of an `etag` we have an `md5` hash value. We did this so its easy to
edit the data file.

Let's now manually reproduce a
[processing chapter](/doc/get-started/connect-code-and-data) from the _Get
Started_ project. Download the example source code archive and unzip it:
[processing chapter](/doc/tutorials/get-started/connect-code-and-data) from the
_Get Started_ project. Download the example source code archive and unzip it:

```dvc
$ wget https://code.dvc.org/get-started/code.zip
Expand Down
5 changes: 3 additions & 2 deletions content/docs/command-reference/import.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ and data `path` fields, and the `outs` section contains the corresponding local
workspace `path` field. This is enough data about the imported data, to enable
DVC efficiently determining whether the local copy is out of date.

To actually [track the data](https://dvc.org/doc/get-started/add-files),
`git add` (and `git commit`) the import stage.
To actually
[track the data](https://dvc.org/doc/tutorials/get-started/add-files), `git add`
(and `git commit`) the import stage.

Note that import stages are considered always locked, meaning that if you run
`dvc repro`, they won't be updated. Use `dvc update` to update the downloaded
Expand Down
4 changes: 2 additions & 2 deletions content/docs/command-reference/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,8 @@ repos:

Let's employ a simple <abbr>workspace</abbr> with some data, code, ML models,
pipeline stages, such as the <abbr>DVC project</abbr> created in our
[Get Started](/doc/get-started) section. Then we can see what happens with
`dvc install` in different situations.
[Get Started](/doc/tutorials/get-started) section. Then we can see what happens
with `dvc install` in different situations.

<details>

Expand Down
4 changes: 2 additions & 2 deletions content/docs/command-reference/metrics/diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ They're calculated between two commits (hash, branch, tag, or any

Let's employ a simple <abbr>workspace</abbr> with some data, code, ML models,
pipeline stages, such as the <abbr>DVC project</abbr> created in our
[Get Started](/doc/get-started) section. Then we can see what happens with
`dvc install` in different situations.
[Get Started](/doc/tutorials/get-started) section. Then we can see what happens
with `dvc install` in different situations.

<details>

Expand Down
6 changes: 3 additions & 3 deletions content/docs/command-reference/metrics/show.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,6 @@ Examples in [add](/doc/command-reference/metrics/add),
[remove](/doc/command-reference/metrics/remove) cover most of the basic cases
for the `dvc metrics show`.

The [Compare Experiments](/doc/get-started/compare-experiments) chapter of our
_Get Started_ section covers the `-a` option to collect and print a metric file
value across all Git branches.
The [Compare Experiments](/doc/tutorials/get-started/compare-experiments)
chapter of our _Get Started_ section covers the `-a` option to collect and print
a metric file value across all Git branches.
2 changes: 1 addition & 1 deletion content/docs/command-reference/pipeline/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# pipeline

A set of commands to manage [pipelines](/doc/get-started/pipeline):
A set of commands to manage [pipelines](/doc/tutorials/get-started/pipeline):
[show](/doc/command-reference/pipeline/show) and
[list](/doc/command-reference/pipeline/list).

Expand Down
24 changes: 0 additions & 24 deletions content/docs/get-started/index.md

This file was deleted.

29 changes: 29 additions & 0 deletions content/docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# DVC Documentation

Welcome! In here you may find all the guiding material and technical documents
needed to learn about DVC: how to use it, how it works, and where to go for
additional resources.

## Before you start

✅ Please join our [community](/community) or use the [support](/support)
channels if you have any questions or need specific help. We are very responsive
⚡.

✅ Check out our [GitHub repository](https://github.com/iterative/dvc) and give
us a ⭐ if you like the project!

✅ Contribute to DVC [on GitHub](https://github.com/iterative/dvc) or help us
improve this [documentation](https://github.com/iterative/dvc.org) 🙏.

## Main topics

- Learn how to [install](/doc/install) and
[get started](/doc/tutorials/get-started) with DVC in the first sections.
- Explore the main [reasons](/doc/use-cases) to adopt DVC.
- Study the detailed [inner-workings](/doc/user-guide) of DVC, as well as each
one of its [commands](/doc/command-reference) and Python
[functions](/doc/api-reference).

Please choose a topic from the sidebar to the left, or click the `Next` button
below to start from the beginning ↘
Loading