Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: create docs #908

Merged
merged 109 commits into from
Mar 8, 2020
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
d56f2d2
api: create index and structure
jorgeorpinel Jan 8, 2020
d53833f
api ref: add summon page, improve format of all pages
jorgeorpinel Jan 8, 2020
aba4954
api ref: complete pages per initial desc.
jorgeorpinel Jan 8, 2020
0f5a3bb
api: better formatting, added links, and examples
jorgeorpinel Jan 9, 2020
701aeb7
api ref: reorder pages, refine descriptions and options, add examples
jorgeorpinel Jan 9, 2020
6a18d61
api: note that env mgr install is required
jorgeorpinel Jan 9, 2020
a3023be
api ref: typo in index
jorgeorpinel Jan 9, 2020
09c76c5
cmd ref: improve `path` description in get and import
jorgeorpinel Jan 9, 2020
8c14cbd
api: improve path/name and repo param descriptions
jorgeorpinel Jan 9, 2020
97c3972
api: improve `repo` param desc
jorgeorpinel Jan 9, 2020
6894e77
api: add assumed defaults to all params in all functions
jorgeorpinel Jan 9, 2020
711838d
api: oops, forgot about `repo` param's default :B
jorgeorpinel Jan 9, 2020
68a3dd0
api: comlpete get_url desc and real life example
jorgeorpinel Jan 9, 2020
95fa070
use-cases: link from data-registries to api ref
jorgeorpinel Jan 10, 2020
09db290
doc: add some basic links between API and cmd refs
jorgeorpinel Jan 10, 2020
bb05e9c
api: better desc. and example explanation in get_url
jorgeorpinel Jan 11, 2020
58e4bb5
api: improve index desc (again?)
jorgeorpinel Jan 11, 2020
08fe226
api: update get_url example to use iterative/dataset-registry repo
jorgeorpinel Jan 12, 2020
c207152
api: complete base examples for get_url, open, and read (they may nee…
jorgeorpinel Jan 12, 2020
4a50af9
install: add notes about installing as a Python lib
jorgeorpinel Jan 14, 2020
89734fd
api ref: add note about file existence in get_url, and related updates
jorgeorpinel Jan 15, 2020
c094233
api: copy edits
jorgeorpinel Jan 16, 2020
8b133ca
api: add notes about possible errors in function arguments
jorgeorpinel Jan 16, 2020
eec0848
api: add return types to first 3 functions
jorgeorpinel Jan 16, 2020
9b58c3f
api: add list of remotes you can strem from for open fn
jorgeorpinel Jan 16, 2020
af3ee37
Merge branch 'master' into api
jorgeorpinel Jan 16, 2020
eeb318c
Merge branch 'master' into api
jorgeorpinel Jan 19, 2020
3e0b909
api ref: remove `summon()` page
jorgeorpinel Jan 19, 2020
9cf939f
api ref: add full modue path to exceptions mentioned so far
jorgeorpinel Jan 21, 2020
59bb2f2
api: fix term "environment manager"
jorgeorpinel Jan 21, 2020
e340321
api ref: separate short and long desc, similar to cmd ref
jorgeorpinel Jan 21, 2020
cc4a7a8
api ref: small language refinements
jorgeorpinel Jan 21, 2020
24f2d67
api ref: add note about `.dir` cache files in get_url
jorgeorpinel Jan 23, 2020
e26ef15
Merge branch 'master' into api
jorgeorpinel Feb 7, 2020
026eaa2
api: correct exception path in get_url
jorgeorpinel Feb 7, 2020
3815a09
api: standard indentation and arg usage in all examples, and
jorgeorpinel Feb 7, 2020
382b195
api: improve last open() example
jorgeorpinel Feb 8, 2020
013f5df
api: std use of single vs double quotes and add mode='rb' in read() e…
jorgeorpinel Feb 8, 2020
e9340ce
api: update index and install section
jorgeorpinel Feb 8, 2020
0a725e8
cmd ref: refactor and simplify notes to emphasize those linking to ap…
jorgeorpinel Feb 8, 2020
d696063
api ref: rewrite get_url intro
jorgeorpinel Feb 8, 2020
f13b311
api ref: simplify note about get_url not checking for file/dir existence
jorgeorpinel Feb 8, 2020
73adff5
api ref: update note about directory JSON .dir files in get_url
jorgeorpinel Feb 9, 2020
6ebe371
api ref: std. param lang style
jorgeorpinel Feb 9, 2020
a921b40
api ref: simplify and improve basic param descs
jorgeorpinel Feb 9, 2020
a6f9eec
api ref: improvements to repo param and get_url example
jorgeorpinel Feb 9, 2020
c1eb598
api ref: further explain URL construction in get_url example
jorgeorpinel Feb 12, 2020
4f42b88
api ref: simplify api index
jorgeorpinel Feb 12, 2020
ffc0117
Merge branch 'master' into api
jorgeorpinel Feb 15, 2020
8208fd9
api: add link to dvcx repo
jorgeorpinel Feb 17, 2020
910d319
Merge branch 'master' into api
jorgeorpinel Feb 17, 2020
1d5d3a9
api: open() and read() support Git-tracked files
jorgeorpinel Feb 17, 2020
a11465e
links: fix link-check for api docs
jorgeorpinel Feb 17, 2020
f86afde
api: typo
jorgeorpinel Feb 17, 2020
979d70c
api: Signature -> definition section in all fns
jorgeorpinel Feb 18, 2020
099dc4e
api: copy edits and term artifact -> file or dir in get_url
jorgeorpinel Feb 18, 2020
9e21825
api: term artifact -> data since open() and read() don't support dirs
jorgeorpinel Feb 18, 2020
4eb85b3
api: typo
jorgeorpinel Feb 18, 2020
b6cd85e
api: improvements to fn params
jorgeorpinel Feb 18, 2020
11fb55e
api: updates to repo param
jorgeorpinel Feb 18, 2020
be2316f
install: reword link to api ref
jorgeorpinel Feb 19, 2020
c94610d
term: GitHub URLs -> hosted on GitHub
jorgeorpinel Feb 19, 2020
9c52cd8
api: add link to read() from open() desc.
jorgeorpinel Feb 19, 2020
4219494
Merge branch 'master' into api
jorgeorpinel Feb 25, 2020
942d81d
api: remove word "directly" fom exception lists
jorgeorpinel Feb 25, 2020
e76bd5f
api: add basic usage sections
jorgeorpinel Feb 25, 2020
2b8b4a7
api: improve model open() example
jorgeorpinel Feb 25, 2020
aa88cea
api: fix typos and remove lines between import stmts
jorgeorpinel Feb 25, 2020
dc93bb8
Merge branch 'api' of github.com:iterative/dvc.org into api
jorgeorpinel Feb 25, 2020
f66826c
api: fix closing parentheses in example
jorgeorpinel Feb 25, 2020
8b3929c
api: remove link to dvcx
jorgeorpinel Feb 25, 2020
3ec79a0
api: updates to open()
jorgeorpinel Feb 25, 2020
fcdf0c4
api: improve open() examples
jorgeorpinel Feb 27, 2020
e5b52ae
api: improve list of 3rd party lib examples in get_url
jorgeorpinel Feb 27, 2020
7f2981f
api ref: compact intro/signature before loner descs
jorgeorpinel Feb 27, 2020
03d4e72
api ref: improve dvc.api.open() desc similar to open() builtin
jorgeorpinel Feb 27, 2020
6ccfc80
api ref: updates to get_url
jorgeorpinel Feb 27, 2020
418651d
api ref: impros to open()
jorgeorpinel Feb 27, 2020
7161e8c
api ref: more impros to open()
jorgeorpinel Feb 27, 2020
a229803
api ref: remove term "source" from params
jorgeorpinel Feb 27, 2020
c43e704
api ref: better wording in path option
jorgeorpinel Feb 27, 2020
5acaffd
api ref: explain mode and encoding options (open/read())
jorgeorpinel Feb 27, 2020
8840338
api ref: typo project->cache
jorgeorpinel Feb 27, 2020
2e9dc3d
api ref: move default param behavior to fn descriptions
jorgeorpinel Feb 27, 2020
35674e3
api ref: add read() snippet in open() example
jorgeorpinel Feb 27, 2020
42e8563
api ref: add read() example explanation and fix link check
jorgeorpinel Feb 28, 2020
ed80616
api ref: name examples
jorgeorpinel Feb 28, 2020
c47b366
api ref: move the default arguments/behavior back to params
jorgeorpinel Feb 29, 2020
a1a6b34
api ref: use simple language in example titles
jorgeorpinel Feb 29, 2020
a251259
api ref: merge local open() example into --rev example
jorgeorpinel Feb 29, 2020
af551be
api ref: change motivation of `remote` arg example in open()
jorgeorpinel Feb 29, 2020
7277e11
api ref: unserialize -> deserialize
jorgeorpinel Feb 29, 2020
4f04a62
api ref: some last refinements 9on this feedback round)
jorgeorpinel Feb 29, 2020
925f520
api ref: rewrite intro blocks for simplicity, use type hints
jorgeorpinel Feb 29, 2020
7105df6
api ref: improve print output of code samples
jorgeorpinel Mar 1, 2020
a02b178
api ref: few text edits to match to core repo docstrings
jorgeorpinel Mar 2, 2020
466694c
api ref: correct docs about UrlNotDvcRepoError – it only exists in ge…
jorgeorpinel Mar 2, 2020
951742d
suggest some minor things to API
shcheklein Mar 3, 2020
410622f
Update public/static/docs/api-reference/get_url.md
shcheklein Mar 3, 2020
fcbdefd
Merge pull request #1032 from iterative/api-suggestions
jorgeorpinel Mar 4, 2020
8089423
api ref: address remaining feedback from PR #1032
jorgeorpinel Mar 4, 2020
e440284
typo
jorgeorpinel Mar 4, 2020
4ee9335
api ref: apply get_url improvements to open and read fns
jorgeorpinel Mar 4, 2020
6cd45b0
api ref: add info. about types returned/generated to open and read
jorgeorpinel Mar 5, 2020
ce5b42a
api ref: minor changes to open
jorgeorpinel Mar 5, 2020
f245cc8
Merge branch 'master' into api
jorgeorpinel Mar 8, 2020
6bd2740
api ref: address another round of feedback on open fn
jorgeorpinel Mar 8, 2020
90cb882
api ref: small change to reaf fn example title
jorgeorpinel Mar 8, 2020
7733fea
api ref: a few last improvements it seems
jorgeorpinel Mar 8, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions public/static/docs/api-reference/get_url.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# dvc.api.get_url()

Returns the full URL to the <abbr>data artifact</abbr> specified by its `path`
in a `repo`.

## Signature

```py
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
get_url(path, repo=None, rev=None, remote=None)
```

## Parameters

- `path` - path to the target artifact relative to the repository's root

- `repo` - path or Git URL of a DVC repository

- `rev` - (optional)
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
[Git revision](https://git-scm.com/book/en/v2/Git-Internals-Git-References)
(such as a branch name, a tag, or a commit hash). This only works with `repo`
URLs.
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

- `remote` - (optional) name of the [DVC remote](/doc/command-reference/remote)
shcheklein marked this conversation as resolved.
Show resolved Hide resolved
to fetch the target artifact from
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

## Example

```py
import dvc.api

resource_url = dvc.api.get_url("data/prepared.tsv", repo="https://github.com/my-org/my-repo.git", remote="my-s3")
```
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
28 changes: 28 additions & 0 deletions public/static/docs/api-reference/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Python API
shcheklein marked this conversation as resolved.
Show resolved Hide resolved

When you [install](/doc/install) DVC with an environment manager like `pip` or
`conda`, the `dvc` package becomes available to the corresponding `python`
interpreter. While most of the package implements our
[command-line tool](/doc/command-reference), we wrote the `dvc.api` module to
expose special functions you can use in your Python source code.

> We **strongly** recommend having `dvc` in requirements or setup file for your
> Python project, and installing it via and env manager such as `pip`.

To import the API, use:

```py
import dvc.api
```

This reference provides the details about our API functions, their purpose,
usage, and examples. Please note that they also have inline documentation, which
you can see in the module's
[source code](https://github.com/iterative/dvc/blob/master/dvc/api.py).

> Please don't hesitate in sending a feature request
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
> [on GitHub](https://github.com/iterative/dvc.org/issues) with ideas of other
> functions we could add to the Python API.

Please choose from the navigation sidebar to the left, or click the `Next`
button below ↘
58 changes: 58 additions & 0 deletions public/static/docs/api-reference/open.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# dvc.api.open()

Opens a file <abbr>artifact</abbr> as a
[file object](https://docs.python.org/3.7/glossary.html#term-file-object). May
only be used as
[context manager](https://www.python.org/dev/peps/pep-0343/#context-managers-in-the-standard-library).

## Signature

```py
open(path, repo=None, rev=None, remote=None, mode="r", encoding=None)
```

## Parameters

- `path` -
[path](https://docs.python.org/3.7/glossary.html#term-path-like-object) to the
target artifact relative to the repository's root

- `repo` - path or Git URL of a DVC repository

- `rev` - (optional)
[Git revision](https://git-scm.com/book/en/v2/Git-Internals-Git-References)
(such as a branch name, a tag, or a commit hash). This only works with `repo`
URLs.

- `remote` - (optional) name of the [DVC remote](/doc/command-reference/remote)
to fetch the target artifact from

- `mode` - (optional) mirrors the namesake parameter in builtin
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
[`open()`](https://docs.python.org/3.7/library/functions.html#open). Defaults
to `"r"` (read).

- `encoding` - (optional) used to decode contents to a string. Mirrors the
namesake parameter in builtin `open()`.

## Example: open from a DVC remote

> See
> [PEP 343 -- The "with" Statement](https://www.python.org/dev/peps/pep-0343/)

```py
with dvc.api.open("data/raw.csv", remote="my-s3", encoding="utf-8") as f:
for line in f:
process(line)
```

## Example: open from a DVC repository

```py
import csv
import dvc.api

with dvc.api.open("dataset/", repo="https://github.com/my-org/my-repo.git") as f:
reader = csv.reader(f)
for row in reader:
# ...
```
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
49 changes: 49 additions & 0 deletions public/static/docs/api-reference/read.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# dvc.api.read()

Returns the contents of a file <abbr>artifact</abbr> as a bytes object or as a
string.

> Wrapper for [`dvc.api.open()`](/doc/api-reference/open) that returns the
> results of the file object's
> [`read()`](https://docs.python.org/3.7/tutorial/inputoutput.html#methods-of-file-objects)
> method.

## Signature

```py
read(path, repo=None, rev=None, remote=None, mode="r", encoding=None)
```

## Parameters

- `path` -
[path](https://docs.python.org/3.7/glossary.html#term-path-like-object) to the
target artifact relative to the repository's root

- `repo` - path or Git URL of a DVC repository

- `rev` - (optional)
[Git revision](https://git-scm.com/book/en/v2/Git-Internals-Git-References)
(such as a branch name, a tag, or a commit hash). This only works with `repo`
URLs.

- `remote` - (optional) name of the [DVC remote](/doc/command-reference/remote)
to fetch the target artifact from

- `mode` - (optional) mirrors the namesake parameter in builtin
[`open()`](https://docs.python.org/3.7/library/functions.html#open). Defaults
to `"r"` (read).

- `encoding` - (optional) used to decode contents to a string. Mirrors the
namesake parameter in builtin `open()`.

## Example

```py
import pickle
import dvc.api

model = pickle.loads(
dvc.api.read("model.pkl", repo="https://github.com/my-org/my-repo.git")
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
)
```
31 changes: 31 additions & 0 deletions public/static/docs/api-reference/summon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# dvc.api.summon()

Instantiate an object, described in a _summon file_.
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

## Signature

```py
def summon(
name,
repo=None,
rev=None,
summon_file="dvcsummon.yaml",
args=None
)
```

## Parameters

- `name` - object to summon

- `repo` - path or Git URL of a DVC repository

- `rev` - (optional)
[Git revision](https://git-scm.com/book/en/v2/Git-Internals-Git-References)
(such as a branch name, a tag, or a commit hash). This only works with `repo`
URLs.

- `summon_file` - YAML file describing the object in question. Defaults to
`dvcsummon.yaml`.

- `args` - arguments to pass onto the object, if any
23 changes: 23 additions & 0 deletions public/static/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,29 @@
}
]
},
{
"slug": "api-reference",
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
"label": "Python API Reference",
"source": "api-reference/index.md",
"children": [
{
"slug": "get_url",
"label": "get_url()"
},
{
"slug": "open",
"label": "open()"
},
{
"slug": "read",
"label": "read()"
},
{
"slug": "summon",
"label": "summon()"
}
]
},
{
"slug": "understanding-dvc",
"label": "Understanding DVC",
Expand Down