Skip to content

Commit

Permalink
docs: api-reference: add config to open/read/dvcfs/etc (#4628)
Browse files Browse the repository at this point in the history
Fixes #4627
  • Loading branch information
efiop authored Jun 21, 2023
1 parent 7feeb93 commit 493d727
Show file tree
Hide file tree
Showing 6 changed files with 80 additions and 2 deletions.
2 changes: 2 additions & 0 deletions content/docs/api-reference/dvcfilesystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ The optional `rev` argument can be passed to open a filesystem from a certain
Git commit (any [revision](https://git-scm.com/docs/revisions) such as a branch
or a tag name, a commit hash, or an [experiment name]).

The optional `config` argument can be passed through to the DVC project.

[experiment name]: /doc/command-reference/exp/run#-n

## Opening a file
Expand Down
6 changes: 6 additions & 0 deletions content/docs/api-reference/exp_show.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ def exp_show(
num: int = 1,
param_deps: bool = False,
force: bool = False,
config: Optional[dict] = None,
) -> List[Dict]:
```

Expand Down Expand Up @@ -69,6 +70,11 @@ the Git `HEAD`.
- `param_deps` - whether to retrieve only params that are stage dependencies.
_Default_: `False`.

- `config` - [config](/doc/command-reference/config) dictionary to pass to the
DVC project. This is merged with the existing project config and can be used
to, for example, provide credentials to the `remote`. See
[dvc.api.open](/doc/api-reference/open) for examples.

## Example: Create a Pandas DataFrame

The format returned by `dvc.api.exp_show()` can be directly converted to a
Expand Down
6 changes: 6 additions & 0 deletions content/docs/api-reference/metrics_show.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ def metrics_show(
*targets: str,
repo: Optional[str] = None,
rev: Optional[str] = None,
config: Optional[dict] = None,
) -> Dict:
```

Expand Down Expand Up @@ -55,6 +56,11 @@ The function parameters (below) let you restrict what's retrieved.
[experiment](/doc/command-reference/exp) name). If `repo` is not a Git repo,
this option is ignored. _Default_: `None` (current working tree will be used)

- `config` - [config](/doc/command-reference/config) dictionary to pass to the
DVC project. This is merged with the existing project config and can be used
to, for example, provide credentials to the `remote`. See
[dvc.api.open](/doc/api-reference/open) for examples.

## Example: Filter by one or more targets

`targets` can be a single name (string):
Expand Down
53 changes: 52 additions & 1 deletion content/docs/api-reference/open.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ def open(path: str,
rev: str = None,
remote: str = None,
mode: str = "r",
encoding: str = None)
encoding: str = None,
config: dict = None)
```

## Usage
Expand Down Expand Up @@ -84,11 +85,16 @@ call – no _context manager_ involved. Neither function utilizes disc space.
only be used in text mode. Defaults to `"utf-8"`. Mirrors the namesake
parameter in builtin `open()`.

- `config` - [config] dictionary to pass to the DVC project. This is merged with
the existing project config and can be used to, for example, provide
credentials to the `remote`.

[revision]: https://git-scm.com/docs/revisions
[experiment name]: /doc/command-reference/exp/run#-n
[dvc remote]: /doc/user-guide/data-management/remote-storage
[default remote]: /doc/command-reference/remote/default
[codec]: https://docs.python.org/3/library/codecs.html#standard-encodings
[config]: /doc/command-reference/config

## Exceptions

Expand Down Expand Up @@ -199,3 +205,48 @@ import dvc.api
with dvc.api.open('data/nlp/words_ru.txt', encoding='koi8_r') as f:
# ... Process Russian words
```
## Example: Specify credentials for specific remote
See [remote modify](/doc/command-reference/remote/modify) for full list of
remote-specific config options.
```py
import dvc.api
config = {
'remote': {
'myremote': {
'access_key_id': 'mykey',
'secret_access_key': 'mysecretkey',
'session_token': 'mytoken',
},
},
}
with dvc.api.open('data', config=config) as f:
# ... Process data
```
## Example: Change default remote and specify credentials for it
See [remote modify](/doc/command-reference/remote/modify) for full list of
remote-specific config options.
```py
import dvc.api
config = {
'core': {'remote': 'myremote'},
'remote': {
'myremote': {
'access_key_id': 'mykey',
'secret_access_key': 'mysecretkey',
'session_token': 'mytoken',
},
},
}
with dvc.api.open('data', config=config) as f:
# ... Process data
```
6 changes: 6 additions & 0 deletions content/docs/api-reference/params_show.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ def params_show(
repo: Optional[str] = None,
rev: Optional[str] = None,
deps: bool = False,
config: Optional[dict] = None,
) -> Dict:
```

Expand Down Expand Up @@ -67,6 +68,11 @@ The function parameters (below) let you restrict what's retrieved.
- `deps` - whether to retrieve only params that are stage dependencies. Accepts
`True` or `False` (_default_).

- `config` - [config](/doc/command-reference/config) dictionary to pass to the
DVC project. This is merged with the existing project config and can be used
to, for example, provide credentials to the `remote`. See
[dvc.api.open](/doc/api-reference/open) for examples.

## Example: Filter by stage name(s)

`stages` can be a single name (string):
Expand Down
9 changes: 8 additions & 1 deletion content/docs/api-reference/read.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ def read(path: str,
rev: str = None,
remote: str = None,
mode: str = "r",
encoding: str = None)
encoding: str = None,
config: dict = None)
```

## Usage
Expand Down Expand Up @@ -75,11 +76,17 @@ Python's [`open()`] built-in, which is used under the hood.
only be used in text mode. Defaults to `"utf-8"`. Mirrors the namesake
parameter in builtin `open()`.

- `config` - [config] dictionary to pass to the DVC project. This is merged with
the existing project config and can be used to, for example, provide
credentials to the `remote`. See [dvc.api.open](/doc/api-reference/open) for
examples.

[revision]: https://git-scm.com/docs/revisions
[experiment name]: /doc/command-reference/exp/run#-n
[dvc remote]: /doc/user-guide/data-management/remote-storage
[default remote]: /doc/command-reference/remote/default
[codec]: https://docs.python.org/3/library/codecs.html#standard-encodings
[config]: /doc/command-reference/config

## Exceptions

Expand Down

0 comments on commit 493d727

Please sign in to comment.