Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add page about Studio REST API #4681

Merged
merged 27 commits into from
Oct 13, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5260c54
add page about REST API
aguschin Jul 6, 2023
c031684
link to adding credentials to studio
aguschin Jul 10, 2023
d10fb04
make link relative
aguschin Jul 10, 2023
9d7f82b
fix api handler name
aguschin Jul 10, 2023
0c7c289
add page about studio python api
aguschin Jul 12, 2023
7d1fb17
fix format of request description
aguschin Jul 12, 2023
832efae
Merge branch 'main' into studio/api
aguschin Jul 12, 2023
ba1fe76
note about latest version
aguschin Jul 12, 2023
5059499
Merge branch 'studio/api' of github.com:iterative/dvc.org into studio…
aguschin Jul 12, 2023
3e22bfc
explain monorepo case
aguschin Jul 12, 2023
4c971c6
add link to apis
aguschin Jul 13, 2023
d777c0a
Update content/docs/studio/rest-api.md
aguschin Jul 13, 2023
2f46406
link from dvc api; add with to download_model
aguschin Jul 13, 2023
ef0c38c
Merge branch 'studio/api' of github.com:iterative/dvc.org into studio…
aguschin Jul 13, 2023
0855839
drop studio python api
Jul 24, 2023
7134202
Merge branch 'main' into studio/api
Jul 24, 2023
a4f66a2
minor edits to studio api
Jul 24, 2023
b291eee
add stage
Jul 25, 2023
f68bf78
studio api: add expiration time
Aug 1, 2023
d0139df
Merge branch 'main' into studio/api
Aug 31, 2023
a6493f4
studio api: add python example
Aug 31, 2023
87f9402
Merge branch 'main' into studio/api
Sep 26, 2023
bb54bda
use model registry abbr
Sep 26, 2023
e2aa45a
Merge branch 'main' into studio/api
Sep 29, 2023
b87e2f7
reference studio rest api in artifacts get and reframe around model r…
Sep 29, 2023
166599d
reference studio rest api in api.artifacts_show
Sep 29, 2023
ef27fc3
reword artifacts get fallback
Oct 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion content/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -728,7 +728,9 @@
]
}
]
}
},
"python-api",
"rest-api"
]
},
{
Expand Down
35 changes: 35 additions & 0 deletions content/docs/studio/python-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Python API

The purpose of Studio Python API is to give programmatic access to information
in Studio and executing actions in it.

We provide Studio Python API as a part of DVC python package. To use it, you
need to [install](/doc/install) DVC with `pip` or `conda`.

This reference provides the details about the functions in the `dvc.studio`
module, which can be loaded in any regular way, for example:

```py
import dvc.studio
dberenbaum marked this conversation as resolved.
Show resolved Hide resolved
```

To use this API, you need to
[generate](/doc/studio/user-guide/account-management#studio-access-token) and
[set up](/doc/studio/user-guide/projects-and-experiments/live-metrics-and-plots#set-up-an-access-token)
Studio access token.

## Download model

Download model binaries for a model from Model Registry. Requires the model to
be stored with DVC with s3 or azure remote. Note, that you need to
dberenbaum marked this conversation as resolved.
Show resolved Hide resolved
[set up remote cloud credentials](/doc/studio/user-guide/account-management#cloud-credentials)
for Studio first.

```py
from dvc.studio.model_registry import download_model
path = download_model(
repo="iterative/demo-bank-customer-churn",
model="randomforest-model",
version="v2.0.0"
)
aguschin marked this conversation as resolved.
Show resolved Hide resolved
```
dberenbaum marked this conversation as resolved.
Show resolved Hide resolved
60 changes: 60 additions & 0 deletions content/docs/studio/rest-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# REST API

The purpose of Studio REST API is to give programmatic access to information in
Studio and executing actions in it.

The API is hosted under the `/api` route on the Studio server:
https://studio.iterative.ai/api or https://your-domain/api in case of
[self-hosted Studio](/doc/studio/self-hosting/installation).

To use API, you need to generate
[Studio access token](/doc/studio/user-guide/account-management#studio-access-token).

## Download model

Get signed url to download the model binaries for a model from Model Registry.
Requires the model to be stored with DVC with s3 or azure remote. Note, that you
need to
[set up remote cloud credentials](/doc/studio/user-guide/account-management#cloud-credentials)
for Studio have rights to sign urls.

```yaml
Endpoint: api/model-registry/get-download-uris
HTTP Method: GET
```

### Request

| param | desc | type | required | example value |
| ------- | ------------- | ------ | -------- | ---------------------------------- |
| repo | Git repo URL | string | true | iterative/demo-bank-customer-churn |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| repo | Git repo URL | string | true | iterative/demo-bank-customer-churn |
| repo | Git repo URL | string | true | [email protected]:iterative/demo-bank-customer-churn |

Does this also support a Studio project ID per https://github.com/iterative/studio/pull/6338#issuecomment-1596468450? My feeling is it still might be useful since users have to do a lookup either way and this requires less escaping than a full repo url.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not rn. But we can add it in some future. Adding this as an item to "out of scope of phase I" https://github.com/iterative/studio/issues/5177

| name | Model name | string | true | randomforest-model |
| version | Model version | string | false | v2.0.0 |
dberenbaum marked this conversation as resolved.
Show resolved Hide resolved

If no version specified, the latest one is returned.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amritghimire Is this implemented? When I try it, I get an error:

$ curl "https://studio.iterative.ai/api/model-registry/[email protected]:iterative/demo-bank-customer-churn.git&name=randomforest-model" --header "Authorization:token ***"
{"type":"unsupported_model","reasons":[],"detail":"Model is not stored with DVC"}%


When your model is annotated in non-root `dvc.yaml` file (typical for monorepo
case), model name will be constructed from two parts separated by colon:
`path/to/dvc/yaml:model_name`. For example, take a loot at this
aguschin marked this conversation as resolved.
Show resolved Hide resolved
[model from example-get-started-experiments repo](https://studio.iterative.ai/user/aguschin/models/VtQdva13kMSPsN_N8004aQ==/pool-segmentation/v1.0.1).
Its full name that you need to use in API is `results/train:pool-segmentation`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an example for monorepo. @dberenbaum wdyt?

Copy link
Contributor

@dberenbaum dberenbaum Jul 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it required to include results/train in this case?

Edit: okay, I think I know that the answer is yes, since artifact names are specific to the dvc.yaml file, right? If there are no conflicts, can we omit results/train?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I think we can omit that. We need to implement it on BE side I believe.


| header | desc | example value |
| ------------- | --------------- | ------------- |
| Authorization | Header for auth | token abc123 |

### Response

Response is a json-encoded dict. If the request was successful, keys will be
paths to files inside the repo, and values will be signed urls you can query to
actually download the model.

### Example curl

```sh
$ curl https://studio.iterative.ai/api/model-registry/[email protected]:iterative/demo-bank-customer-churn.git&name=randomforest-model&version=v2.0.0 --header "Authorization:token <TOKEN>"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I try to wget the resulting model, I'm getting a 403 forbidden error.

Also, let's remember that we need to replace this example with one that works in example-get-started-experiments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated link isn't correct because iterative/demo-bank-customer-churn.git project doesn't have creds set up. Adding this as a bug to that umbrella issue.


{
dberenbaum marked this conversation as resolved.
Show resolved Hide resolved
".mlem/model/clf-model": "https://sandbox-datasets-iterative.s3.amazonaws.com/bank-customer-churn/86/bd02376ac675568ba2fac566169ef9?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAU7UXIWDIQFPCO76Q%2F20230706%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230706T134619Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=6807259ddd1f4448ed1e3c5d4503039884f7779381ee556175096b0a884ba1a6"
}
```
3 changes: 2 additions & 1 deletion content/docs/studio/user-guide/account-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ page.
### Studio access token

Iterative Studio uses access tokens to authorize [DVC] and [DVCLive] to send
experiment updates.
experiment updates, and to authenticate you in Studio
[REST API](/doc/studio/rest-api).

In the `Studio access token` section of your [Profile] page, you can generate a
new token as well as regenerate (replace) or delete your access token.
Expand Down