-
Notifications
You must be signed in to change notification settings - Fork 12
Sagemaker docs #182
Sagemaker docs #182
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -4,7 +4,114 @@ Implements MlemEnv, MlemDeployment and DeployState to work with AWS SageMaker | |||||
|
||||||
## Description | ||||||
|
||||||
**TODO** | ||||||
MLEM SageMaker allow you to deploy MLEM models to AWS SageMaker. You can learn | ||||||
more about SageMaker | ||||||
[here](https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html). | ||||||
|
||||||
### Preparing infrastructre | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. typo., could you please run Grammarly or something on top of the text? (I see other typos, English syntax mistakes, etc) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I use Code Spell Checker on VS Code and it helps fix typos, which is often ~ 80% of the problem. I've just recently started trying Grammarly too so I'm not yet sure whether to recommend it. But it can certainly help improve syntax marginally at least for some people. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, I'd call this section
Suggested change
so it's more general (applicable to all pages) |
||||||
|
||||||
To be able to deploy to SageMaker you need to do some AWS configuration. This is | ||||||
not MLEM specific requirements, rather it's needed for any SageMaker | ||||||
interaction. | ||||||
|
||||||
Here is the list: | ||||||
|
||||||
- AWS User Credentials | ||||||
- SageMaker access for this user (policy | ||||||
`arn:aws:iam::aws:policy/AmazonSageMakerFullAccess`) | ||||||
- ECR access for this user (policy | ||||||
`arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess`) | ||||||
- AWS IAM Role with SageMaker access | ||||||
- S3 Access | ||||||
|
||||||
You can configure those manually or use existing ones. You can also use | ||||||
[terraform](https://www.terraform.io/) with | ||||||
[this template](https://github.com/iterative/mlem/tree/main/mlem/contrib/sagemaker/mlem_sagemaker.tf) | ||||||
and | ||||||
[helper script](https://github.com/iterative/mlem/tree/main/mlem/contrib/sagemaker/env_setup.py) | ||||||
(terraform needs to be installed). | ||||||
|
||||||
> This script is not part of MLEM public API, so you'll need to run it manually | ||||||
> like this | ||||||
|
||||||
```python | ||||||
from mlem.contrib.sagemaker.env_setup import sagemaker_terraform | ||||||
|
||||||
sagemaker_terraform(export_secret="creds.csv") | ||||||
``` | ||||||
|
||||||
It's recommended to use [aws cli](https://aws.amazon.com/cli/) with separate | ||||||
profile configured for MLEM. You can also provide credentials with | ||||||
[AWS environment variables](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html). | ||||||
|
||||||
### Configuring and running deployment | ||||||
|
||||||
[SageMaker Environment](#class-sagemakerenv) declaration can be used to hold | ||||||
your SageMaker configuration. | ||||||
|
||||||
TODO | ||||||
|
||||||
You can also pre-declare [SageMaker Deployment](#class-sagemakerdeployment) | ||||||
itself. | ||||||
|
||||||
TODO | ||||||
|
||||||
To run deployment, run | ||||||
|
||||||
```cli | ||||||
$ mlem deployment run ... --model <path> | ||||||
``` | ||||||
|
||||||
### What happens internally | ||||||
|
||||||
Once you run our this sweet `mlem deployment run ...` command, a number of | ||||||
things will happen. | ||||||
|
||||||
1. If you did not specify pre-built image, a new docker image will be built. It | ||||||
will include all model's requirements. This image will be pushed to | ||||||
configured ECR repository. | ||||||
2. Model is packaged and uploaded to configured s3 bucket as per | ||||||
[this doc](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-deployment.html#realtime-endpoints-deployment-create-model) | ||||||
3. Enpoint Configuration is created as per | ||||||
[this doc](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-deployment.html#realtime-endpoints-deployment-create-endpoint-config) | ||||||
4. Model is deployed thus creating a SageMaker Endpoint | ||||||
|
||||||
After this command exits, however it can take some time on SageMakers side to | ||||||
actually run VMs with your model. You can check status with | ||||||
|
||||||
```cli | ||||||
$ mlem deployment status ... | ||||||
``` | ||||||
|
||||||
or block until model is ready with | ||||||
|
||||||
```cli | ||||||
$ mlem deployment wait ... -i starting | ||||||
``` | ||||||
|
||||||
### Making requests | ||||||
|
||||||
MLEM SageMaker deployments are fully compatible with SageMaker | ||||||
[InvokeEndpoint](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html) | ||||||
API, however it's a lot easier to use | ||||||
[MLEM SagemakerClient](#class-sagemakerclient). To obtain one, just call | ||||||
`get_client` method on your deployment object. | ||||||
|
||||||
```python | ||||||
from mlem.api import load_meta | ||||||
|
||||||
service = load_meta("...") | ||||||
client = service.get_client() | ||||||
``` | ||||||
|
||||||
You can then use this `client` instance to invoke your model as if it is local. | ||||||
|
||||||
```python | ||||||
data = ... # pd.DataFrame or whatever model.predict accepts | ||||||
preds = client.predict(data) | ||||||
``` | ||||||
|
||||||
> MLEM do not support batch invocations. We will add support for them soon | ||||||
|
||||||
## Requirements | ||||||
|
||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be the intro sentence TBH (before Description or maybe we don't even need a
## Description
header (maybe thus move this one change to #179?).We can condense it even more too:
One question on terminology though: is "MLEM model" the same as "MLEM model object"? Also sounds a lot like "ML model". We have to think this through so it's clear, I think. Maybe phrase it as
deploy Ml models from MLEM to