Skip to content

Commit

Permalink
add bedrock blueprint doc
Browse files Browse the repository at this point in the history
  • Loading branch information
jhart0 committed Oct 4, 2023
1 parent a0c20e6 commit e83abc5
Showing 1 changed file with 106 additions and 0 deletions.
106 changes: 106 additions & 0 deletions docs/remote_inference_blueprints/bedrock_connector_blueprint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
### Bedrock connector blueprint example

1. Add connector endpoint to trusted URLs:

```json
PUT /_cluster/settings
{
"persistent": {
"plugins.ml_commons.trusted_connector_endpoints_regex": [
"^https://bedrock-runtime\\..*[a-z0-9-]\\.amazonaws\\.com/.*$"
]
}
}
```

2. Create connector for Amazon Bedrock:

```json
POST /_plugins/_ml/model_groups/_register
{
"name": "Amazon Bedrock",
"description": "Test connector for Amazon Bedrock",
"version": 1,
"protocol": "aws_sigv4",
"credential": {
"roleArn": "<PLEASE ADD YOUR AWS ROLE ARN HERE>"
},
"parameters": {
"region": "<PLEASE ADD YOUR AWS REGION HERE>",
"service_name": "bedrock"
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"headers": {
"content-type": "application/json"
},
"url": "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke",
"request_body": "{\"prompt\":\"\\n\\nHuman: ${parameters.inputs}\\n\\nAssistant:\",\"max_tokens_to_sample\":300,\"temperature\":0.5,\"top_k\":250,\"top_p\":1,\"stop_sequences\":[\"\\\\n\\\\nHuman:\"]}"
}
]
}
```

Response:
```json
{"connector_id":"SHDj-ooB0wiuGR4S5sM4"}
```

3. Create model group:

```json
POST /_plugins/_ml/model_groups/_register
{
"name": "remote_model_group",
"description": "This is an example description"
}
```

Response:
```json
{"model_group_id":"SXDn-ooB0wiuGR4SrcNN","status":"CREATED"}
```

4. Register model to model group & deploy model:

```json
POST /_plugins/_ml/models/_register
{
"name": "anthropic.claude-v2",
"function_name": "remote",
"model_group_id": "SXDn-ooB0wiuGR4SrcNN",
"description": "test model",
"connector_id": "SHDj-ooB0wiuGR4S5sM4"
}
```

Response:
```json
{"task_id":"SnDo-ooB0wiuGR4SfMNS","status":"CREATED"}
```

```json
GET /_plugins/_ml/tasks/SnDo-ooB0wiuGR4SfMNS
```

```json
POST /_plugins/_ml/models/S3Do-ooB0wiuGR4SfcNv/_deploy
```

6. Test model inference

```json
POST /_plugins/_ml/models/S3Do-ooB0wiuGR4SfcNv/_predict
{
"parameters": {
"inputs": "What is the meaning of life?"
}
}
```

Response:
```json
{"inference_results":[{"output":[{"name":"response","dataAsMap":{"completion":" There is no single, universally agreed upon meaning of life. The meaning of life is subjective and personal. Some common perspectives include finding happiness, purpose, spiritual fulfillment, connecting with others, contributing value, and leaving a positive legacy. Ultimately, the meaning of life is what you make of it.","stop_reason":"stop_sequence"}}]}]}
```

0 comments on commit e83abc5

Please sign in to comment.