Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Conversation API] initial docs setup #4419

Merged
merged 24 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
7bb54b0
inital docs
hhunter-ms Nov 5, 2024
b0da189
Merge branch 'v1.15' of https://github.com/dapr/docs into issue_4374-1
hhunter-ms Nov 5, 2024
b193c23
Merge branch 'v1.15' into issue_4374-1
hhunter-ms Nov 5, 2024
e1fcc09
anthropic spec
hhunter-ms Nov 5, 2024
c591ceb
update specs
hhunter-ms Nov 5, 2024
b457368
Merge branch 'issue_4374-1' of https://github.com/dapr/docs into issu…
hhunter-ms Nov 5, 2024
0324a42
rename
hhunter-ms Nov 6, 2024
2cffb91
rename
hhunter-ms Nov 6, 2024
3eb82d6
add api reference doc and update overview
hhunter-ms Nov 6, 2024
122d00c
Merge branch 'v1.15' into issue_4374-1
hhunter-ms Nov 19, 2024
a342779
add to api doc
hhunter-ms Nov 19, 2024
3163833
Merge branch 'issue_4374-1' of https://github.com/dapr/docs into issu…
hhunter-ms Nov 19, 2024
3429445
add supported sdks to overview
hhunter-ms Nov 20, 2024
0c802fa
updates from mark and yaron
hhunter-ms Dec 2, 2024
5380bbd
Merge branch 'v1.15' of https://github.com/dapr/docs into issue_4374-1
hhunter-ms Dec 3, 2024
d4452a5
update diagrams to include conversation api
hhunter-ms Dec 3, 2024
a0b86a8
Merge branch 'v1.15' into issue_4374-1
hhunter-ms Dec 4, 2024
6419bc0
add as alpha api
hhunter-ms Dec 5, 2024
b1da0a2
add alpha note to docs
hhunter-ms Dec 5, 2024
cf70616
Merge branch 'issue_4374-1' of https://github.com/dapr/docs into issu…
hhunter-ms Dec 5, 2024
910c476
start how-to
hhunter-ms Dec 5, 2024
3a37846
updates from mark
hhunter-ms Dec 9, 2024
1e45a06
Merge branch 'v1.15' into issue_4374-1
hhunter-ms Dec 9, 2024
43fc32b
Merge branch 'v1.15' into issue_4374-1
hhunter-ms Dec 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions daprdocs/content/en/concepts/building-blocks-concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,4 @@ Dapr provides the following building blocks:
| [**Distributed lock**]({{< ref "distributed-lock-api-overview.md" >}}) | `/v1.0-alpha1/lock` | The distributed lock API enables you to take a lock on a resource so that multiple instances of an application can access the resource without conflicts and provide consistency guarantees.
| [**Cryptography**]({{< ref "cryptography-overview.md" >}}) | `/v1.0-alpha1/crypto` | The Cryptography API enables you to perform cryptographic operations, such as encrypting and decrypting messages, without exposing keys to your application.
| [**Jobs**]({{< ref "jobs-overview.md" >}}) | `/v1.0-alpha1/jobs` | The Jobs API enables you to schedule and orchestrate jobs. Example scenarios include: <ul><li>Schedule batch processing jobs to run every business day</li><li>Schedule various maintenance scripts to perform clean-ups</li><li>Schedule ETL jobs to run at specific times (hourly, daily) to fetch new data, process it, and update the data warehouse with the latest information.</li></ul>
| [**Conversation**]({{< ref "conversation-overview.md" >}}) | `/v1.0-alpha1/conversation` | The Conversation API enables you to supply prompts to converse with different large language models (LLMs) and includes features such as prompt caching and personally identifiable information (PII) obfuscation.
11 changes: 9 additions & 2 deletions daprdocs/content/en/concepts/components-concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,18 @@ Lock components are used as a distributed lock to provide mutually exclusive acc

### Cryptography

[Cryptography]({{< ref cryptography-overview.md >}}) components are used to perform crypographic operations, including encrypting and decrypting messages, without exposing keys to your application.
[Cryptography]({{< ref cryptography-overview.md >}}) components are used to perform cryptographic operations, including encrypting and decrypting messages, without exposing keys to your application.

- [List of supported cryptography components]({{< ref supported-cryptography >}})
- [Cryptography implementations](https://github.com/dapr/components-contrib/tree/master/crypto)

### Conversation

Dapr provides developers a way to abstract interactions with large language models (LLMs) with built-in security and reliability features. Use [conversation]({{< ref conversation-overview.md >}}) components to send prompts to different LLMs, along with the conversation context.

- [List of supported conversation components]({{< ref supported-conversation >}})
- [Conversation implementations](https://github.com/dapr/components-contrib/tree/main/conversation)

### Middleware

Dapr allows custom [middleware]({{< ref "middleware.md" >}}) to be plugged into the HTTP request processing pipeline. Middleware can perform additional actions on an HTTP request (such as authentication, encryption, and message transformation) before the request is routed to the user code, or the response is returned to the client. The middleware components are used with the [service invocation]({{< ref "service-invocation-overview.md" >}}) building block.
Expand All @@ -136,4 +143,4 @@ Dapr allows custom [middleware]({{< ref "middleware.md" >}}) to be plugged into

{{% alert title="Note" color="primary" %}}
hhunter-ms marked this conversation as resolved.
Show resolved Hide resolved
Since pluggable components are not required to be written in Go, they follow a different implementation process than built-in Dapr components. For more information on developing built-in components, read [developing new components](https://github.com/dapr/components-contrib/blob/master/docs/developing-component.md).
hhunter-ms marked this conversation as resolved.
Show resolved Hide resolved
{{% /alert %}}
{{% /alert %}}
1 change: 1 addition & 0 deletions daprdocs/content/en/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ Each of these building block APIs is independent, meaning that you can use any n
| [**Distributed lock**]({{< ref "distributed-lock-api-overview.md" >}}) | The distributed lock API enables your application to acquire a lock for any resource that gives it exclusive access until either the lock is released by the application, or a lease timeout occurs.
| [**Cryptography**]({{< ref "cryptography-overview.md" >}}) | The cryptography API provides an abstraction layer on top of security infrastructure such as key vaults. It contains APIs that allow you to perform cryptographic operations, such as encrypting and decrypting messages, without exposing keys to your applications.
| [**Jobs**]({{< ref "jobs-overview.md" >}}) | The jobs API enables you to schedule jobs at specific times or intervals.
| [**Conversation**]({{< ref "conversation-overview.md" >}}) | The conversation API enables you to abstract the complexities of interacting with large language models (LLMs) and includes features such as prompt caching and personally identifiable information (PII) obfuscation. Using [conversation components]({{< ref supported-conversation >}}), you can supply prompts to converse with different LLMs.

### Cross-cutting APIs

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
type: docs
title: "Conversation"
linkTitle: "Conversation"
weight: 130
description: "Utilize prompts with Large Language Models (LLMs)"
---
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
type: docs
title: "Conversation overview"
linkTitle: "Overview"
weight: 1000
description: "Overview of the conversation API building block"
---

{{% alert title="Alpha" color="primary" %}}
The conversation API is currently in [alpha]({{< ref "certification-lifecycle.md#certification-levels" >}}).
{{% /alert %}}


Using the Dapr conversation API, you can reduce the complexity of interacting with Large Language Models (LLMs) and enable critical performance and security functionality with features like prompt caching and personally identifiable information (PII) data obfuscation.

## Features

hhunter-ms marked this conversation as resolved.
Show resolved Hide resolved
### Prompt caching

To significantly reduce latency and cost, frequent prompts are stored in a cache to be reused, instead of reprocessing the information for every new request. Prompt caching optimizes performance by storing and reusing prompts that are often repeated across multiple API calls.

### Personally identifiable information (PII) obfuscation

The PII obfuscation feature identifies and removes any PII from a conversation response. This feature protects your privacy by eliminating sensitive details like names, addresses, phone numbers, or other details that could be used to identify an individual.

## Try out conversation

### Quickstarts and tutorials

Want to put the Dapr conversation API to the test? Walk through the following quickstart and tutorials to see it in action:

| Quickstart/tutorial | Description |
| ------------------- | ----------- |
| [Conversation quickstart](todo) | . |

### Start using the conversation API directly in your app

Want to skip the quickstarts? Not a problem. You can try out the conversation building block directly in your application. After [Dapr is installed]({{< ref "getting-started/_index.md" >}}), you can begin using the conversation API starting with [the how-to guide]({{< ref howto-conversation-layer.md >}}).

## Next steps

- [How-To: Converse with an LLM using the conversation API]({{< ref howto-conversation-layer.md >}})
- [Conversation API components]({{< ref supported-conversation >}})
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
type: docs
title: "How-To: Converse with an LLM using the conversation API"
linkTitle: "How-To: Converse"
weight: 2000
description: "Learn how to abstract the complexities of interacting with large language models"
---

{{% alert title="Alpha" color="primary" %}}
The conversation API is currently in [alpha]({{< ref "certification-lifecycle.md#certification-levels" >}}).
{{% /alert %}}

Let's get started using the [conversation API]({{< ref conversation-overview.md >}}). In this guide, you'll learn how to:

- Set up one of the available Dapr components (echo) that work with the conversation API.
- Add the conversation client to your application.

## Set up the conversation component

Create a new configuration file called `conversation.yaml` and save to a components or config sub-folder in your application directory.

Select your [preferred conversation component spec]({{< ref supported-conversation >}}) for your `conversation.yaml` file.

For this scenario, we use a simple echo component.

```yml
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: echo
spec:
type: conversation.echo
version: v1
```

## Connect the conversation client


{{< tabs ".NET" "Go" "Rust" >}}


<!-- .NET -->
{{% codetab %}}

```dotnet
todo
```

{{% /codetab %}}

<!-- Go -->
{{% codetab %}}

```go
package main

import (
"context"
"fmt"
dapr "github.com/dapr/go-sdk/client"
"log"
)

func main() {
client, err := dapr.NewClient()
if err != nil {
panic(err)
}

input := dapr.ConversationInput{
Message: "hello world",
// Role: nil, // Optional
// ScrubPII: nil, // Optional
}

fmt.Printf("conversation input: %s\n", input.Message)

var conversationComponent = "echo"

request := dapr.NewConversationRequest(conversationComponent, []dapr.ConversationInput{input})

resp, err := client.ConverseAlpha1(context.Background(), request)
if err != nil {
log.Fatalf("err: %v", err)
}

fmt.Printf("conversation output: %s\n", resp.Outputs[0].Result)
}
```

{{% /codetab %}}

<!-- Rust -->
{{% codetab %}}

```rust
use dapr::client::{ConversationInputBuilder, ConversationRequestBuilder};
use std::thread;
use std::time::Duration;

type DaprClient = dapr::Client<dapr::client::TonicClient>;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Sleep to allow for the server to become available
thread::sleep(Duration::from_secs(5));

// Set the Dapr address
let address = "https://127.0.0.1".to_string();

let mut client = DaprClient::connect(address).await?;

let input = ConversationInputBuilder::new("hello world").build();

let conversation_component = "echo";

let request =
ConversationRequestBuilder::new(conversation_component, vec![input.clone()]).build();

println!("conversation input: {:?}", input.message);

let response = client.converse_alpha1(request).await?;

println!("conversation output: {:?}", response.outputs[0].result);
Ok(())
}
```

{{% /codetab %}}

{{< /tabs >}}


## Next steps

- [Conversation API reference guide]({{< ref conversation_api.md >}})
- [Available conversation components]({{< ref supported-conversation >}})
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Want to put workflows to the test? Walk through the following quickstart and tut
| Quickstart/tutorial | Description |
| ------------------- | ----------- |
| [Workflow quickstart]({{< ref workflow-quickstart.md >}}) | Run a workflow application with four workflow activities to see Dapr Workflow in action |
| [Workflow Python SDK example](https://github.com/dapr/python-sdk/tree/master/examples/demo_workflow) | Learn how to create a Dapr Workflow and invoke it using the Python `DaprClient` package. |
| [Workflow Python SDK example](https://github.com/dapr/python-sdk/tree/master/examples/demo_workflow) | Learn how to create a Dapr Workflow and invoke it using the Python `dapr-ext-workflow` package. |
| [Workflow JavaScript SDK example](https://github.com/dapr/js-sdk/tree/main/examples/workflow) | Learn how to create a Dapr Workflow and invoke it using the JavaScript SDK. |
| [Workflow .NET SDK example](https://github.com/dapr/dotnet-sdk/tree/master/examples/Workflow) | Learn how to create a Dapr Workflow and invoke it using ASP.NET Core web APIs. |
| [Workflow Java SDK example](https://github.com/dapr/java-sdk/tree/master/examples/src/main/java/io/dapr/examples/workflows) | Learn how to create a Dapr Workflow and invoke it using the Java `io.dapr.workflows` package. |
Expand Down
4 changes: 3 additions & 1 deletion daprdocs/content/en/operations/support/alpha-beta-apis.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ description: "List of current alpha and beta APIs"
| Bulk Publish | [Bulk publish proto](https://github.com/dapr/dapr/blob/5aba3c9aa4ea9b3f388df125f9c66495b43c5c9e/dapr/proto/runtime/v1/dapr.proto#L59) | `v1.0-alpha1/publish/bulk` | The bulk publish API allows you to publish multiple messages to a topic in a single request. | [Bulk Publish and Subscribe API]({{< ref "pubsub-bulk.md" >}}) | v1.10 |
| Bulk Subscribe | [Bulk subscribe proto](https://github.com/dapr/dapr/blob/5aba3c9aa4ea9b3f388df125f9c66495b43c5c9e/dapr/proto/runtime/v1/appcallback.proto#L57) | N/A | The bulk subscribe application callback receives multiple messages from a topic in a single call. | [Bulk Publish and Subscribe API]({{< ref "pubsub-bulk.md" >}}) | v1.10 |
| Cryptography | [Crypto proto](https://github.com/dapr/dapr/blob/5aba3c9aa4ea9b3f388df125f9c66495b43c5c9e/dapr/proto/runtime/v1/dapr.proto#L118) | `v1.0-alpha1/crypto` | The cryptography API enables you to perform **high level** cryptography operations for encrypting and decrypting messages. | [Cryptography API]({{< ref "cryptography-overview.md" >}}) | v1.11 |
| Jobs | [Jobs proto](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto#L198-L204) | `v1.0-alpha1/jobs` | The jobs API enables you to schedule and orchestrate jobs. | [Jobs API]({{< ref "jobs-overview.md" >}}) | v1.14 |
| Jobs | [Jobs proto](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto#L212-219) | `v1.0-alpha1/jobs` | The jobs API enables you to schedule and orchestrate jobs. | [Jobs API]({{< ref "jobs-overview.md" >}}) | v1.14 |
| Conversation | [Conversation proto](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto#L221-222) | `v1.0-alpha1/conversation` | Converse between different large language models using the conversation API. | [Conversation API]({{< ref "conversation-overview.md" >}}) | v1.15 |


## Beta APIs

Expand Down
74 changes: 74 additions & 0 deletions daprdocs/content/en/reference/api/conversation_api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
type: docs
title: "Conversation API reference"
linkTitle: "Conversation API"
description: "Detailed documentation on the conversation API"
weight: 1400
---

{{% alert title="Alpha" color="primary" %}}
The conversation API is currently in [alpha]({{< ref "certification-lifecycle.md#certification-levels" >}}).
{{% /alert %}}

Dapr provides an API to interact with Large Language Models (LLMs) and enables critical performance and security functionality with features like prompt caching and PII data obfuscation.

## Converse

This endpoint lets you converse with LLMs.

```
POST /v1.0-alpha1/conversation/<llm-name>/converse
```

### URL parameters

| Parameter | Description |
| --------- | ----------- |
| `llm-name` | The name of the LLM component. [See a list of all available conversation components.]({{< ref supported-conversation >}})

### Request body

| Field | Description |
| --------- | ----------- |
| `conversationContext` | |
| `inputs` | |
| `parameters` | |


hhunter-ms marked this conversation as resolved.
Show resolved Hide resolved
### Request content

```json
REQUEST = {
"inputs": ["what is Dapr", "Why use Dapr"],
"parameters": {},
}
```

### HTTP response codes

Code | Description
---- | -----------
`202` | Accepted
`400` | Request was malformed
`500` | Request formatted correctly, error in dapr code or underlying component

### Response content

```json
RESPONSE = {
"outputs": {
{
"result": "Dapr is distribution application runtime ...",
"parameters": {},
},
{
"result": "Dapr can help developers ...",
"parameters": {},
}
},
}
```

## Next steps

[Conversation API overview]({{< ref conversation-overview.md >}})
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
type: docs
title: "Conversation component specs"
linkTitle: "Conversation"
weight: 9000
description: The supported conversation components that interface with Dapr
no_list: true
---

{{< partial "components/description.html" >}}

{{< partial "components/conversation.html" >}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
type: docs
title: "Anthropic"
linkTitle: "Anthropic"
description: Detailed information on the Anthropic conversation component
---

## Component format

A Dapr `conversation.yaml` component file has the following structure:

```yaml
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: anthropic
spec:
type: conversation.anthropic
metadata:
- name: key
value: "mykey"
- name: model
value: claude-3-5-sonnet-20240620
- name: cacheTTL
value: 10m
```

{{% alert title="Warning" color="warning" %}}
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets, as described [here]({{< ref component-secrets.md >}}).
{{% /alert %}}

## Spec metadata fields

| Field | Required | Details | Example |
|--------------------|:--------:|---------|---------|
| `key` | Y | API key for Anthropic. | `"mykey"` |
| `model` | N | The Anthropic LLM to use. Defaults to `claude-3-5-sonnet-20240620` | `claude-3-5-sonnet-20240620` |
| `cacheTTL` | N | A time-to-live value for a prompt cache to expire. Uses Golang duration format. | `10m` |

## Related links

- [Conversation API overview]({{< ref conversation-overview.md >}})
Loading
Loading