Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenAI Files for OpenAI assistant #3228

Merged
merged 16 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions docs/docs/modules/agents/agent_types/openai_file.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# OpenAI File

:::info
The [OpenAI File is used in OpenAI Assistant API](https://platform.openai.com/docs/assistants/how-it-works/creating-assistants) is still in beta.
[API reference](https://platform.openai.com/docs/api-reference/files/object)
:::

Files are used to upload documents that can be used with features like Assistants and Fine-tuning.

We've implemented the File API in LangChain with create and delete.

## Creating an File

```typescript
import * as fs from "fs";
import * as path from "path";
import { OpenAIFiles } from "langchain/experimental/openai_files";

const file = await OpenAIFiles.create({
file: fs.createReadStream(path.resolve(__dirname, `./test.txt`)),
purpose: "assistants",
});
/**
* Output
{
"id": "file-BK7bzQj3FfZFXr7DbL6xJwfo",
"object": "file",
"bytes": 120000,
"created_at": 1677610602,
"filename": "salesOverview.pdf",
"purpose": "assistants",
}
*/
```

## Use File in AI Assistant

```typescript
import * as fs from "fs";
import * as path from "path";
import { OpenAIAssistants } from "langchain/experimental/openai_assistants";
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
import { OpenAIFiles } from "langchain/experimental/openai_files";

const file = await OpenAIFiles.create({
file: fs.createReadStream(path.resolve(__dirname, `./test.txt`)),
purpose: "assistants",
});

const agent = await OpenAIAssistantRunnable.createAssistant({
model: "gpt-3.5-turbo-1106",
instructions:
"You are a weather bot. Use the provided functions to answer questions.",
name: "Weather Assistant",
tools,
asAgent: true,
file_ids: [file.id],
});
```

## Delete an File

```typescript
import * as fs from "fs";
import * as path from "path";
import { OpenAIFiles } from "langchain/experimental/openai_files";

const result = await OpenAIFiles.del({ fileId: file.id });
/**
* Output:
{
"id": "file-abc123",
"object": "file",
"deleted": true
}
*/
```
3 changes: 3 additions & 0 deletions langchain/src/experimental/openai_assistant/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,13 @@ export class OpenAIAssistantRunnable<
clientOptions,
asAgent,
pollIntervalMs,
file_ids,
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
}: Omit<OpenAIAssistantRunnableInput<AsAgent>, "assistantId"> & {
model: string;
name?: string;
instructions?: string;
tools?: OpenAIToolType | Array<StructuredTool>;
file_ids?: string[];
}) {
const formattedTools =
tools?.map((tool) => {
Expand All @@ -80,6 +82,7 @@ export class OpenAIAssistantRunnable<
instructions,
tools: formattedTools,
model,
file_ids,
});

return new this({
Expand Down
26 changes: 26 additions & 0 deletions langchain/src/experimental/openai_files/index.ts
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import { ClientOptions, OpenAI as OpenAIClient } from "openai";
import fs from "fs";
castroCrea marked this conversation as resolved.
Show resolved Hide resolved

export type OpenAIFilesCreate = {
file: fs.ReadStream;
purpose: "assistants" | "fine-tune";
};
export type OpenAIFilesInput = {
client?: OpenAIClient;
clientOptions?: ClientOptions;
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
};

export class OpenAIFiles {
static async create({ file, purpose }: OpenAIFilesCreate) {
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
const oaiClient = new OpenAIClient();
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
return oaiClient.files.create({
file,
purpose,
});
}

static async del({ fileId }: { fileId: string }) {
castroCrea marked this conversation as resolved.
Show resolved Hide resolved
const oaiClient = new OpenAIClient();
return oaiClient.files.del(fileId);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import * as fs from "fs";
import * as path from "path";
import { OpenAIFiles } from "../index.js";

test("Send a file to Open AI", async () => {
const file = await OpenAIFiles.create({
file: fs.createReadStream(path.resolve(__dirname, `./test.txt`)),
purpose: "assistants",
});
console.log(file);
expect(file.id).toBeDefined();
expect(file.object).toBe("file");
/**
* Output
{
"id": "file-BK7bzQj3FfZFXr7DbL6xJwfo",
"object": "file",
"bytes": 120000,
"created_at": 1677610602,
"filename": "salesOverview.pdf",
"purpose": "assistants",
}
*/
const result = await OpenAIFiles.del({ fileId: file.id });
console.log(result);
expect(result.id).toBe(file.id);
expect(result.deleted).toBeTruthy();
/**
* Output:
{
"id": "file-abc123",
"object": "file",
"deleted": true
}
*/
});
1 change: 1 addition & 0 deletions langchain/src/experimental/openai_files/tests/test.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello world