Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

langchain[major]: LangChain community #3581

Merged
merged 24 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 7 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ services:
- ./environment_tests/scripts:/scripts
- ./langchain:/langchain
- ./langchain-core:/langchain-core
- ./libs/langchain-community:/langchain-community
- ./libs/langchain-anthropic:/langchain-anthropic
- ./libs/langchain-openai:/langchain-openai
command: bash /scripts/docker-ci-entrypoint.sh
Expand All @@ -25,6 +26,7 @@ services:
- ./environment_tests/scripts:/scripts
- ./langchain:/langchain
- ./langchain-core:/langchain-core
- ./libs/langchain-community:/langchain-community
- ./libs/langchain-anthropic:/langchain-anthropic
- ./libs/langchain-openai:/langchain-openai
command: bash /scripts/docker-ci-entrypoint.sh
Expand All @@ -39,6 +41,7 @@ services:
- ./environment_tests/scripts:/scripts
- ./langchain:/langchain
- ./langchain-core:/langchain-core
- ./libs/langchain-community:/langchain-community
- ./libs/langchain-anthropic:/langchain-anthropic
- ./libs/langchain-openai:/langchain-openai
command: bash /scripts/docker-ci-entrypoint.sh
Expand All @@ -53,6 +56,7 @@ services:
- ./environment_tests/scripts:/scripts
- ./langchain:/langchain
- ./langchain-core:/langchain-core
- ./libs/langchain-community:/langchain-community
- ./libs/langchain-anthropic:/langchain-anthropic
- ./libs/langchain-openai:/langchain-openai
command: bash /scripts/docker-ci-entrypoint.sh
Expand All @@ -67,6 +71,7 @@ services:
- ./environment_tests/scripts:/scripts
- ./langchain:/langchain
- ./langchain-core:/langchain-core
- ./libs/langchain-community:/langchain-community
- ./libs/langchain-anthropic:/langchain-anthropic
- ./libs/langchain-openai:/langchain-openai
command: bash /scripts/docker-ci-entrypoint.sh
Expand All @@ -81,6 +86,7 @@ services:
- ./environment_tests/scripts:/scripts
- ./langchain:/langchain
- ./langchain-core:/langchain-core
- ./libs/langchain-community:/langchain-community
- ./libs/langchain-anthropic:/langchain-anthropic
- ./libs/langchain-openai:/langchain-openai
command: bash /scripts/docker-ci-entrypoint.sh
Expand All @@ -92,6 +98,7 @@ services:
# - ./environment_tests/scripts:/scripts
# - ./langchain:/langchain-workspace
# - ./langchain-core:/langchain-core
# - ./libs/langchain-community:/langchain-community-workspace
# - ./libs/langchain-anthropic:/langchain-anthropic-workspace
# command: bash /scripts/docker-bun-ci-entrypoint.sh
success:
Expand Down
92 changes: 92 additions & 0 deletions docs/core_docs/docs/expression_language/get_started.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
sidebar_position: 0
title: Get started
---

import CodeBlock from "@theme/CodeBlock";
import BasicExample from "@examples/guides/expression_language/get_started/basic.ts";
import BasicPromptExample from "@examples/guides/expression_language/get_started/prompt.ts";
import BasicChatModelExample from "@examples/guides/expression_language/get_started/chat_model.ts";
import BasicLLMModelExample from "@examples/guides/expression_language/get_started/llm_model.ts";
import BasicOutputParserExample from "@examples/guides/expression_language/get_started/output_parser.ts";
import BasicRagExample from "@examples/guides/expression_language/get_started/rag.ts";

# Get started

LCEL makes it easy to build complex chains from basic components, and supports out of the box functionality such as streaming, parallelism, and logging.

## Basic example: prompt + model + output parser

The most basic and common use case is chaining a prompt template and a model together. To see how this works, let's create a chain that takes a topic and generates a joke:

<CodeBlock language="typescript">{BasicExample}</CodeBlock>

:::tip

[LangSmith trace](https://smith.langchain.com/public/dcac6d79-5254-4889-a974-4b3abaf605b4/r)

:::

Notice in this line we're chaining our prompt, LLM model and output parser together:

```typescript
const chain = prompt.pipe(model).pipe(outputParser);
```

The `.pipe()` method allows for chaining together any number of runnables. It will pass the output of one through to the input of the next.

Here, the prompt is passed a `topic` and when invoked it returns a formatted string with the `{topic}` input variable replaced with the string we passed to the invoke call.
That string is then passed as the input to the LLM which returns a `BaseMessage` object. Finally, the output parser takes that `BaseMessage` object and returns the content of that object as a string.

### 1. Prompt

`prompt` is a `BasePromptTemplate`, which means it takes in an object of template variables and produces a `PromptValue`.
A `PromptValue` is a wrapper around a completed prompt that can be passed to either an `LLM` (which takes a string as input) or `ChatModel` (which takes a sequence of messages as input).
It can work with either language model type because it defines logic both for producing BaseMessages and for producing a string.

<CodeBlock language="typescript">{BasicPromptExample}</CodeBlock>

### 2. Model

The `PromptValue` is then passed to `model`. In this case our `model` is a `ChatModel`, meaning it will output a `BaseMessage`.

<CodeBlock language="typescript">{BasicChatModelExample}</CodeBlock>

If our model was an LLM, it would output a string.

<CodeBlock language="typescript">{BasicLLMModelExample}</CodeBlock>

### 3. Output parser

And lastly we pass our `model` output to the `outputParser`, which is a `BaseOutputParser` meaning it takes either a string or a `BaseMessage` as input. The `StringOutputParser` specifically simple converts any input into a string.

<CodeBlock language="typescript">{BasicOutputParserExample}</CodeBlock>

## RAG Search Example

For our next example, we want to run a retrieval-augmented generation chain to add some context when responding to questions.

<CodeBlock language="typescript">{BasicRagExample}</CodeBlock>

:::tip

[LangSmith trace](https://smith.langchain.com/public/f0205e20-c46f-47cd-a3a4-6a95451f8a25/r)

:::

In this chain we add some extra logic around retrieving context from a vector store.

We first instantiated our model, vector store and output parser. Then we defined our prompt, which takes in two input variables:

- `context` -> this is a string which is returned from our vector store based on a semantic search from the input.
- `question` -> this is the question we want to ask.

Next we created a `setupAndRetriever` runnable. This has two components which return the values required by our prompt:

- `context` -> this is a `RunnableLambda` which takes the input from the `.invoke()` call, makes a request to our vector store, and returns the first result.
- `question` -> this uses a `RunnablePassthrough` which simply passes whatever the input was through to the next step, and in our case it returns it to the key in the object we defined.

Both of these are wrapped inside a `RunnableMap`. This is a special type of runnable that takes an object of runnables and executes them all in parallel.
It then returns an object with the same keys as the input object, but with the values replaced with the output of the runnables.

Finally, we pass the output of the `setupAndRetriever` to our `prompt` and then to our `model` and `outputParser` as before.
1 change: 1 addition & 0 deletions environment_tests/scripts/docker-ci-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ cp ../root/yarn.lock ../root/.yarnrc.yml .
# Avoid calling "yarn add ../langchain" as yarn berry does seem to hang for ~30s
# before installation actually occurs
sed -i 's/"@langchain\/core": "workspace:\*"/"@langchain\/core": "..\/langchain-core"/g' package.json
sed -i 's/"@langchain\/community": "workspace:\*"/"@langchain\/community": "..\/langchain-community"/g' package.json
sed -i 's/"@langchain\/anthropic": "workspace:\*"/"@langchain\/anthropic": "..\/langchain-anthropic"/g' package.json
sed -i 's/"@langchain\/openai": "workspace:\*"/"@langchain\/openai": "..\/langchain-openai"/g' package.json
sed -i 's/"langchain": "workspace:\*"/"langchain": "..\/langchain"/g' package.json
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-bun/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"license": "MIT",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! I noticed the addition of a new dependency, "@langchain/community", in the package.json file. This seems to be a new peer/dev/hard dependency, and I'm flagging this for the maintainers to review. Great work on the PR!

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"d3-dsv": "2",
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-cf/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! 👋 I noticed that this PR introduces a new dependency, "@langchain/community", with a version specified as "workspace:*". This comment is to flag the dependency change for maintainers to review, especially regarding its type (peer/dev/hard). Keep up the great work!

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"langchain": "workspace:*"
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-cjs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
"license": "MIT",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! 👋 I noticed that this PR adds a new hard dependency "@langchain/community" to the project's package.json. Just flagging this for the maintainers to review. Great work overall! 🚀

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"d3-dsv": "2",
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-esbuild/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"license": "MIT",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! 👋 I noticed that this PR adds a new dependency to the project, "@langchain/community". It's important for maintainers to review this change to ensure it aligns with the project's dependency requirements. Thank you for flagging this for review! 🚀

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"d3-dsv": "2",
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-esm/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"license": "MIT",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! I noticed that the addition of "@langchain/community" as a dependency in the package.json file might impact the project's peer dependencies. I've flagged this for your review as it's an important change to the project's dependencies. Keep up the great work!

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"d3-dsv": "2",
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-vercel/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! 👋 Just wanted to flag for the maintainers that a new dev dependency, "@langchain/community", has been added in this PR. It's always good to review dependency changes to ensure everything is in order. Thanks!

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"@types/node": "18.15.11",
Expand Down
1 change: 1 addition & 0 deletions environment_tests/test-exports-vite/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! I noticed that the addition of "@langchain/community" as a dependency in the package.json file might impact the project's dependencies. I've flagged this for your review to ensure it aligns with the intended dependency type (peer/dev/hard). Keep up the great work!

"dependencies": {
"@langchain/anthropic": "workspace:*",
"@langchain/community": "workspace:*",
"@langchain/core": "workspace:*",
"@langchain/openai": "workspace:*",
"langchain": "workspace:*"
Expand Down
4 changes: 2 additions & 2 deletions examples/src/experimental/masking/basic.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ maskingParser.addTransformer(piiMaskingTransformer);

const input =
"Contact me at [email protected] or 555-123-4567. Also reach me at [email protected]";
const masked = await maskingParser.parse(input);
const masked = await maskingParser.mask(input);

console.log(masked);
// Contact me at [email-a31e486e324f6] or [phone-da8fc1584f224]. Also reach me at [email-d5b6237633d95]

const rehydrated = maskingParser.rehydrate(masked);
const rehydrated = await maskingParser.rehydrate(masked);
console.log(rehydrated);
// Contact me at [email protected] or 555-123-4567. Also reach me at [email protected]
2 changes: 1 addition & 1 deletion examples/src/experimental/masking/kitchen_sink.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ const message =

// Mask and rehydrate the message
maskingParser
.parse(message)
.mask(message)
.then((maskedMessage: string) => {
console.log(`Masked message: ${maskedMessage}`);
return maskingParser.rehydrate(maskedMessage);
Expand Down
11 changes: 8 additions & 3 deletions examples/src/experimental/masking/next.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ export async function POST(req: Request) {
const formattedPreviousMessages = messages.slice(0, -1).map(formatMessage);
const currentMessageContent = messages[messages.length - 1].content; // Extract the content of the last message
// Mask sensitive information in the current message
const guardedMessageContent = await maskingParser.parse(
const guardedMessageContent = await maskingParser.mask(
currentMessageContent
);
// Mask sensitive information in the chat history
const guardedHistory = await maskingParser.parse(
const guardedHistory = await maskingParser.mask(
formattedPreviousMessages.join("\n")
);

Expand All @@ -64,6 +64,11 @@ export async function POST(req: Request) {
headers: { "content-type": "text/plain; charset=utf-8" },
});
} catch (e: any) {
return Response.json({ error: e.message }, { status: 500 });
return new Response(JSON.stringify({ error: e.message }), {
status: 500,
headers: {
"content-type": "application/json",
},
});
}
}
20 changes: 20 additions & 0 deletions examples/src/guides/expression_language/get_started/basic.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { ChatOpenAI } from "langchain/chat_models/openai";
import { ChatPromptTemplate } from "langchain/prompts";
import { StringOutputParser } from "langchain/schema/output_parser";

const prompt = ChatPromptTemplate.fromMessages([
["human", "Tell me a short joke about {topic}"],
]);
const model = new ChatOpenAI({});
const outputParser = new StringOutputParser();

const chain = prompt.pipe(model).pipe(outputParser);

const response = await chain.invoke({
topic: "ice cream",
});
console.log(response);
/**
Why did the ice cream go to the gym?
Because it wanted to get a little "cone"ditioning!
*/
14 changes: 14 additions & 0 deletions examples/src/guides/expression_language/get_started/chat_model.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { ChatOpenAI } from "langchain/chat_models/openai";

const model = new ChatOpenAI({});
const promptAsString = "Human: Tell me a short joke about ice cream";

const response = await model.invoke(promptAsString);
console.log(response);
/**
AIMessage {
content: 'Sure, here you go: Why did the ice cream go to school? Because it wanted to get a little "sundae" education!',
name: undefined,
additional_kwargs: { function_call: undefined, tool_calls: undefined }
}
*/
12 changes: 12 additions & 0 deletions examples/src/guides/expression_language/get_started/llm_model.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { OpenAI } from "langchain/llms/openai";

const model = new OpenAI({});
const promptAsString = "Human: Tell me a short joke about ice cream";

const response = await model.invoke(promptAsString);
console.log(response);
/**
Why did the ice cream go to therapy?

Because it was feeling a little rocky road.
*/
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { AIMessage } from "langchain/schema";
import { StringOutputParser } from "langchain/schema/output_parser";

const outputParser = new StringOutputParser();
const message = new AIMessage(
'Sure, here you go: Why did the ice cream go to school? Because it wanted to get a little "sundae" education!'
);
const parsed = await outputParser.invoke(message);
console.log(parsed);
/**
Sure, here you go: Why did the ice cream go to school? Because it wanted to get a little "sundae" education!
*/
34 changes: 34 additions & 0 deletions examples/src/guides/expression_language/get_started/prompt.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { ChatPromptTemplate } from "langchain/prompts";

const prompt = ChatPromptTemplate.fromMessages([
["human", "Tell me a short joke about {topic}"],
]);
const promptValue = await prompt.invoke({ topic: "ice cream" });
console.log(promptValue);
/**
ChatPromptValue {
messages: [
HumanMessage {
content: 'Tell me a short joke about ice cream',
name: undefined,
additional_kwargs: {}
}
]
}
*/
const promptAsMessages = promptValue.toChatMessages();
console.log(promptAsMessages);
/**
[
HumanMessage {
content: 'Tell me a short joke about ice cream',
name: undefined,
additional_kwargs: {}
}
]
*/
const promptAsString = promptValue.toString();
console.log(promptAsString);
/**
Human: Tell me a short joke about ice cream
*/
Loading