-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add inference for RunnableMap RunOutput type #3517
Merged
+133
−47
Merged
Changes from all commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
c7cb793
add RunnableMapLike to infer RunnableMap output
davidilling d6cfaae
remove unneeded changes
davidilling eab8457
fix linting
davidilling acdab12
format
davidilling 095a4e8
fix runnable_stream_log.test
davidilling 723e2ed
upgrade typescript version
davidilling 102042b
clean types
davidilling 954a14f
fix structured_output_runnables.int.test
davidilling b84f3e4
Merge remote-tracking branch 'upstream' into infer-map-types
davidilling a8acd75
ts version ~5.1.6
davidilling 6e97687
Merge remote-tracking branch 'upstream/main' into infer-map-types
davidilling 7652bf2
remove unused eslint-disable-next-line
davidilling 5cb1edd
remove another disable no-explicit-any
davidilling d07ce12
remove another no-explicit-any
davidilling ca586cb
Merge branch 'main' into infer-map-types
dilling 7826562
move eslint
davidilling 1979976
Merge branch 'main' of https://github.com/hwchase17/langchainjs into …
jacoblee93 602be50
Format
jacoblee93 fd3204f
Merge branch 'main' into infer-map-types
dilling 503ceda
Merge branch 'main' of https://github.com/hwchase17/langchainjs into …
jacoblee93 a81428b
Default runnable maps to any type in case inference is not possible
jacoblee93 361c250
Add tests
jacoblee93 f206dcb
Merge branch 'main' of https://github.com/hwchase17/langchainjs into …
jacoblee93 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
105 changes: 105 additions & 0 deletions
105
langchain-core/src/runnables/tests/runnable_map.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
/* eslint-disable no-promise-executor-return */ | ||
/* eslint-disable @typescript-eslint/no-explicit-any */ | ||
|
||
import { StringOutputParser } from "../../output_parsers/string.js"; | ||
import { | ||
ChatPromptTemplate, | ||
SystemMessagePromptTemplate, | ||
HumanMessagePromptTemplate, | ||
} from "../../prompts/chat.js"; | ||
import { | ||
FakeLLM, | ||
FakeChatModel, | ||
FakeRetriever, | ||
} from "../../utils/testing/index.js"; | ||
import { RunnableSequence, RunnableMap } from "../base.js"; | ||
import { RunnablePassthrough } from "../passthrough.js"; | ||
|
||
test("Create a runnable sequence with a runnable map", async () => { | ||
const promptTemplate = ChatPromptTemplate.fromMessages<{ | ||
documents: string; | ||
question: string; | ||
}>([ | ||
SystemMessagePromptTemplate.fromTemplate(`You are a nice assistant.`), | ||
HumanMessagePromptTemplate.fromTemplate( | ||
`Context:\n{documents}\n\nQuestion:\n{question}` | ||
), | ||
]); | ||
const llm = new FakeChatModel({}); | ||
const inputs = { | ||
question: (input: string) => input, | ||
documents: RunnableSequence.from([ | ||
new FakeRetriever(), | ||
(docs: Document[]) => JSON.stringify(docs), | ||
]), | ||
extraField: new FakeLLM({}), | ||
}; | ||
const runnable = new RunnableMap({ steps: inputs }) | ||
.pipe(promptTemplate) | ||
.pipe(llm); | ||
const result = await runnable.invoke("Do you know the Muffin Man?"); | ||
console.log(result); | ||
expect(result.content).toEqual( | ||
`You are a nice assistant.\nContext:\n[{"pageContent":"foo","metadata":{}},{"pageContent":"bar","metadata":{}}]\n\nQuestion:\nDo you know the Muffin Man?` | ||
); | ||
}); | ||
|
||
test("Test map inference in a sequence", async () => { | ||
const prompt = ChatPromptTemplate.fromTemplate( | ||
"context: {context}, question: {question}" | ||
); | ||
const chain = RunnableSequence.from([ | ||
{ | ||
question: new RunnablePassthrough(), | ||
context: async () => "SOME STUFF", | ||
}, | ||
prompt, | ||
new FakeLLM({}), | ||
new StringOutputParser(), | ||
]); | ||
const response = await chain.invoke("Just passing through."); | ||
console.log(response); | ||
expect(response).toBe( | ||
`Human: context: SOME STUFF, question: Just passing through.` | ||
); | ||
}); | ||
|
||
test("Should not allow mismatched inputs", async () => { | ||
const prompt = ChatPromptTemplate.fromTemplate( | ||
"context: {context}, question: {question}" | ||
); | ||
const badChain = RunnableSequence.from([ | ||
{ | ||
// @ts-expect-error TS compiler should flag mismatched input types | ||
question: new FakeLLM({}), | ||
context: async (input: number) => input, | ||
}, | ||
prompt, | ||
new FakeLLM({}), | ||
new StringOutputParser(), | ||
]); | ||
console.log(badChain); | ||
}); | ||
|
||
test("Should not allow improper inputs into a map in a sequence", async () => { | ||
const prompt = ChatPromptTemplate.fromTemplate( | ||
"context: {context}, question: {question}" | ||
); | ||
const map = RunnableMap.from({ | ||
question: new FakeLLM({}), | ||
context: async (_input: string) => 9, | ||
}); | ||
// @ts-expect-error TS compiler should flag mismatched output types | ||
const runnable = prompt.pipe(map); | ||
console.log(runnable); | ||
}); | ||
|
||
test("Should not allow improper outputs from a map into the next item in a sequence", async () => { | ||
const map = RunnableMap.from({ | ||
question: new FakeLLM({}), | ||
context: async (_input: string) => 9, | ||
}); | ||
// @ts-expect-error TS compiler should flag mismatched output types | ||
const runnable = map.pipe(new FakeLLM({})); | ||
console.log(runnable); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did you drop the generic passing here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So when passing as RunnableMap<RunInput, RunOutput>, the type of RunOutput does not line up until the line below
Passing just the RunInput defaults the RunOutput to Record<string, any>. So you end up having to pass both values (which is not possible to the reason above) or neither