Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Prompt Caching for Anthropic #817

Open
MichaelDoyle opened this issue Aug 24, 2024 · 2 comments
Open

Support Prompt Caching for Anthropic #817

MichaelDoyle opened this issue Aug 24, 2024 · 2 comments

Comments

@MichaelDoyle
Copy link
Member

This one is really interesting. Not sure the best way to go about it (extend Genkit's message abstraction, middleware, etc). Worth additional discussion.

See: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching

@chrisraygill
Copy link
Contributor

@MichaelDoyle is this available through Anthropic on Vertex AI? Otherwise, we can generally make this possible to implement, but the actual change will need to happen in the community plugin repo.

@MichaelDoyle
Copy link
Member Author

Agreed - I suspect there is a brief discussion that needs to happen on "how" we want this to be supported in the core library, and then implementation details fall into the plugins.

When I get some free cycles I can look closer at Vertex. I suspect it might be supported in the Anthropic SDK but not Vertex AI SDK. That's how it appears to me based on the change log: anthropics/anthropic-sdk-typescript@vertex-sdk-v0.4.0...vertex-sdk-v0.4.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants