Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Claude-3 is not working properly with the OpenAI-compatible API. #32

Closed
MorningInsight3 opened this issue Apr 13, 2024 · 9 comments

Comments

@MorningInsight3
Copy link

**Claude-3 is not working properly with the OpenAI-compatible API.

Hello, I've used a proxy to make Claude-3 compatible with the OpenAI API, and it works normally in Next-Chat, Lobe-Chat, and Bob, but it doesn't work in your Morphic. Can you share the reason for this?**

Error code from vercel LOG as follows:
AI_APICallError: Invalid JSON response
at (node_modules/ai/openai/dist/index.mjs:312:0)
at (node_modules/ai/openai/dist/index.mjs:197:0)
at (node_modules/ai/openai/dist/index.mjs:578:0)
at (node_modules/ai/dist/index.mjs:405:0)
at (node_modules/ai/dist/index.mjs:538:0)
at (lib/agents/task-manager.tsx:13:17)
at (app/action.tsx:43:24) {
name: 'AI_APICallError',
url: 'https://likegpt.xx.xx.xx/api/v1/chat/completions',
requestBodyValues: {
model: 'anthropic.claude-3-sonnet-20240229-v1:0',
logit_bias: undefined,
user: undefined,
max_tokens: undefined,
temperature: undefined,
top_p: undefined,
frequency_penalty: undefined,
presence_penalty: undefined,
seed: undefined,
messages: [
{
role: 'system',
content: 'As a professional web researcher, your primary objective is to fully comprehend the user's query, conduct thorough web searches to gather the necessary information, and provide an appropriate response.\n To achieve this, you must first analyze the user's input and determine the optimal course of action. You have two options at your disposal:\n 1. "proceed": If the provided information is sufficient to address the query effectively, choose this option to proceed with the research and formulate a response.\n 2. "inquire": If you believe that additional information from the user would enhance your ability to provide a comprehensive response, select this option. You may present a form to the user, offering default selections or free-form input fields, to gather the required details.\n Your decision should be based on a careful assessment of the context and the potential for further information to improve the quality and relevance of your response.\n For example, if the user asks, "What are the key features of the latest iPhone model?", you may choose to "proceed" as the query is clear and can be answered effectively with web research alone.\n However, if the user asks, "What's the best smartphone for my needs?", you may opt to "inquire" and present a form asking about their specific requirements, budget, and preferred features to provide a more tailored recommendation.\n Make your choice wisely to ensure that you fulfill your mission as a web researcher effectively and deliver the most valuable assistance to the user.\n '
},
{
role: 'user',
content: [
{
type: 'text',
text: '{"input":"Is the Apple Vision Pro worth buying?"}'
}
]
}
],
tool_choice: { type: 'function', function: { name: 'json' } },
tools: [
{
type: 'function',
function: {
name: 'json',
description: 'Respond with a JSON object.',
parameters: {
type: 'object',
properties: { next: { type: 'string', enum: [ 'inquire', 'proceed' ] } },
required: [ 'next' ],
additionalProperties: false,
$schema: 'http://json-schema.org/draft-07/schema#'
}
}
}
]
},
statusCode: 200,
responseBody: '{"id":"msg_01RJEqLhDqDqriF5XktcB8Vc","created":1713013011,"model":"anthropic.claude-3-sonnet-20240229-v1:0","system_fingerprint":"fp","choices":[{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"a1e851c7","type":"function","function":{"name":"json","arguments":"{\"next\": \"inquire\"}"}}]}}],"object":"chat.completion","usage":{"prompt_tokens":546,"completion_tokens":44,"total_tokens":590}}',
cause: AI_TypeValidationError: Type validation failed: Value: {"id":"msg_01RJEqLhDqDqriF5XktcB8Vc","created":1713013011,"model":"anthropic.claude-3-sonnet-20240229-v1:0","system_fingerprint":"fp","choices":[{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","tool_calls":[{"id":"a1e851c7","type":"function","function":{"name":"json","arguments":"{"next": "inquire"}"}}]}}],"object":"chat.completion","usage":{"prompt_tokens":546,"completion_tokens":44,"total_tokens":590}}.
Error message: [
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
"choices",
0,
"message",
"content"
],
"message": "Required"
}
]
at (node_modules/ai/openai/dist/index.mjs:73:0)
at (node_modules/ai/openai/dist/index.mjs:116:28)
at (vc/edge/function:2

@MorningInsight3
Copy link
Author

image

@MorningInsight3 MorningInsight3 changed the title Claude-3 is not working properly with the OpenAI-compatible API. [Feature] Claude-3 is not working properly with the OpenAI-compatible API. Apr 15, 2024
@miurla
Copy link
Owner

miurla commented Apr 20, 2024

We cannot support parts that require tool calling and JSON output, but I 've made it possible to specify OpenAI-compatible API for the final output. Please give it a try.

#58

@MorningInsight3
Copy link
Author

Thank you for your reply. I quickly went through it again, and there seems to be another error:

image

@MorningInsight3
Copy link
Author

AI_NoTextGeneratedError: No text generated.
at (node_modules/ai/dist/index.mjs:480:0)
at (lib/agents/task-manager.tsx:14:19)
at (app/action.tsx:49:24) {
name: 'AI_NoTextGeneratedError'
}

@miurla
Copy link
Owner

miurla commented Apr 22, 2024

at (lib/agents/task-manager.tsx:14:19)

An error is occurring in the task manager.
This solution cannot be applied to all APIs.
Please delete specifications such as the OPENAI_BASE_URL, then try the following solution.

#58 (comment)

@MorningInsight3
Copy link
Author

MorningInsight3 commented Apr 23, 2024

Thank you for your reply. I quickly went through it again, and there seems to be another error:

image AI_LoadAPIKeyError: OpenAI API key is missing. Pass it using the 'apiKey' parameter or the OPENAI_API_KEY environment variable. at (node_modules/@ai-sdk/provider-utils/dist/index.mjs:45:0) at (node_modules/@ai-sdk/openai/dist/index.mjs:766:43) at (node_modules/@ai-sdk/openai/dist/index.mjs:245:0) at (node_modules/ai/dist/index.mjs:462:0) at (node_modules/ai/dist/index.mjs:328:0) at (node_modules/ai/dist/index.mjs:317:6) at (node_modules/ai/dist/index.mjs:461:0) at (lib/agents/task-manager.tsx:14:25) at (app/action.tsx:49:31) at (app/action.tsx:121:2) { name: 'AI_LoadAPIKeyError' }

WITH upgraded Environment Variables:

image

miurla added a commit that referenced this issue Apr 23, 2024
Add MDX components and layouts for legal pages
@superlbr
Copy link
Contributor

superlbr commented May 8, 2024

AI_NoTextGeneratedError: No text generated. just hang whole task, how about add a timeout if not work

@denispol
Copy link

I always get this response when I query using Claude 3 Haiku:

image

Here is the activity log from the API endpoint provider:
image

@miurla
Copy link
Owner

miurla commented Jun 22, 2024

Added Anthropic Provider support: #239

@miurla miurla closed this as completed Jun 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants