Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(OpenAI Node, Basic LLM Chain Node, Tool Agent Node): Better OpenAI API rate limit errors #10797

Merged
merged 2 commits into from
Sep 18, 2024

Conversation

burivuhster
Copy link
Contributor

Summary

In this PR some custom handling for the OpenAI rate limit errors is introduced.
According to OpenAI's docs, there might be two sets of errors ending up with 429 HTTP response: rate limit errors and insufficient quota errors (https://platform.openai.com/docs/guides/error-codes/api-errors)
We can look at the code in the response to see what kind of error it was.

Also, we override a default message for those errors with a more descriptive and openai-specific ones.
This should also help with the issue, when a long poorly formatted text is displayed in some cases of rate limit errors.

Related Linear tickets, Github issues, and Community forum posts

https://linear.app/n8n/issue/AI-313/openai-is-returning-429-errors-quite-often
https://linear.app/n8n/issue/AI-342/bad-error-in-openai-node

Review / Merge checklist

  • PR title and summary are descriptive. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

@burivuhster burivuhster marked this pull request as ready for review September 12, 2024 15:24
@n8n-assistant n8n-assistant bot added the n8n team Authored by the n8n team label Sep 12, 2024
Copy link
Contributor

@jeanpaul jeanpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments to clarify whether this is actually only triggered by OpenAI models or not.

if (error instanceof NodeApiError) {
// If the error is an OpenAI's rate limit error, we want to handle it differently
// because OpenAI has multiple different rate limit errors
const openAiErrorCode: string | undefined = (error.cause as any).error?.code;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you know, at this point, that it's an actual OpenAI error, and not some other API error that happens to have an error-code? Please check if this check is correct, and/or if the name of the variable could signal something better :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've added a check that the underlying error comes from OpenAI

@burivuhster burivuhster force-pushed the ai-313-openai-is-returning-429-errors-quite-often branch from cf7976f to 780cb8d Compare September 17, 2024 08:56
@jeanpaul
Copy link
Contributor

image Some models might be blocked by the policies of a project too, which results in a weird error "Forbidden":
  "errorMessage": "Forbidden - perhaps check your credentials?",
  "errorDescription": "Project `proj_r4lQnxwlsEgGpJiweSPiNTfO` does not have access to model `gpt-3.5-turbo`",
  "errorDetails": {
    "rawErrorMessage": [
      "403 - {\"error\":{\"message\":\"Project `proj_r4lQnxwlsEgGpJiweSPiNTfO` does not have access to model `gpt-3.5-turbo`\",\"type\":\"invalid_request_error\",\"param\":null,\"code\":\"model_not_found\"}}"
    ],
    "httpCode": "403"
  },
  "n8nDetails": {
    "nodeName": "OpenAI",
    "nodeType": "@n8n/n8n-nodes-langchain.openAi",
    "nodeVersion": 1.5,
    "resource": "text",
    "operation": "message",
    "itemIndex": 0,
    "time": "9/17/2024, 4:51:36 PM",
    "n8nVersion": "1.59.0 (Self Hosted)",
    "binaryDataMode": "default",
    "stackTrace": [
      "NodeApiError: Forbidden - perhaps check your credentials?",
      "    at Object.requestWithAuthentication (n8n/packages/core/src/NodeExecuteFunctions.ts:1924:10)",
...
    ]
  }
}```

Copy link
Contributor

@jeanpaul jeanpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be cool if you could add the "forbidden" error to the error messages too!

Copy link
Contributor

✅ All Cypress E2E specs passed

Copy link

cypress bot commented Sep 18, 2024

n8n    Run #6935

Run Properties:  status check passed Passed #6935  •  git commit 780cb8dbce: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 burivuhster 🗃️ e2e/*
Project n8n
Branch Review ai-313-openai-is-returning-429-errors-quite-often
Run status status check passed Passed #6935
Run duration 04m 43s
Commit git commit 780cb8dbce: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 burivuhster 🗃️ e2e/*
Committer Eugene Molodkin
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 3
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 430
View all changes introduced in this branch ↗︎

@burivuhster burivuhster merged commit ab83c4b into master Sep 18, 2024
40 of 42 checks passed
@burivuhster burivuhster deleted the ai-313-openai-is-returning-429-errors-quite-often branch September 18, 2024 08:52
@github-actions github-actions bot mentioned this pull request Sep 18, 2024
@janober
Copy link
Member

janober commented Sep 18, 2024

Got released with [email protected]

riascho pushed a commit that referenced this pull request Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants