Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agent): Implement more tolerant json_loads function #7016

Conversation

kcze
Copy link
Contributor

@kcze kcze commented Mar 14, 2024

OPEN-183

Background

OpenAI API tends to return broken json responses (such as trailing commas, extra newlines) which fail to be parsed by built-in json.loads.
This change introduces relaxed parsing function json_loads using demjson3 that handles minor syntax issues in json and is used where json.loads was used previously to parse responses from llm.
It can deal with:

  • Missing, extra and trailing commas
  • Extraneous newlines and whitespace outside of string literals
  • Inconsistent spacing after colons and commas
  • Missing closing brackets or braces
  • Numbers: binary, hex, octal, trailing and prefixed decimal points
  • Different encodings
  • Comments

Changes 🏗️

  • Add dependency for demjson3 ^3.0.0 (LGPLv3 license)
  • json_loads function
  • Swap json.loads with json_loads across multiple files
  • Tests

Copy link

netlify bot commented Mar 14, 2024

Deploy Preview for auto-gpt-docs ready!

Name Link
🔨 Latest commit 03df00b
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/65fb52da4f4bfa0007abe779
😎 Deploy Preview https://deploy-preview-7016--auto-gpt-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@ntindle
Copy link
Member

ntindle commented Mar 14, 2024

Do you have benchmark stats for this? I’d like to see if it improves our score much

autogpts/autogpt/pyproject.toml Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/json_utils/utilities.py Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/json_utils/utilities.py Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/commands/image_gen.py Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/commands/web_search.py Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/core/json_utils/utils.py Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/core/json_utils/utils.py Outdated Show resolved Hide resolved
autogpts/autogpt/autogpt/core/json_utils/utils.py Outdated Show resolved Hide resolved
@kcze kcze requested a review from Pwuts March 18, 2024 19:16
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Mar 20, 2024
Pwuts
Pwuts previously approved these changes Mar 20, 2024
@Pwuts Pwuts changed the title feat(autogpt): Implement tolerant json_loads function feat(autogpt): Implement more tolerant json_loads function Mar 20, 2024
Copy link
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Mar 20, 2024
Copy link
Member

@Pwuts Pwuts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, LGTM!

@Pwuts Pwuts changed the title feat(autogpt): Implement more tolerant json_loads function feat(agent): Implement more tolerant json_loads function Mar 21, 2024
@Pwuts Pwuts merged commit 76d6e61 into Significant-Gravitas:master Mar 21, 2024
11 of 14 checks passed
@kcze kcze deleted the kpczerwinski/open-183-llm-returned-incorrectly-formatted-json-jsondecodeerror branch March 21, 2024 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants