Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Agents Eval in v1.12.3 - error when computing ToolActionMatch "Key Error: res_tool_action" #252

Closed
cyclingdata opened this issue Oct 9, 2024 · 0 comments · Fixed by #253
Assignees
Labels
bug Something isn't working

Comments

@cyclingdata
Copy link

Expected Behavior

Getting a lot of errors with new SCRAPI 1.12.3 released today that I didn't have with 1.12.2

Current Behavior

When running run_query_and_eval method, getting an error in Computing ToolActionMatch

[/usr/local/lib/python3.10/dist-packages/dfcx_scrapi/tools/metrics.py](https://localhost:8080/#) in __call__(self, inputs)
    166         tool_action_match = self.text_match.score(
    167             reference=inputs["tool_action"],
--> 168             prediction=inputs["res_tool_action"]
    169         )
    170 

KeyError: 'res_tool_action'

Possible Solution

N/A

Steps to Reproduce

Will share bot and conversations separately

@cyclingdata cyclingdata added the bug Something isn't working label Oct 9, 2024
@kmaphoenix kmaphoenix linked a pull request Oct 10, 2024 that will close this issue
@kmaphoenix kmaphoenix self-assigned this Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants