You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Integrate the DeepEval evaluation framework by implementing a new DeepEvalEvaluator component
Detailed design
Based on the proposal deepset-ai/haystack#6784 we need to implement an DeepEvalEvaluator component that is initialized with one metric and an OpenAI API key. Depending on the particular metric, the component expects for example question, answer and context as input and outputs a dictionary with the calculated score.
Checklist
If the request is accepted, ensure the following checklist is complete before closing this issue.
The code is documented with docstrings and was merged in the main branch
Summary and motivation
Integrate the DeepEval evaluation framework by implementing a new DeepEvalEvaluator component
Detailed design
Based on the proposal deepset-ai/haystack#6784 we need to implement an DeepEvalEvaluator component that is initialized with one metric and an OpenAI API key. Depending on the particular metric, the component expects for example question, answer and context as input and outputs a dictionary with the calculated score.
Checklist
If the request is accepted, ensure the following checklist is complete before closing this issue.
main
branchintegration:<your integration name>
has been added to this repoThe text was updated successfully, but these errors were encountered: