Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduled test cases fix #71

Merged
merged 3 commits into from
Jul 15, 2024
Merged

Scheduled test cases fix #71

merged 3 commits into from
Jul 15, 2024

Conversation

raspawar
Copy link
Collaborator

@raspawar raspawar commented Jul 12, 2024

Fix for the scheduled integration test_chat_model test case failure

Screenshot 2024-07-12 at 7 42 32 PM

Reason for failure: mistralai model does not allow AIMessage followed by SystemMessage

Successful Workflow

@raspawar raspawar requested a review from mattf July 12, 2024 14:15
@raspawar raspawar added the bug Something isn't working label Jul 12, 2024
@mattf
Copy link
Collaborator

mattf commented Jul 12, 2024

@raspawar ok, if this is the issue then the server side implementation must have changed. our options are to 0. change the default model to one that passes all the tests, 1. mark this test as xfail.

go w/ (1) for now and investigate (0).

@raspawar
Copy link
Collaborator Author

@mattf My guess is ==> It is not server side implementation
This test cases were added in recent PR: #63
And Integration test cases run only in Scheduled tests github workflow action, so after it got merged, scheduled test cases started failing.

@mattf
Copy link
Collaborator

mattf commented Jul 15, 2024

My guess is ==> It is not server side implementation This test cases were added in recent PR: #63 And Integration test cases run only in Scheduled tests github workflow action, so after it got merged, scheduled test cases started failing.

#63 added test_stop while the failures are in test_messages. the test_stop tests are already marked xfail

change test_messages to also be xfail for now.

@raspawar
Copy link
Collaborator Author

ok, I will keep looking, meanwhile I have marked test_messages as xfail, so can we close this as the scheduled test cases will keep failing

@mattf mattf merged commit c6c0c33 into main Jul 15, 2024
12 checks passed
@mattf mattf deleted the raspawar/mix_test_case_fix branch July 15, 2024 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants