-
Notifications
You must be signed in to change notification settings - Fork 669
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update ray.go not to fail when going suspend state. #5816
Conversation
We have a Queue to schedule our ray jobs, so we need it to wait in Suspended state of k8s Signed-off-by: Amin Maghsodi <[email protected]>
Thank you for opening this pull request! 🙌 These tips will help get your PR across the finish line:
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5816 +/- ##
==========================================
+ Coverage 36.35% 36.81% +0.45%
==========================================
Files 1304 1309 +5
Lines 110147 130899 +20752
==========================================
+ Hits 40042 48184 +8142
- Misses 65938 78533 +12595
- Partials 4167 4182 +15
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Signed-off-by: Amin Maghsodi <[email protected]>
Improve test cov Signed-off-by: Amin Maghsodi <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks. just curious, which gang scheduler are you using? do you need to change anything in Flyte to make your gang scheduler work with Ray task?
@@ -755,7 +755,8 @@ func TestGetTaskPhase(t *testing.T) { | |||
{rayv1.JobDeploymentStatusRunning, pluginsCore.PhaseRunning, false}, | |||
{rayv1.JobDeploymentStatusComplete, pluginsCore.PhaseSuccess, false}, | |||
{rayv1.JobDeploymentStatusFailed, pluginsCore.PhasePermanentFailure, false}, | |||
{rayv1.JobDeploymentStatusSuspended, pluginsCore.PhaseUndefined, true}, | |||
{rayv1.JobDeploymentStatusSuspended, pluginsCore.PhaseQueued, true}, | |||
{rayv1.JobDeploymentStatusSuspending, pluginsCore.PhaseUndefined, true}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@aminmaghsodi could we map it to pluginsCore.PhaseQueued
as well?
Signed-off-by: Kevin Su <[email protected]>
Congrats on merging your first pull request! 🎉 |
We have a Queue to schedule our ray jobs, so we need it to wait in Suspended state of k8s
Now workflow fails, exactly when job state goes to 'Suspended'