Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making task execution in interactive non-blocking #303

Closed
BoPeng opened this issue Oct 8, 2020 · 2 comments
Closed

Making task execution in interactive non-blocking #303

BoPeng opened this issue Oct 8, 2020 · 2 comments

Comments

@BoPeng
Copy link
Contributor

BoPeng commented Oct 8, 2020

Right now when we run

task: ...

without

%run -q &

the cell will block, making the checking of job status not possible. It makes sense to make cells non-blocking if all tasks have been submitted.

@BoPeng
Copy link
Contributor Author

BoPeng commented Oct 22, 2020

The problem is that sos-notebook has a single execution thread for cell execution, so the task-execution process has to stop before users can run the next cell.

The solution would be moving the burden of checking task status to the controller so that the controller can continue to check the status and update notebook even sos notebook is working on other cells.

@BoPeng
Copy link
Contributor Author

BoPeng commented Dec 1, 2020

We need to rearrange the task execution logics.

Right now,

  1. The step executor creates the task, sends the task IDs to workflow executor, and put itself to sleep (with port waiting).
  2. The workflow executor submits the task, check the status of tasks of steps with status task_pendsing from time to time. After the task is done, it sends task results to substep, effectively wake up the substep for it to continue.

We need to move the job from workflow executor to controller so that workflow executor can terminate , basically,

  1. The step executor creates the task, sends the task IDs to workflow executor, and can terminate itself.
  2. The workflow executor submits the task and sends the IDs to controller.
  3. The workflow executor can terminate, or ask the controller the status of the tasks. The workflow executor can terminate and the task would continue to execute and complete as long as the controller is running,

BoPeng pushed a commit to vatlab/sos that referenced this issue Dec 10, 2020
@BoPeng BoPeng closed this as completed Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant