the loop started by dask-ctl.lifecycle.create_cluster conflicts with jupyterlab's loop
#66
Comments
|
Yeah wrapping async code in IPython/Jupyter is a minefield. The code here is well overdue an overhaul so I'm not surprised things like this come up. I spent a bunch of time thinking about this in There's also asyncer which tries to solve a lot of the same problems but with a slightly different use case in mind. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I tried to launch a local cluster on jupyterlab using the python API, and received an error saying that the loop was already running:
traceback
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) Cell In[1], line 17 14 path = pathlib.Path("spec.yaml") 15 path.write_text(spec) ---> 17 dask_ctl.lifecycle.create_cluster(str(path)) File .../lib/python3.12/site-packages/dask_ctl/lifecycle.py:73, in create_cluster(spec_path, local_fallback, asynchronous) 70 cluster.shutdown_on_close = False 71 return cluster ---> 73 return loop.run_sync(_create_cluster) File .../lib/python3.12/site-packages/tornado/ioloop.py:533, in IOLoop.run_sync(self, func, timeout) 530 self.stop() 532 timeout_handle = self.add_timeout(self.time() + timeout, timeout_callback) --> 533 self.start() 534 if timeout is not None: 535 self.remove_timeout(timeout_handle) File .../lib/python3.12/site-packages/tornado/platform/asyncio.py:205, in BaseAsyncIOLoop.start(self) 204 def start(self) -> None: --> 205 self.asyncio_loop.run_forever() File .../lib/python3.12/asyncio/base_events.py:630, in BaseEventLoop.run_forever(self) 628 """Run until stop() is called.""" 629 self._check_closed() --> 630 self._check_running() 631 self._set_coroutine_origin_tracking(self._debug) 633 old_agen_hooks = sys.get_asyncgen_hooks() File .../lib/python3.12/asyncio/base_events.py:622, in BaseEventLoop._check_running(self) 620 def _check_running(self): 621 if self.is_running(): --> 622 raise RuntimeError('This event loop is already running') 623 if events._get_running_loop() is not None: 624 raise RuntimeError( 625 'Cannot run the event loop while another loop is running') RuntimeError: This event loop is already runningversions
As far as I can tell, the reason for this is that
ipykernelruns a event loop in the background (to support top-levelawaitcalls), which causes the loop startedtornado.ioloop.IOLoop.run_syncto error.Getting this to work is an absolute rabbit hole, though: nesting loops is – intentionally, it seems – not supported by
asynciowhich is whattornado.ioloopwraps starting withtornado>=6, so this will likely need a separate thread for the new event loop.dask/distributedmight have a mechanism to execute async code synchronously already, even injupyterlab, which could be reused here?The text was updated successfully, but these errors were encountered: