Python 3.10 - DeprecationWarning: There is no current event loop

[avylove]

I'm not using pyzmq directly, but it's a dependency of other imports. I notice deprecation warnings being raised

/lib64/python3.10/site-packages/zmq/_future.py:499: DeprecationWarning: There is no current event loop
  f = future or self._Future()

That's referencing this code

pyzmq/zmq/_future.py

Lines 496 to 498 in 6064159

	def _add_send_event(self, kind, msg=None, kwargs=None, future=None):
	"""Add a send event, returning the corresponding Future"""
	f = future or self._Future()

Checking the docs, it looks like the deprecation was introduced in 3.10

Deprecated since version 3.10: Deprecation warning is emitted if loop is not specified and there is no running event loop.

More information is available here.

This is with pyzmq 24.0.1.

[minrk]

I don't believe this is pyzmq issue. It means that an async pyzmq call is being made outside an event loop, and the warning is being shown correctly, but it is highlighting a pyzmq line as the direct caller of the asyncio API, instead of the originating call, which is whatever code invoked the async pyzmq method.

What package is using pyzmq in your case? That's where the deprecation should be reported.

[avylove]

I would disagree. A library should never propagate a deprecation warning. This is an indicator the the way the code is being called will break in a future release.

I think the root of this is the way the future is being created. It seems the code is effectively creating a future with asyncio.Future(). The side effect is this creates an event loop if a running one isn't found.

However, according to the docs referenced above, this should be done with asyncio.get_running_loop().create_future() which would raise a RuntimeError if there is no running event loop found.

If the current behavior of creating a new loop if one does not exist is the intended behavior, it should be done explicitly by catching the RuntimeError and calling asyncio.new_event_loop()

By treating the warning as an error, I was able to get a stacktrace

  File "/lib/python3.10/site-packages/nbconvert/preprocessors/execute.py", line 86, in preprocess
    info_msg = self.wait_for_reply(self.kc.kernel_info())
  File "/lib/python3.10/site-packages/jupyter_client/asynchronous/client.py", line 16, in _
    msg_id = meth(self, *args, **kwargs)
  File "/lib/python3.10/site-packages/jupyter_client/client.py", line 742, in kernel_info
    self.shell_channel.send(msg)
  File "/lib/python3.10/site-packages/jupyter_client/channels.py", line 263, in send
    self.session.send(self.socket, msg)
  File "/lib/python3.10/site-packages/jupyter_client/session.py", line 855, in send
    stream.send_multipart(to_send, copy=copy)
  File "/lib64/python3.10/site-packages/zmq/_future.py", line 322, in send_multipart
    return self._add_send_event('send_multipart', msg=msg_parts, kwargs=kwargs)
  File "/lib64/python3.10/site-packages/zmq/_future.py", line 498, in _add_send_event
    f = future or self._Future()
DeprecationWarning: There is no current event loop

[minrk]

A library should never propagate a deprecation warning.

I don't mean that we should propagate it programmatically. I just mean that the code relying on deprecated behavior is not in pyzmq, it's in jupyter-client, but the warning is misleading about which code is responsible due to an implementation detail.

This is an indicator the the way the code is being called will break in a future release.

Yes, and the code that will break is in jupyter_client, not in pyzmq. Jupyter client is currently using async methods without an event loop, and that's deprecated by asyncio. pyzmq just happens to be an intermediate layer that instantiates the Future for this async method using a not deprecated method of asyncio.

pyzmq's not using any deprecated APIs here. What's deprecated is the context in which it's called, and that's not up to pyzmq.

I think it's an accident in jupyter-client to be passing async sockets to session.send at all, and that's being cleaned up in jupyter/jupyter_client#835.

If the current behavior of creating a new loop if one does not exist is the intended behavior, it should be done explicitly by catching the RuntimeError and calling asyncio.new_event_loop()

The intended behavior is to follow asyncio's own behavior (deprecate implicit loop creation which will eventually be a RuntimeError in a later Python version), which pyzmq follows exactly.

If there were a change to make in pyzmq, it would be to show the same deprecation warning in the same circumstances with the same behavior, so that the code actually responsible gets the warning about asyncio's deprecated behavior. Raising a RuntimeError would be introducing a breaking change that Python already decided should be brought in slowly via a deprecation. Implicitly creating the loop is also established as a problematic pattern that should be removed, but cannot be removed immediately because it is relied upon, hence the deprecation. So I'm not sure of the benefit of recreating an existing deprecation message that's already there under the right circumstances, and not a deprecation in pyzmq at all, but a deprecation in Python itself.

[avylove]

Thanks for the detailed response.

If it were me, I'd use asyncio.get_running_loop().create_future() (with some additional logic for Python <3.7) instead of implicitly getting the event loop with Future(). Then I'd catch the runtime error, raise a deprecation warning, and fall back to new_event_loop(). Then, on the next major version, drop the fallback.

The advantage here is you can create a clear warning message specific to zmq users and target the unsupported behavior rather than wait for it to work it's way out over the years through Python upgrades. I can see the advantage of leveraging the Python deprecation too, it's just not the way I would go.

As for me, this is only used in testing and only on the latest stable version, so I'd expect, by the time it would break, jupyter_client would have made sure their code works on the latest version.

[minrk]

Thanks for your perspective. I think there's value in getting the same warning for all instances of the same deprecated behavior, since that improves searchability. I think there's also value in that deprecation being tied to just one version of one package (Python itself), rather than the same deprecation occurring in two different versions of two different packages. This kind of situation is not so uncommon in asyncio where it's the context not the calling code that's responsible and needs to change, so the library triggering deprecated behavior gets misplaced fingers pointed at it.