Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-120012: clarify the behaviour of multiprocessing.Queue.empty on closed queues. #120102

Merged
merged 10 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Doc/library/multiprocessing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -837,6 +837,10 @@ For an example of the usage of queues for interprocess communication see
Return ``True`` if the queue is empty, ``False`` otherwise. Because of
multithreading/multiprocessing semantics, this is not reliable.

If :meth:`put` or :meth:`put_nowait` has been called previously (i.e.,
picnixz marked this conversation as resolved.
Show resolved Hide resolved
if the feeder thread has been started) and if the queue is closed, this
method raises an :exc:`OSError` instead.

.. method:: full()

Return ``True`` if the queue is full, ``False`` otherwise. Because of
Expand Down Expand Up @@ -940,6 +944,10 @@ For an example of the usage of queues for interprocess communication see

Return ``True`` if the queue is empty, ``False`` otherwise.

Unlike :meth:`Queue.empty`, if the queue is closed and this
method is called, this raises an :exc:`OSError` even if the
queue was never used.

.. method:: get()

Remove and return an item from the queue.
Expand Down
19 changes: 19 additions & 0 deletions Lib/test/_test_multiprocessing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1332,6 +1332,19 @@ def _on_queue_feeder_error(e, obj):
self.assertTrue(not_serializable_obj.reduce_was_called)
self.assertTrue(not_serializable_obj.on_queue_feeder_error_was_called)

def test_closed_queue_empty_exceptions(self):
picnixz marked this conversation as resolved.
Show resolved Hide resolved
for q in multiprocessing.Queue(), multiprocessing.JoinableQueue():
q.close() # this is a no-op since the feeder thread is None
q.join_thread() # this is also a no-op
self.assertTrue(q.empty())

for q in multiprocessing.Queue(), multiprocessing.JoinableQueue():
q.put('foo')
q.close() # close the feeder thread
q.join_thread() # make sure to join the feeder thread
with self.assertRaisesRegex(OSError, 'is closed'):
q.empty()

def test_closed_queue_put_get_exceptions(self):
for q in multiprocessing.Queue(), multiprocessing.JoinableQueue():
q.close()
Expand Down Expand Up @@ -5815,6 +5828,12 @@ def _test_empty(cls, queue, child_can_start, parent_can_continue):
finally:
parent_can_continue.set()

def test_empty_exceptions(self):
q = multiprocessing.SimpleQueue()
q.close() # close the pipe
with self.assertRaisesRegex(OSError, 'is closed'):
q.empty()

def test_empty(self):
queue = multiprocessing.SimpleQueue()
child_can_start = multiprocessing.Event()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Clarify the behaviour of :meth:`multiprocessing.Queue.empty` on closed
picnixz marked this conversation as resolved.
Show resolved Hide resolved
queues. Patch by Bénédikt Tran.
Loading