Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all deprecation warnings are disabled for the whole process while new unix asyncio subprocesses are created #98327

Closed
graingert opened this issue Oct 16, 2022 · 7 comments
Labels
3.12 bugs and security fixes topic-asyncio

Comments

@graingert
Copy link
Contributor

graingert commented Oct 16, 2022

on python3.10 the following prints:

python3.10 demo.py 
/home/graingert/projects/cpython/demo.py:40: DeprecationWarning: demo warning
  warnings.warn("demo warning", DeprecationWarning)
/home/graingert/projects/cpython/demo.py:48: DeprecationWarning: demo warning2
  warnings.warn("demo warning2", DeprecationWarning)

but on python main it prints:

./python demo.py      
/home/graingert/projects/cpython/demo.py:40: DeprecationWarning: demo warning
  warnings.warn("demo warning", DeprecationWarning)
import sys
import threading
import asyncio
import concurrent.futures
import warnings


async def acreate_process_then_sleep(started_event, stop_event):
    event = asyncio.Event()

    async def create_process():
        loop = asyncio.get_running_loop()
        loop.call_soon(event.set)
        proc = await asyncio.create_subprocess_exec(
            sys.executable,
            "-c",
            "print('hello')",
            stdin=None,
            stdout=asyncio.subprocess.PIPE,
            stderr=asyncio.subprocess.STDOUT,
        )
        await proc.communicate()

    async def sleep():
        await event.wait()
        started_event.set()
        stop_event.wait()

    await asyncio.gather(create_process(), sleep())


def create_process_then_sleep(*args, **kwargs):
    asyncio.run(acreate_process_then_sleep(*args, **kwargs))


def main():
    with concurrent.futures.ThreadPoolExecutor() as tpe:
        stop_event = threading.Event()
        started_event = threading.Event()
        warnings.warn("demo warning", DeprecationWarning)
        fut = tpe.submit(
            create_process_then_sleep,
            started_event=started_event,
            stop_event=stop_event,
        )
        try:
            started_event.wait()
            warnings.warn("demo warning2", DeprecationWarning)
        finally:
            stop_event.set()
            fut.result()


if __name__ == "__main__":
    sys.exit(main())

Originally posted by @graingert in #98215 (comment)

@graingert graingert added 3.12 bugs and security fixes topic-asyncio labels Oct 16, 2022
@graingert
Copy link
Contributor Author

cc @kumaraditya303

@ezio-melotti ezio-melotti moved this to Todo in asyncio Oct 16, 2022
@graingert graingert changed the title all deprecation warnings are disabled for the whole process while new unix asyncio processes are created all deprecation warnings are disabled for the whole process while new unix asyncio subprocesses are created Oct 16, 2022
@graingert
Copy link
Contributor Author

graingert commented Oct 16, 2022

see also #91505, #81785 and #50896

@kumaraditya303
Copy link
Contributor

Okay, I'll be AFK for some time so feel free to create a PR. I doubt that any one will hit this in practise though.

@gvanrossum
Copy link
Member

Ah, I guess I was right about that context manager. :-)

@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Oct 16, 2022

#98333 fixes this, I had underestimated warnings module :-)

gvanrossum pushed a commit that referenced this issue Oct 17, 2022
…ort (#98333)

Alas, warnings.catch_warnings() has global scope, not thread scope, so this is still not perfect, but it reduces the time during which warnings are ignored. Better solution welcome.
@gvanrossum
Copy link
Member

We're still looking for a perfect solution (see discussion in PR), so keeping this issue open.

carljm added a commit to carljm/cpython that referenced this issue Oct 17, 2022
* main: (31 commits)
  pythongh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (pythonGH-98345)
  pythongh-95914: Add What's New item describing PEP 670 changes (python#98315)
  Remove unused arrange_output_buffer function from zlibmodule.c. (pythonGH-98358)
  pythongh-98174: Handle EPROTOTYPE under macOS in test_sendfile_fallback_close_peer_in_the_middle_of_receiving (python#98316)
  pythonGH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (python#98333)
  pythongh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (pythonGH-98001)
  pythongh-97669: Create Tools/build/ directory (python#97963)
  pythongh-95534: Improve gzip reading speed by 10% (python#97664)
  pythongh-95913: Forward-port int/str security change to 3.11 What's New in main (python#98344)
  pythonGH-91415: Mention alphabetical sort ordering in the Sorting HOWTO (pythonGH-98336)
  pythongh-97930: Merge with importlib_resources 5.9 (pythonGH-97929)
  pythongh-85525: Remove extra row in doc (python#98337)
  pythongh-85299: Add note warning about entry point guard for asyncio example (python#93457)
  pythongh-97527: IDLE - fix buggy macosx patch (python#98313)
  pythongh-98307: Add docstring and documentation for SysLogHandler.createSocket (pythonGH-98319)
  pythongh-94808: Cover `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule` (python#98158)
  pythonGH-94597: Deprecate child watcher getters and setters (python#98215)
  pythongh-98254: Include stdlib module names in error messages for NameErrors (python#98255)
  Improve speed. Reduce auxiliary memory to 16.6% of the main array. (pythonGH-98294)
  [doc] Update logging cookbook with an example of custom handling of levels. (pythonGH-98290)
  ...
@kumaraditya303
Copy link
Contributor

kumaraditya303 commented May 11, 2023

I'm closing this as done with good enough, there are plenty of places in stdlib where this happens and in those places some real work in done rather than just a function call. No point in spending more time on this.

@github-project-automation github-project-automation bot moved this from Todo to Done in asyncio May 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 bugs and security fixes topic-asyncio
Projects
Status: Done
Development

No branches or pull requests

3 participants