-
-
Notifications
You must be signed in to change notification settings - Fork 348
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix leaking file descriptors when subprocess creation fails #2193
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2193 +/- ##
==========================================
+ Coverage 99.51% 99.55% +0.04%
==========================================
Files 114 114
Lines 14759 14778 +19
Branches 2344 2344
==========================================
+ Hits 14687 14712 +25
+ Misses 47 44 -3
+ Partials 25 22 -3
|
So this is my best suggestion for testing this behavior. Happy to discuss alternatives! |
If subprocess spawning fails (i.e. unsuccessful call to `exec`), the created trio resources (i.e. the trio ends of the pipes that will be used to send/receive data to the child) don't get cleaned up until the next garbage collection run (which will call their `__del__`). It would be nicer if this was deterministic, and that resources created during the run of `open_process` were either completely transferred to the `Process` object (which can take over the management of them) or cleaned up (on failure). To implement this, two ExitStacks are used: One will manage the resources that need cleaned up unconditionally (the child ends of the pipe); the other will manage the resources that need cleaned up on failure. Trio currently supports versions of python that lack `contextlib.AsyncExitStack`. While there is a backport for 3.6 available on PyPI, the objects that represent the pipes (_FdHolder and _HandleHolder) don't really need to have an async close. They are internal implementation details anyway, so this refactors their implementation to have the Stream/AsyncResource methods on the actual Stream objects themselves. This allows us to use the regular ExitStack and avoid an extra dep
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Realistically I don't see any issue with just using /dev/fds
because I don't see any actual possibility for the closing behaviour to change on Windows.
If subprocess spawning fails (i.e. unsuccessful call to
exec
), thecreated trio resources (i.e. the trio ends of the pipes that will be
used to send/receive data to the child) don't get cleaned up until the
next garbage collection run (which will call their
__del__
).It would be nicer if this was deterministic, and that resources
created during the run of
open_process
were either completelytransferred to the
Process
object (which can take over themanagement of them) or cleaned up (on failure).
To implement this, two ExitStacks are used: One will manage the
resources that need cleaned up unconditionally (the child ends of the
pipe); the other will manage the resources that need cleaned up on
failure.
Trio currently supports versions of python that lack
contextlib.AsyncExitStack
. While there is a backport for 3.6available on PyPI, the objects that represent the pipes (_FdHolder and
_HandleHolder) don't really need to have an async close. They are
internal implementation details anyway, so this refactors their
implementation to have the Stream/AsyncResource methods on the actual
Stream objects themselves. This allows us to use the regular ExitStack
and avoid an extra dep