Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current Thread #105109

Conversation

ericsnowcurrently
Copy link
Member

@ericsnowcurrently ericsnowcurrently commented May 30, 2023

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

This is a much simpler approach to solving the race than in other PRs I've posted. I'd still like to pursue some of those other ideas but that can be done separately for 3.13+.

(The idea for this approach came out of discussions with @markshannon.)

Python/ceval_gil.c Show resolved Hide resolved
@@ -625,10 +632,12 @@ _PyEval_AcquireLock(PyThreadState *tstate)
}

void
_PyEval_ReleaseLock(PyThreadState *tstate)
_PyEval_ReleaseLock(PyInterpreterState *interp, PyThreadState *tstate)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other key part of this change is here, where we pass in the interpreter separately from the thread state, which allows tstate to be NULL.

@ericsnowcurrently ericsnowcurrently marked this pull request as ready for review May 30, 2023 21:10
Python/ceval_gil.c Show resolved Hide resolved
Python/pystate.c Outdated Show resolved Hide resolved
@@ -278,6 +278,10 @@ static void recreate_gil(struct _gil_runtime_state *gil)
static void
drop_gil(struct _ceval_state *ceval, PyThreadState *tstate)
{
/* We shouldn't be using a thread state that isn't viable any more. */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is cryptic here. It doesn't say why it's here nor in which situation the "non-viable thread state" occurs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clarified the comment.

@ericsnowcurrently
Copy link
Member Author

@gpshead, @pitrou, do you have any objections to this PR at this point? I'd like to land this ASAP for beta 2. Thanks!

@gpshead
Copy link
Member

gpshead commented Jun 1, 2023

any objections to this PR at this point? I'd like to land this ASAP for beta 2. Thanks!

I think we're good to go: no objections from me. I never assume I can have full confidence about modifications to this code, but CI and buildbots and ultimately beta2 testing should reveal more if anything else lurks...

@ericsnowcurrently
Copy link
Member Author

Thanks you!

@ericsnowcurrently ericsnowcurrently merged commit 3698fda into python:main Jun 1, 2023
@miss-islington
Copy link
Contributor

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry @ericsnowcurrently, I had trouble checking out the 3.12 backport branch.
Please retry by removing and re-adding the "needs backport to 3.12" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 3698fda06eefb3c01e78c4c07f46fcdd0559e0f6 3.12

@ericsnowcurrently ericsnowcurrently deleted the fix-threading-module-shutdown-3.12 branch June 1, 2023 22:24
@ericsnowcurrently ericsnowcurrently added needs backport to 3.12 bug and security fixes and removed needs backport to 3.12 bug and security fixes labels Jun 1, 2023
@miss-islington
Copy link
Contributor

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 1, 2023
… the Current Thread (pythongh-105109)

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

(The idea for this approach came out of discussions with @markshannon.)
(cherry picked from commit 3698fda)

Co-authored-by: Eric Snow <[email protected]>
@bedevere-bot
Copy link

GH-105209 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 bug and security fixes label Jun 1, 2023
ericsnowcurrently pushed a commit that referenced this pull request Jun 1, 2023
…g the Current Thread (gh-105109) (gh-105209)

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

(The idea for this approach came out of discussions with @markshannon.)
(cherry picked from commit 3698fda)

Co-authored-by: Eric Snow [email protected]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants