gh-104324: Reinstate GIL cleanup during interpreter deletion. #104325
Conversation
|
Most changes to Python require a NEWS entry. Please add it using the blurb_it web app or the blurb command-line tool. |
|
All commit authors signed the Contributor License Agreement. |
|
I was lulled into a false sense of security by the comments on I checked the Azure failure and it's in The daemon worker threads are blowing up inside int drop_requested = 0;
while (_Py_atomic_load_relaxed(&gil->locked)) {
unsigned long saved_switchnum = gil->switch_number;
unsigned long interval = (gil->interval >= 1 ? gil->interval : 1);
int timed_out = 0;
COND_TIMED_WAIT(gil->cond, gil->mutex, interval, timed_out);
/* If we timed out and no switch occurred in the meantime, it is time
to ask the GIL-holding thread to drop it. */
if (timed_out &&
_Py_atomic_load_relaxed(&gil->locked) &&
gil->switch_number == saved_switchnum)
{
if (tstate_must_exit(tstate)) {
MUTEX_UNLOCK(gil->mutex);
// gh-96387: If the loop requested a drop request in a previous
// iteration, reset the request. Otherwise, drop_gil() can
// block forever waiting for the thread which exited. Drop
// requests made by other threads are also reset: these threads
// may have to request again a drop request (iterate one more
// time).
if (drop_requested) {
RESET_GIL_DROP_REQUEST(interp);
}
PyThread_exit_thread();
}
assert(is_tstate_valid(tstate));
SET_GIL_DROP_REQUEST(interp);
drop_requested = 1;
}
}as the main thread has torn down the CS used as the GIL mutex after the worker's been woken, but before it tries to re-acquire the mutex: I can see how the idea of deferring the deletion of the GIL mutex/condition variable to the next In the case where you do: Py_Initialize();
<create daemon thread>;
Py_FinalizeEx();
Py_Initialize();Say (as per the above worker callstack), a thread is signalled but then doesn't get to run until the main thread has called Now, ignoring the issue of whether It seems to me that the mutex and condition variable need to be allocated in some form of context which can be reference-counted. Daemon threads could own a reference to this context and then when they detect (via There is also a need to be able to call Do you think this is a viable way forward, in order to resolve this crash? |
Reinstate GIL cleanup during interpreter deletion, after all threads (including daemons, as per bpo-9901) have been destroyed.
This ensures the GIL is correctly destroyed before being re-initialised.