Detect GIL on python 3.12+

[benfred]

Python 3.12 moved from having the GIL held in _PyRuntime.tstate_current to being in thread local storage #633 (comment) . This means that we currently can't detect which thread is holding the GIL

[ddelange]

also worth mentioning: deadsnakes is already building python3.13-nogil, meaning you'll have a /usr/bin/python3.13-nogil executable which will run with GIL disabled

on the nogil build you can turn it on and off for instance via env var ref https://github.com/python/cpython/blob/v3.13.0/Lib/test/test_cmd_line.py#L882

add-apt-repository ppa:deadsnakes/ppa
apt-get update
apt-get install python3.13-nogil

[olejorgenb]

To check my understanding - this means that results can be quite skewed in multithreaded programs right where more than one thread does significant work? If two threads are working in function f and g respectively both f and g will accumulate work/samples each time py-spy samples?

[benfred]

To check my understanding - this means that results can be quite skewed in multithreaded programs right where more than one thread does significant work? If two threads are working in function f and g respectively both f and g will accumulate work/samples each time py-spy samples?

No - the idle detection code in py-spy should prevent skewing the results here . Not having GIL support in python 3.12 means that we can't use the --gil flag in py-spy to only profile threads with the GIL - not that the results will be less accurate in multi-threaded programs when not using the --gil flag

[benfred]

@ddelange I'm excited to try out the deadsnakes nogil option =)

[zanieb]

Similarly, uv supports installing freethreaded Python distributions that could be used for testing here.

[olejorgenb]

No - the idle detection code in py-spy should prevent skewing the results here . Not having GIL support in python 3.12 means that we can't use the --gil flag in py-spy to only profile threads with the GIL - not that the results will be less accurate in multi-threaded programs when not using the --gil flag

Ah, that's great! So the only downside is more sampling overhead in some cases 👌

[benfred]

Fwiw - fixing this turned out to be easier than I initially thought (We can read the gil_runtimestate from the PyInterpreterState directly), and this feature will be much easier to maintain for future python releases