v3.13.0rc2 Segmentation Fault in gc.get_referents. PyCapsule->traverse_func is NULL

[justinjhendrick]

Crash report

What happened?

Hello! Thank you for all your work on CPython. Hopefully this crash report is helpful.

While working on matplotlib/matplotlib#28861, we noticed that our tests were segfaulting, and only on python3.13. Here is a minimal reproduction of that crash. I've also run it with python3.13-dbg and -X dev to hopefully provide more details.

Error Message

python: ../Objects/capsule.c:321: capsule_traverse: Assertion `capsule->traverse_func != NULL' failed.

Minimal Reproducible Example:

$ python3.13 -m venv venv_313
$ source venv_313/bin/activate
$ pip install matplotlib==3.9.2
$ python -X dev test.py

test.py

import gc
from collections import deque
from matplotlib.figure import Figure
from matplotlib.lines import Line2D
from matplotlib.collections import PathCollection

def get_children(parent):
    if parent.__class__.__qualname__ == "PyCapsule":
        print(f"About to call gc.get_referents on {repr(parent)}")
    children = gc.get_referents(parent) # gc.get_referrers does not segfault, but gc.get_referents does
    if parent.__class__.__qualname__ == "PyCapsule":
        print(f"There are {len(children)} children")
    return children

def breadth_first_search(start):
    to_visit = deque([start])
    explored = set()
    try:
        gc.disable()  # Can repro without this, but it should make behavior more consistent
        while len(to_visit) > 0:
            parent = to_visit.popleft()
            for child in get_children(parent):
                if id(child) in explored:
                    continue
                explored.add(id(child))
                to_visit.append(child)
    finally:
        gc.enable()

def find_reference_cycles():
    fig = Figure()
    ax = fig.add_subplot()
    ax.plot([1])
    fig.clear()
    breadth_first_search(fig)

if __name__ == "__main__":
    find_reference_cycles()

with python3.13 on Ubuntu 22.04.4

$ python -VV
Python 3.13.0rc2 (main, Sep  9 2024, 22:55:42) [GCC 11.4.0]
$ python -X dev test.py
About to call gc.get_referents on <capsule object "pybind11_function_record_capsule" at 0x7f79954f34d0>
Fatal Python error: Segmentation fault

Current thread 0x00007f79e687f280 (most recent call first):
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 10 in get_children
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 22 in breadth_first_search
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 35 in find_reference_cycles
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 38 in <module>

Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, PIL._imaging, kiwisolver._cext (total: 4)
Segmentation fault (core dumped)

with python3.13-dbg on Ubuntu 22.04.4:

$ python -VV
Python 3.13.0rc2 (main, Sep  9 2024, 22:55:42) [GCC 11.4.0]
$ python -X dev test.py
About to call gc.get_referents on <capsule object "pybind11_function_record_capsule" at 0x7f0c31adba80>
python: ../Objects/capsule.c:321: capsule_traverse: Assertion `capsule->traverse_func != NULL' failed.
Fatal Python error: Aborted

Current thread 0x00007f0c82d1a280 (most recent call first):
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 10 in get_children
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 22 in breadth_first_search
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 35 in find_reference_cycles
  File "/home/justin/tmp/matplotlib_py313_segfault/test.py", line 38 in <module>

Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, PIL._imaging, kiwisolver._cext (total: 4)
Aborted (core dumped)

It does not crash with python3.12 on Ubuntu 22.04.4

$ python -VV
Python 3.12.6 (main, Sep 10 2024, 00:05:17) [GCC 11.4.0]

$ python -X dev test.py
About to call gc.get_referents on <capsule object "pybind11_function_record_capsule" at 0x7f85a852a570>
There are 0 children
[ ... many similar log lines removed ... ]

Side-note:
I tested this on both python3.12 and 3.13, but not the main branch. The Github Issue dropdown doesn't have a 3.13rc2 option for me to check.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0rc2 (main, Sep 9 2024, 22:55:42) [GCC 11.4.0]

Linked PRs

• gh-124538: Fix crash when using gc.get_referents on an untracked capsule object #124559
• [3.13] gh-124538: Fix crash when using gc.get_referents on an untracked capsule object (GH-124559) #124588

[ZeroIntensity]

I'll take a look at this soon. At a glance, we just need to have capsule_traverse do nothing if _PyCapsule_SetTraverse wasn't called (under general circumstances, capsule_traverse shouldn't ever be called if _PyCapsule_SetTraverse never was, but calling gc.get_referents forces that.)

[ZeroIntensity]

This is a 3.13 regression! Looks like the bad commit was #108339. It's a pretty simple fix though.

[zware]

The Github Issue dropdown doesn't have a 3.13rc2 option for me to check.

Sorry about that! That will be fixed by GH-124560.

[ZeroIntensity]

Thank you for the report, @justinjhendrick!