gh-120496: Make enum_iter thread safe

[eendebakpt]

We make enum_iter thread-safe using a critical section. We use the same approach as in #119438 to allow for exits in the middle of the critical section. The method enum_iter_long is guarded by the critical section from enum_long.

Without making enum_iter thread safe problems can occur:

• For enumerate(range(10)) pairs (n, m) with n unqual to m can be generated
• When iterating over sys.maxsize, e.g. enumerate(range(sys.maxsize - 10, sys.maxsize + 10)) an overflow can occur.
• As an optimization enum_iter keeps track of the returned tuple. In the free-threaded build multiple threads can operate on the same tuple.

To determine the overhead of making enumerate thread-safe here are some benchmarking results for a single-threaded application. Performance when actually using multiple-threads to readout the enumerate is considered less important.

Benchmark script:

import pyperf
runner = pyperf.Runner()

setup = """

def weighted_sum(k):
    value = 0
    for n, m in enumerate(k):
        value += n * m 
    return value

range10 = range(10)
range1000 = range(1000)
x = list(range(10))
"""

runner.timeit(name="enumerate(range10)", stmt="enumerate(range10)", setup=setup)
runner.timeit(name="list(enumerate(range10))", stmt="list(enumerate(range10))", setup=setup)
runner.timeit(name="list(enumerate(range1000))", stmt="list(enumerate(range1000))", setup=setup)
runner.timeit(name="weighted_sum", stmt="weighted_sum(x)", setup=setup)

Results of main versus free-threading:

enumerate(range10): Mean +- std dev: [enumerate_main] 73.2 ns +- 3.5 ns -> [enumerate_ft] 94.1 ns +- 3.5 ns: 1.29x slower
list(enumerate(range10)): Mean +- std dev: [enumerate_main] 292 ns +- 10 ns -> [enumerate_ft] 351 ns +- 19 ns: 1.20x slower
list(enumerate(range1000)): Mean +- std dev: [enumerate_main] 31.1 us +- 1.6 us -> [enumerate_ft] 32.5 us +- 1.4 us: 1.04x slower
weighted_sum: Mean +- std dev: [enumerate_main] 433 ns +- 17 ns -> [enumerate_ft] 938 ns +- 49 ns: 2.16x slower

Geometric mean: 1.37x slower

Results of free-threading vs. free-threading with this PR:

list(enumerate(range10)): Mean +- std dev: [enumerate_ft] 351 ns +- 19 ns -> [enumerate_ft_pr] 410 ns +- 18 ns: 1.17x slower
list(enumerate(range1000)): Mean +- std dev: [enumerate_ft] 32.5 us +- 1.4 us -> [enumerate_ft_pr] 40.9 us +- 1.7 us: 1.26x slower

Benchmark hidden because not significant (2): enumerate(range10), weighted_sum

Geometric mean: 1.10x slower

The case enumerate(range10) is not affected by the PR (used to check the benchmarking is stable). The cases list(enumerate(range(x))) slow down a bit. More representative is perhaps the weighted_sum benchmark where the enumerate is used a for loop with a minimal amount of work. There the overhead of the locking not significant.

• Issue: Sequence iterator thread-safety #120496

[eendebakpt]

Since we hold the only references to result this can be outside the critical section.

[eendebakpt]

Closing in favor of #125734