Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_interpreters fails when running tests sequentially #115490

Closed
Yhg1s opened this issue Feb 14, 2024 · 3 comments
Closed

test_interpreters fails when running tests sequentially #115490

Yhg1s opened this issue Feb 14, 2024 · 3 comments
Assignees
Labels
topic-subinterpreters type-bug An unexpected behavior, bug, or error

Comments

@Yhg1s
Copy link
Member

Yhg1s commented Feb 14, 2024

Bug report

Bug description:

test_interpreters fails when running tests sequentially (which we do as part of the release process). The easiest reproducer seems to be python -m test test_interpreters test_interpreters.test_channels:

% bin/python -m test test_interpreters test_interpreters.test_channels
0:00:00 load avg: 0.86 [1/2] test_interpreters
0:00:02 load avg: 0.87 [2/2] test_interpreters.test_channels
test test_interpreters.test_channels crashed -- Traceback (most recent call last):
  File "/tmp/testinstall/lib/python3.13/test/libregrtest/single.py", line 178, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/tmp/testinstall/lib/python3.13/test/libregrtest/single.py", line 125, in _load_run_test
    test_mod = importlib.import_module(module_name)
               ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/tmp/testinstall/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1014, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/tmp/testinstall/lib/python3.13/test/test_interpreters/test_channels.py", line 10, in <module>
    from test.support.interpreters import channels
  File "/tmp/testinstall/lib/python3.13/test/support/interpreters/channels.py", line 171, in <module>
    _channels._register_end_types(SendChannel, RecvChannel)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: already registered

The cause is libregrtest's behaviour of unloading newly imported modules from sys.modules after each test run. Either test.support.interpreters.channels should support being imported multiple times (perhaps by not doing any setup at import time, but via deliberate calls?), or libregrtest should be taught to skip unloading test.support.interpreters modules.

@ericsnowcurrently, since this seems to be your area, any opinion making test.support.interpreters do the right thing here?

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

@ericsnowcurrently
Copy link
Member

Yeah, I see what the problem is.

@ericsnowcurrently
Copy link
Member

gh-115493 should take care of it. (I double-checked locally and it fixed it for me.)

Yhg1s added a commit that referenced this issue Feb 15, 2024
…g unloading (#115515)

Work around test.support.interpreters.channels not handling unloading, which
regrtest does when running tests sequentially, by explicitly skipping the
unloading of test.support.interpreters and its submodules.

This can be rolled back once test.support.interpreters.channels supports
unloading, if we are keeping sequential runs in the same process around.
@encukou
Copy link
Member

encukou commented Feb 19, 2024

With the workaround, this is no longer a blocker.

ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue Mar 4, 2024
ericsnowcurrently added a commit that referenced this issue Mar 4, 2024
…es Handle Reloading Properly (gh-115493)

The problem manifested when the .py module got reloaded and the corresponding extension module didn't. The .py module registers types with the extension and the extension was not allowing that to happen more than once. The solution: let it happen more than once.
@github-project-automation github-project-automation bot moved this from Todo to Done in Subinterpreters Mar 4, 2024
adorilson pushed a commit to adorilson/cpython that referenced this issue Mar 25, 2024
… Modules Handle Reloading Properly (pythongh-115493)

The problem manifested when the .py module got reloaded and the corresponding extension module didn't. The .py module registers types with the extension and the extension was not allowing that to happen more than once. The solution: let it happen more than once.
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
… Modules Handle Reloading Properly (pythongh-115493)

The problem manifested when the .py module got reloaded and the corresponding extension module didn't. The .py module registers types with the extension and the extension was not allowing that to happen more than once. The solution: let it happen more than once.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-subinterpreters type-bug An unexpected behavior, bug, or error
Projects
Status: Done
Development

No branches or pull requests

3 participants