Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_peg_generator and test_freeze require cpu #108386

Merged
merged 1 commit into from
Aug 23, 2023

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Aug 23, 2023

The test_peg_generator and test_tools.test_freeze tests now require the 'cpu' resource. Skip these tests unless the 'cpu' resource is enabled (it is disabled by default).

These tests are no longer skipped if Python is built with ASAN or MSAN sanitizer.

The test_peg_generator and test_tools.test_freeze tests now require
the 'cpu' resource. Skip these tests unless the 'cpu' resource is
enabled (it is disabled by default).

These tests are no longer skipped if Python is built with ASAN or
MSAN sanitizer.
@vstinner vstinner changed the title test_peg_generator and test_freeze requires cpu test_peg_generator and test_freeze require cpu Aug 23, 2023
@vstinner
Copy link
Member Author

Comparision, total timing BEFORE => AFTER:

  • Windows x86: 22 min 20 sec => 19 min 26 sec (- 3 min)
  • Windows x64: 19 min 28 sec => 30 min 34 sec (ignore this one, test_concurrent_futures was re-run!)
  • macOS: 22 min 13 sec => 17 min 34 sec (- 5 min)
  • Ubuntu: 15 min 2 sec => 12 min 18 sec (-3 min)
  • Address Sanitizer: 15 min 38 sec => 15 min 24 sec (same)

The exact timing depends when slowest tests are run. If slowest tests are run at the end, the difference of this PR is less.

GHA jobs timing without this PR

Use #108383 as reference but use #108381 for Windows x86 (since in the other PR, test_concurrent_futures was re-run!).

10 slowest tests:
- test_multiprocessing_spawn: 6 min 7 sec
- test_peg_generator: 4 min 15 sec
- test_importlib: 2 min 45 sec
- test_concurrent_futures: 2 min 16 sec
- test_compileall: 2 min 9 sec
- test_venv: 1 min 49 sec
- test_regrtest: 1 min 46 sec
- test_tarfile: 1 min 41 sec
- test_socket: 1 min 41 sec
- test_fstring: 1 min 27 sec

Total duration: 22 min 20 sec

Windows x64:

10 slowest tests:
- test_peg_generator: 7 min 5 sec
- test_multiprocessing_spawn: 5 min 2 sec
- test_importlib: 3 min 24 sec
- test_compileall: 2 min 16 sec
- test_concurrent_futures: 2 min 13 sec
- test_regrtest: 2 min 9 sec
- test_venv: 1 min 47 sec
- test_tarfile: 1 min 41 sec
- test_subprocess: 1 min 35 sec
- test_socket: 1 min 30 sec

Total duration: 19 min 28 sec

macOS:

10 slowest tests:
- test_tools: 10 min 51 sec
- test_multiprocessing_spawn: 6 min 20 sec
- test_concurrent_futures: 3 min 45 sec
- test_tarfile: 2 min 41 sec
- test_signal: 2 min 40 sec
- test_pickle: 2 min 30 sec
- test_multiprocessing_forkserver: 2 min 17 sec
- test_cppext: 1 min 45 sec
- test_venv: 1 min 27 sec
- test_regrtest: 1 min 25 sec

Total duration: 22 min 13 sec

Ubuntu:

10 slowest tests:
- test_tools: 9 min 7 sec
- test_multiprocessing_spawn: 3 min 24 sec
- test_concurrent_futures: 3 min 4 sec
- test_gdb: 2 min 51 sec
- test_multiprocessing_forkserver: 1 min 51 sec
- test_multiprocessing_fork: 1 min 27 sec
- test_venv: 1 min 17 sec
- test_subprocess: 1 min 7 sec
- test_peg_generator: 1 min 4 sec
- test_socket: 1 min 4 sec

Total duration: 15 min 2 sec

Address Sanitizer:

10 slowest tests:
- test_subprocess: 2 min 42 sec
- test_compileall: 2 min 21 sec
- test_venv: 2 min 17 sec
- test_tarfile: 1 min 49 sec
- test_gdb: 1 min 33 sec
- test_pickle: 1 min 29 sec
- test_zipfile: 1 min 22 sec
- test_weakref: 1 min 18 sec
- test_multiprocessing_main_handling: 1 min 12 sec
- test_decimal: 1 min 10 sec

Total duration: 15 min 38 sec

GHA jobs timing with the PR

Windows x86:

10 slowest tests:
- test_multiprocessing_spawn: 7 min 43 sec
- test_compileall: 2 min 32 sec
- test_concurrent_futures: 2 min 24 sec
- test_venv: 2 min 2 sec
- test_fstring: 1 min 35 sec
- test_tarfile: 1 min 25 sec
- test_regrtest: 1 min 24 sec
- test_io: 1 min 15 sec
- test_pickle: 1 min 13 sec
- test_zipfile: 1 min 10 sec

Total duration: 19 min 26 sec

Windows x64 (test_concurrent_futures failed and had to be re-run!):

10 slowest tests:
- test_multiprocessing_spawn: 7 min 19 sec
- test_tarfile: 2 min 6 sec
- test_compileall: 1 min 53 sec
- test_venv: 1 min 50 sec
- test_importlib: 1 min 43 sec
- test_io: 1 min 33 sec
- test_zipfile: 1 min 31 sec
- test_mmap: 1 min 29 sec
- test_socket: 1 min 24 sec
- test_regrtest: 1 min 15 sec

Total duration: 30 min 34 sec

macOS:

10 slowest tests:
- test_multiprocessing_spawn: 6 min 14 sec
- test_concurrent_futures: 3 min 50 sec
- test_multiprocessing_forkserver: 2 min 47 sec
- test_tarfile: 2 min 5 sec
- test_signal: 1 min 48 sec
- test_decimal: 1 min 22 sec
- test_subprocess: 1 min 21 sec
- test_compileall: 1 min 21 sec
- test_regrtest: 1 min 20 sec
- test_cppext: 1 min 15 sec

Total duration: 17 min 34 sec

Ubuntu:

10 slowest tests:
- test_multiprocessing_spawn: 3 min 56 sec
- test_concurrent_futures: 3 min 5 sec
- test_gdb: 2 min 50 sec
- test_multiprocessing_forkserver: 1 min 57 sec
- test_multiprocessing_fork: 1 min 25 sec
- test_compileall: 1 min 11 sec
- test_venv: 1 min 6 sec
- test_tarfile: 1 min 3 sec
- test_cppext: 1 min 2 sec
- test_zipfile: 1 min 2 sec

Total duration: 12 min 18 sec

Address Sanitizer:

10 slowest tests:
- test_venv: 2 min 13 sec
- test_compileall: 2 min 12 sec
- test_subprocess: 2 min 10 sec
- test_tarfile: 1 min 47 sec
- test_gdb: 1 min 37 sec
- test_zipfile: 1 min 28 sec
- test_weakref: 1 min 24 sec
- test_source_encoding: 1 min 14 sec
- test_decimal: 1 min 12 sec
- test_regrtest: 1 min 11 sec

Total duration: 15 min 24 sec

@vstinner vstinner merged commit 7a6cc3e into python:main Aug 23, 2023
16 checks passed
@vstinner vstinner deleted the requires_cpu branch August 23, 2023 22:57
vstinner added a commit to vstinner/cpython that referenced this pull request Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants