Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-117657: Add TSAN suppressions for the free-threaded build #117736

Merged
merged 4 commits into from
Apr 15, 2024

Conversation

mpage
Copy link
Contributor

@mpage mpage commented Apr 10, 2024

Add TSAN suppressions for existing races in the free-threaded build.

Additionally, skip a few tests that take a prohibitively long amount of time (> 1 hour) when TSAN is enabled and the GIL is disabled. I'm not sure why these are so much slower with TSAN (> 1 hour) vs without it (~1 minute). perf shows that all of the time is being spent in inclusively in GC in both configurations. When TSAN is enabled ~all of the exclusive cycles are in TSAN code (e.g __tsan_read8, __tsan_read1, etc.).

I cherry-picked this on top of the nogil-integration branch and verified that the free-threading TSAN tests pass locally and in CI.

Add TSAN suppressions for existing races in the free-threaded build.
@mpage mpage requested a review from colesbury April 11, 2024 00:13
@mpage mpage marked this pull request as ready for review April 11, 2024 00:13
Copy link
Contributor

@colesbury colesbury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Now that we have a list we can split up the work on fixing the data races.

Tools/tsan/supressions.txt Outdated Show resolved Hide resolved
Tools/tsan/suppressions_free_threading.txt Outdated Show resolved Hide resolved
@mpage
Copy link
Contributor Author

mpage commented Apr 11, 2024

Now that we have a list we can split up the work on fixing the data races.

Yep! I'll create issues and link them to #117657.

Copy link
Member

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well LGTM.

Lib/test/test_weakref.py Outdated Show resolved Hide resolved

## Free-threaded suppressions

race:_add_to_weak_set
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there GH issues open for all these suppressions?

Copy link
Contributor Author

@mpage mpage Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet, working on it :) I want to provide a bit more context (i.e. sample stacktraces + repro instructions) rather than just cutting issues without context.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to start with a simple list in an omnibus issue like #117657 so that there's a single place that people can go look to track who is working on what races.

I expect many of the races to have simple fixes, but we can break off the more complex races into their own issues as needed.

The work of collecting stacktraces and repros is work that can be split up once this PR is landed and we have a way of coordinating the work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I threw together a quick script to massage the TSAN output from the tests into an index that we can use to look for stacktraces and tests that should reproduce the races.

@colesbury colesbury merged commit 4783206 into python:main Apr 15, 2024
34 checks passed
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
…ython#117736)

Additionally, reduce the iterations for a few weakref tests that would
otherwise take a prohibitively long amount of time (> 1 hour) when TSAN
is enabled and the GIL is disabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants