gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe #120165
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Fidget-Spinner
commented
Jun 6, 2024
•
Fidget-Spinner
commented
- Issue: Make TSAN tests pass with the GIL disabled in free-threaded builds #117657
|
Do you have a test / stack trace of the reported race? I don't think we should be modifying an object's type ( |
|
Example code: class Foo:
pass
class Bar:
pass
import threading
import os
X = Foo()
def work():
foo = X
for _ in range(10000):
foo.__class__ = Bar
type(foo)
foo.__class__ = Foo
type(foo)
threads = []
for i in range(os.cpu_count() - 1):
thread = threading.Thread(target=work)
thread.start()
threads.append(thread)
work()
for thread in threads:
thread.join()TSAN output: Py_SET_TYPE: |
|
Wow... the test case actually causes a segfault even with the fixes, not just a tsan warning. |
Well, Py_TYPE() returns a borrow reference which is a bad thing :-/ But it shouldn't matter in this case, since the type is not deleted by the test. |
|
Can't merge this until we fix the actual crasher code #120198. It seems even default build is affected. |
Fidget-Spinner
commented
Jun 8, 2024
picnixz
reviewed
Jun 9, 2024
Co-authored-by: Bénédikt Tran <[email protected]>
vstinner
reviewed
Jun 12, 2024
| @@ -6630,12 +6630,18 @@ object_set_class(PyObject *self, PyObject *value, void *closure) | |||
| return -1; | |||
| } | |||
| } | |||
| Py_BEGIN_CRITICAL_SECTION(self); | |||
| if (newto->tp_flags & Py_TPFLAGS_HEAPTYPE) { | |||
| Py_INCREF(newto); | |||
There was a problem hiding this comment.
Is it important to do the INCREF(newto) and DECREF(oldto) inside the critical section?
Or it it possible to restrict the critical section to the following code?
oldto = Py_TYPE(self);
Py_SET_TYPE(self, newto);There was a problem hiding this comment.
You're right. I can shrink it!
Co-Authored-By: Nadeshiko Manju <[email protected]>
vstinner
approved these changes
Jun 12, 2024
|
Thanks for the review Victor! |
|
Thanks @Fidget-Spinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13. |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Jun 12, 2024
…20165) (cherry picked from commit e16aed6) Co-authored-by: Ken Jin <[email protected]> Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
|
GH-120403 is a backport of this pull request to the 3.13 branch. |
Fidget-Spinner
added a commit
that referenced
this pull request
Jun 12, 2024
GH-120403) gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165) (cherry picked from commit e16aed6) Co-authored-by: Ken Jin <[email protected]> Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
|
I think we should address
The thread-safety guarantees should be something like:
@Fidget-Spinner, would you like to work on this? |
|
@colesbury yeah I can revert the atomics in Py_TYPE and PY_SET_TYPE and stop the world instead. |
mrahtz
pushed a commit
to mrahtz/cpython
that referenced
this pull request
Jun 30, 2024
…20165) Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
noahbkim
pushed a commit
to hudson-trading/cpython
that referenced
this pull request
Jul 11, 2024
…20165) Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
estyxx
pushed a commit
to estyxx/cpython
that referenced
this pull request
Jul 17, 2024
…20165) Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.