-
Notifications
You must be signed in to change notification settings - Fork 15.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python: Stop using global static state in C extension #10143
Comments
Please try version 4.21.x -- the C extension is completely rewritten and does not use global static state anymore. The new code is here: https://github.com/protocolbuffers/upb/tree/main/python I expect the new version should be friendly to sub-interpreters. The code you are looking at is the old code, which will not be used in future versions of protobuf. |
Great! I will try that and report back. I fear that |
@cretz where you able to confirm this works for you? |
I am afraid not because Python gRPC does not support 4.x sadly. I am closing this issue however as it's unreasonable to expect this to apply to the 3.x versions. If I do continue to have issues with import caching on 4.x if/when I can ever get there, I will make a new issue. |
Reopening. @haberman et al - I have upgraded to 4.x and am still experiencing global state and/or import reload issues. If I use an existing
This is because the generated code of If I also reload the
This appears to be because the C code checks against Of course everything works fine if I set the |
The first error ("AttributeError: 'NoneType' object has no attribute 'message_types_by_name'") is the bug described in #10075. A fix for this has been committed and will be released imminently. For the second error, your link points to code that is no longer used in 4.x. The 4.x code is here: https://github.com/protocolbuffers/upb/blob/41335a03becd3df1371de035dcbc787c4ce49ae3/python/message.c#L1772-L1780 The new code stores the class in module state, so it's not immediately obvious what the problem is here. |
Awesome, thanks
Ah thanks! I followed from the top of |
Looking at the functions at https://docs.python.org/3/c-api/module.html#module-lookup, I think that the interpreter keeps a Perhaps we need to be calling Can you add a unit test for this in https://github.com/protocolbuffers/upb/tree/main/python and see if adding |
I am not sure Maybe the code can be changed from checking that a protobuf class extends a specific type created the original module to whether the protobuf class extends that type by name. Technically there can be many This isn't urgent for me because the soon-to-be-released #10075 at least lets me re-import. |
@haberman Understood the issue is reported and fixed in the new version. Any ideas what version historically does not have this problem working with python3.6, it seems I cannot find a good combination. |
Unfortunately I think there might not be any combination that will work for you. Python 3.6 has been EOL for almost a year, we don't support it anymore, and the older versions that do support it don't have this fix. |
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment. This issue is labeled |
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it. This issue was closed and archived because there has been no new activity in the 14 days since the |
What language does this apply to?
Python
Describe the problem you are trying to solve.
Would like to be able to use the Python extension in subinterpreters or after reload.
Because of the assignment at
protobuf/python/google/protobuf/pyext/message.cc
Line 3053 in 6e34912
protobuf/python/google/protobuf/pyext/message.cc
Lines 216 to 217 in 6e34912
Describe the solution you'd like
Options:
The text was updated successfully, but these errors were encountered: