Steps towards Python 3.11 support #3231
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Replace "Py_TYPE(obj) = type" with: "Py_SET_TYPE(obj, type)" * Replace "Py_REFCNT(Py_None) += n" with: "Py_SET_REFCNT(Py_None, Py_REFCNT(Py_None) + n)" * Add pythoncapi_compat.h to get Py_SET_TYPE() and Py_SET_REFCNT() on Python 3.9 and older. File copied from: https://github.com/pythoncapi/pythoncapi_compat On Python 3.10, Py_REFCNT() can no longer be used to set a reference count: * https://docs.python.org/dev/c-api/structures.html#c.Py_REFCNT * https://docs.python.org/dev/whatsnew/3.10.html#id2 On Python 3.11, Py_TYPE() can no longer be used to set an object type: * https://docs.python.org/dev/c-api/structures.html#c.Py_TYPE * https://docs.python.org/dev/whatsnew/3.11.html#id2
|
Thanks @vstinner. I guess python As for the python |
On the other side, |
|
I built manually Python 3.11 from source on Linux, and I installed it in /opt/py311 (pick your favorite directory): Currently, building datatable with Python 3.11 fails with 2 compiler errors on Py_TYPE() used as an l-value: With this PR, it builds successfully: |
I agree that it's tedious to use Python 3.11 right now. I'm using Fedora 35 which does provide Python 3.11 (Python version 3.11.0a3)! ... But I got issues while trying to build database using My concern is getting as many Python dependencies as possible compatible with Python 3.11 before Python 3.11 final will be released next October. Note: I'm also the author of https://www.python.org/dev/peps/pep-0674/ which introduced the incompatible change in Python :-) The PEP explains the rationale for these changes. |
|
Great, thank you for merging my fix :-) Ping me if you get more Python 3.11 compatibility issues ;-) |
|
We will add python 3.11 to our pipeline as long as it is released. If by that time you introduce some more incompatible changes, your help is very much appreciated :) |
|
@vstinner Do you think you can fix these warnings on a separate PR? |
|
Oh. I can reproduce some warnings on Linux using the command: I see multiple warnings. NULL: size_t vs Py_ssize_t: "use of old-style cast": |
I see different options:
|
|
Thanks, @vstinner. So |
|
I believe As for the static casting, it doesn't work Could you please provide some details as to what kind of casting you're doing in |
|
I'm working on a pythoncapi_compat.h update to make it compatible with C++. I created #3237 to check if there are still warnings on pythoncapi_compat.h when built by datatable, before merging it in my upstream project. |
|
I don't understand why the C++ compiler only complains about pythoncapi_compat.h whereas Python.h is far from being C++ compatible. For example, Python.h defines Py_REFCNT() which also uses old-style cast: datatable uses |
|
@vstinner which python is it? For Python 3.9 |
|
Python.h includes object.h which defines the Py_REFCNT() macro and static inline function. In recent Python functions, a static inline fnuction is used. Previously, it was only a macro. |
This was referenced May 3, 2022
@vstinner Actually, I'm now trying to build datatable with the officially released python 3.11 and get quite a lot of warnings/errors. Wasn't it the case for you when you built datatable with the python 3.11 dev version? |
Hi, I suggest you to open a separated issue. Your errors are different than the ones that I tried to address. For exaple: You should not redefine Py_buffer type in datatable. |
|
It is actually python 3.11 that redefined it, as we had this type for a long type in datatable :) Anyways, this issue could be easily fixed. What I wonder is why you had no issues compiling with py311 at the time of this PR and, if something important changed in py311 meanwhile, do you have any suggestions or updates for |
I tested alpha/beta versions of Python 3.11. It was still modified until the Python 3.11 final release. For example, _PySys_GetObjectId() got removed, datatable should not use this private function ;-) |
Sorry, I don't have the bandwidth to investigate these build issues. |
|
I see, what I meant was if you did any improvements to |
oleksiyskononenko
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
"Py_SET_TYPE(obj, type)"
"Py_SET_REFCNT(Py_None, Py_REFCNT(Py_None) + n)"
Python 3.9 and older. File copied from:
https://github.com/pythoncapi/pythoncapi_compat
On Python 3.10, Py_REFCNT() can no longer be used to set a reference
count:
On Python 3.11, Py_TYPE() can no longer be used to set an object
type: