Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for PEP 387 exception: Py_SET_TYPE #79

Closed
encukou opened this issue Oct 13, 2021 · 10 comments
Closed

Request for PEP 387 exception: Py_SET_TYPE #79

encukou opened this issue Oct 13, 2021 · 10 comments

Comments

@encukou
Copy link
Member

encukou commented Oct 13, 2021

In Python 3.11.0a1, the Py_TYPE macro was changed to a function, meaning that it cannot be used as a r-value (i.e. assignment target). All C code of the forms:

Py_TYPE(obj) = new_type;
obj->ob_type = new_type;

must be converted to

Py_SET_TYPE(obj, new_type);

This is documented in What's New in Python 3.11
The Py_SET_TYPE function was added in Python 3.9.

This change breaks existing code (though many popular projects have already been adapted in the 3.9-3.10 development window).
The change did not follow the process in PEP 387 -- Backwards Compatibility Policy, which calls for a warning for at least two releases before things break. Unfortunately, it is not feasible to only emit a warning when the macro is used as an assignment target.

The reasons for the change are given in a thread on capi-sig, specifically:

I'm not the best person to ask for this exception, as I disagree with making the change now. I believe that:

  • the experiments this allows can be run with the patch as part of the experiment;
  • third-party extensions needed to validate the experiments can be changed, and they'll usually accept patches if we just document that using Py_TYPE as r-value is deprecated;
  • for third parties, making the change will be easier after 3.8 is EOL and Py_SET_TYPE is in all supported Python versions.

I also think that PEP 387 says pretty clearly that exceptions from the policy need a Steering Council exception, but couldn't reach a friendly agreement on that :(
So, here we are: Please consider the change.

@vstinner
Copy link
Member

See also https://www.python.org/dev/peps/pep-0670/ which is not directly related.

@encukou
Copy link
Member Author

encukou commented Nov 25, 2021

PEP 670 was just updated to explicitly say it's not related.

@pablogsal
Copy link
Member

pablogsal commented Nov 28, 2021

Thanks for the request @encukou and thanks for raising this to our attention.

After deliberating over this, the Steering Council has agreed that the aforementioned changes to PY_TYPE can stay in the 3.11 release. This decision is based on the fact that is not possible to reliably add compiler/preprocessor warnings to the macro and that the impact of this change should be limited (and the fix is not very complex).

Being said that, we also want to add the following:

  • If this change results in a severe enough disruption to the community (and therefore our initial assessment about the impact of the change is wrong), the authors of the changes (@vstinner) must be prepared to revert the change before the first beta, given notice from the Steering Council.
  • We understand that announcing these changes is challenging given that there is no easy way to add automatic deprecation warnings, but we still believe these changes must be announced better and in general the deprecation notice needs to stay for at least two releases in the documentation.

@pablogsal
Copy link
Member

Please, feel free to reopen this issue if there is something missing or anything you would like to further discuss.

@encukou
Copy link
Member Author

encukou commented Nov 30, 2021

I do.

Apparently there is a lot of other macros that are also being converted now. Does this decision apply to all of these? And for any future ones?

Also:

these changes must be announced better and in general the deprecation notice needs to stay for at least two releases in the documentation.

There is currently no deprecation warning for this change. What do you mean by "the deprecation notice"? (The closest I can find is a silent documentation change between 3.9 and 3.10, and a What's New entry for 3.11.)
Should a notice be added now and stay until 3.12, or should it have been added in 3.9?

@encukou
Copy link
Member Author

encukou commented Nov 30, 2021

(I don't have the permission to reopen the issue.)

@pablogsal pablogsal reopened this Nov 30, 2021
@pablogsal
Copy link
Member

(Reopened)

@pablogsal
Copy link
Member

Apparently there is a lot of other macros that are also being converted now. Does this decision apply to all of these? And for any future ones?

This decision only applies to Py_SET_TYPE

@pablogsal
Copy link
Member

There is currently no deprecation warning for this change. What do you mean by "the deprecation notice"? (The closest I can find is a silent documentation change between 3.9 and 3.10, and a What's New entry for 3.11.)
Should a notice be added now and stay until 3.12, or should it have been added in 3.9?

We mean that in general (not for this change as we have made an exception per our previous argument), there should be a deprecation notice in some way or form (in these cases must be in the docs/what's new as there is no way to make the preprocessor warn only when we need it). We just wanted to reiterate that in general, we still want two releases with some deprecation notice for any deprecation, unless exempt.

Should a notice be added now and stay until 3.12, or should it have been added in 3.9?

For this case, just the What's New entry for 3.11.

@encukou
Copy link
Member Author

encukou commented Nov 30, 2021

Thanks for the clarifications!

@encukou encukou closed this as completed Nov 30, 2021
icanhasmath added a commit to ActiveState/numpy that referenced this issue Apr 3, 2024
icanhasmath added a commit to ActiveState/numpy that referenced this issue Apr 3, 2024
icanhasmath added a commit to ActiveState/numpy that referenced this issue Apr 3, 2024
icanhasmath added a commit to ActiveState/numpy that referenced this issue Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants