Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C API] Move private PyCode and PyOptimizer API to the internal C API #120642

Closed
vstinner opened this issue Jun 17, 2024 · 7 comments
Closed

[C API] Move private PyCode and PyOptimizer API to the internal C API #120642

vstinner opened this issue Jun 17, 2024 · 7 comments

Comments

@vstinner
Copy link
Member

vstinner commented Jun 17, 2024 •
edited by bedevere-app bot
Loading

The private PyCode and PyOptimizer APIs use "unnamed structs/unions" which are not allowed in ISO C99: see #120293.

I propose to move private PyCode and PyOptimizer APIs to the internal C API.

6 PyUnstable optimizer functions are moved to the internal C API:

  • PyUnstable_Replace_Executor()
  • PyUnstable_SetOptimizer()
  • PyUnstable_GetOptimizer()
  • PyUnstable_GetExecutor()
  • PyUnstable_Optimizer_NewCounter()
  • PyUnstable_Optimizer_NewUOpOptimizer()

Linked PRs
vstinner added a commit to vstinner/cpython that referenced this issue Jun 17, 2024
@vstinner
pythongh-120642: Move private PyCode APIs to the internal C API
118f88f 
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
@bedevere-app bedevere-app bot mentioned this issue Jun 17, 2024
Merged
@ronaldoussoren
Copy link
Contributor

These functions were added in #104584, and may be intended as an API used by other JITs as well (possibly with some changes).

@vstinner
Copy link
Member Author

These functions were added in #104584, and may be intended as an API used by other JITs as well (possibly with some changes).

These functions are still exported, but the few other JITs would have to update their code to include pycore_optimizer.h. Is it ok?

@ronaldoussoren
Copy link
Contributor

These functions were added in #104584, and may be intended as an API used by other JITs as well (possibly with some changes).

These functions are still exported, but the few other JITs would have to update their code to include pycore_optimizer.h. Is it ok?

I don't know. @markshannon added these function and may have an opinion on this.

@vstinner
Copy link
Member Author

vstinner commented Jun 20, 2024 •
edited
Loading

@iritkatriel @brandtbucher @Fidget-Spinner: Do you know anyone using these unstable functions outside Python itself? They were added to Python 3.13. I propose to move them to <pycore_optimizer.h>. In short, are you ok with #120643 ?

I prefer to move private APIs to the internal C API, and the API is causing C99 compatibility issues in strict mode (pedantic).

@Fidget-Spinner
Copy link
Member

From what I am aware of, I don't know anyone using these APIs outside of Python itself. However, they are meant to be used as unstable APIs, so not sure where they are supposed to go normally?

@markshannon
Copy link
Member

markshannon commented Jun 20, 2024 •
edited
Loading

I think it is fine to remove them, as no one is using this API.
If there is interest in the future for plug in optimizers, we would probably want to offer a quite different API that allowed plugins to add additional passes, rather replace the whole of tier 2.

@vstinner
Copy link
Member Author

I think it is fine to remove them, as no one is using this API.

Oh ok. I updated my PR to remove the 6 PyUnstable functions from the public C API. Most of them have to be kept for tests, but I renamed them and moved them to the internal C API.

vstinner added a commit that referenced this issue Jun 26, 2024
@vstinner
gh-120642: Move private PyCode APIs to the internal C API (#120643)
9e4a81f 
* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
vstinner added a commit to vstinner/cpython that referenced this issue Jun 26, 2024
@vstinner
pythongh-120642: Move private PyCode APIs to the internal C API (pyth…
a00e53f 
…on#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()

(cherry picked from commit 9e4a81f)
vstinner added a commit to vstinner/cpython that referenced this issue Jun 26, 2024
@vstinner
pythongh-120642: Move private PyCode APIs to the internal C API (pyth…
c3f2f6e 
…on#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()

(cherry picked from commit 9e4a81f)
@bedevere-app bedevere-app bot mentioned this issue Jun 26, 2024
Merged
vstinner added a commit that referenced this issue Jun 26, 2024
@vstinner
[3.13] gh-120642: Move private PyCode APIs to the internal C API (#12…
e26e098 
…0643) (#121043)

gh-120642: Move private PyCode APIs to the internal C API (#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()

(cherry picked from commit 9e4a81f)
mrahtz pushed a commit to mrahtz/cpython that referenced this issue Jun 30, 2024
@vstinner @mrahtz
pythongh-120642: Move private PyCode APIs to the internal C API (pyth…
a3adbfb 
…on#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
noahbkim pushed a commit to hudson-trading/cpython that referenced this issue Jul 11, 2024
@vstinner @noahbkim
pythongh-120642: Move private PyCode APIs to the internal C API (pyth…
bedb5f1 
…on#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
vstinner added a commit to vstinner/cpython that referenced this issue Jul 12, 2024
@vstinner
pythongh-120642: Move _PyCode_CODE() to the internal C API
57190ff 
Move _PyCode_CODE() and _PyCode_NBYTES() macros to the internal C API
since they use _Py_CODEUNIT which is only part of the internal C API.
@bedevere-app bedevere-app bot mentioned this issue Jul 12, 2024
Merged
vstinner added a commit that referenced this issue Jul 13, 2024
@vstinner
gh-120642: Move _PyCode_CODE() to the internal C API (#121644)
a2bec77 
Move _PyCode_CODE() and _PyCode_NBYTES() macros to the internal C API
since they use _Py_CODEUNIT which is only part of the internal C API.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 13, 2024
@vstinner @miss-islington
pythongh-120642: Move _PyCode_CODE() to the internal C API (pythonGH-…
711cd69 
…121644)

Move _PyCode_CODE() and _PyCode_NBYTES() macros to the internal C API
since they use _Py_CODEUNIT which is only part of the internal C API.
(cherry picked from commit a2bec77)

Co-authored-by: Victor Stinner <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Jul 13, 2024
Merged
vstinner added a commit that referenced this issue Jul 13, 2024
@miss-islington @vstinner
[3.13] gh-120642: Move _PyCode_CODE() to the internal C API (GH-121644)…
48a96d2 
… (#121729)

gh-120642: Move _PyCode_CODE() to the internal C API (GH-121644)

Move _PyCode_CODE() and _PyCode_NBYTES() macros to the internal C API
since they use _Py_CODEUNIT which is only part of the internal C API.
(cherry picked from commit a2bec77)

Co-authored-by: Victor Stinner <[email protected]>
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@vstinner @estyxx
pythongh-120642: Move private PyCode APIs to the internal C API (pyth…
7f14dc3 
…on#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@vstinner @estyxx
pythongh-120642: Move _PyCode_CODE() to the internal C API (python#12…
560c73f 
…1644)

Move _PyCode_CODE() and _PyCode_NBYTES() macros to the internal C API
since they use _Py_CODEUNIT which is only part of the internal C API.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@vstinner @ronaldoussoren @markshannon @Fidget-Spinner