[C API] Deprecate legacy API for configure Python initialization #88279

vstinner · 2021-05-12T10:20:54Z

BPO	44113
Nosy	@vstinner, @corona10, @joukewitteveen
PRs	bpo-44113: Deprecate old functions to config Python init #26060 bpo-43956: Doc/c-api: Fix default program name #24876 bpo-44113: Update __xxtestfuzz not to use Py_SetProgramName #26083 bpo-44113: Fix compiler warning in PySys_AddWarnOption() #26084 bpo-44113: Update fromzenmain not to use Py_SetProgramName #26085 bpo-44113: Move the What's New entry to Deprecate section #28974

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-05-14.12:21:54.201>
created_at = <Date 2021-05-12.10:20:54.214>
labels = ['expert-C-API', '3.11', 'docs']
title = '[C API] Deprecate legacy API for configure Python initialization'
updated_at = <Date 2021-10-15.11:43:51.791>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-10-15.11:43:51.791>
actor = 'vstinner'
assignee = 'docs@python'
closed = True
closed_date = <Date 2021-05-14.12:21:54.201>
closer = 'vstinner'
components = ['Documentation', 'C API']
creation = <Date 2021-05-12.10:20:54.214>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 44113
keywords = ['patch']
message_count = 8.0
messages = ['393499', '393550', '393551', '393553', '393641', '393647', '393648', '404005']
nosy_count = 4.0
nosy_names = ['vstinner', 'docs@python', 'corona10', 'joukewitteveen']
pr_nums = ['26060', '24876', '26083', '26084', '26085', '28974']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue44113'
versions = ['Python 3.11']

vstinner · 2021-05-12T10:20:54Z

The https://docs.python.org/dev/c-api/init.html documentation lists many functions which is the legacy way to configure the Python initialization. These functions are kept for backward compatibility but have flaws and are less reliable than the new PyConfig API (PEP-587) documented at https://docs.python.org/dev/c-api/init_config.html

I propose to deprecate the legacy functions to configure the Python initialization. Examples:

Py_SetPath()
Py_SetProgramName()
Py_SetPythonHome()
Py_SetStandardStreamEncoding()
PySys_AddWarnOption()
PySys_AddWarnOptionUnicode()
PySys_AddXOption()

I don't propose to schedule the removal of these functions, only mark them as deprecated in the *documentation*.

Related issue: bpo-43956 "C-API: Incorrect default value for Py_SetProgramName" and PR 24876.

vstinner · 2021-05-12T22:00:44Z

PyConfig API was added to Python 3.8, so it's now widely available. Python 3.8 always switched to security fixes only phase. So IMO it's ok to deprecate the old API now.

vstinner · 2021-05-12T22:49:10Z

"""
Check warning on line 82 in Python/frozenmain.c
GitHub Actions / Ubuntu
‘Py_SetProgramName’ is deprecated [-Wdeprecated-declarations]
"""

Well, Py_FrozenMain() should be rewritten with PyConfig, I want to do that for a long time, but I don't know how to test it. It would be nice to have a very basic test for Py_FrozenMain().

See also:
https://mail.python.org/archives/list/[email protected]/thread/5QLI3NUP3OSGLCCIBAQOTX4GEJQBWJ6F/

vstinner · 2021-05-13T00:31:40Z

commit 6cd0446
Author: Victor Stinner <[email protected]>
Date: Wed May 12 23:59:25 2021 +0200

bpo-44113: Deprecate old functions to config Python init (GH-26060)

Deprecate the following functions to configure the Python
initialization:

* PySys_AddWarnOption()
* PySys_AddWarnOptionUnicode()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()

Use the new PyConfig API of the Python Initialization Configuration
instead (PEP-587).

commit a0ccc40
Author: Dong-hee Na <[email protected]>
Date: Thu May 13 08:22:18 2021 +0900

bpo-44113: Update __xxtestfuzz not to use Py_SetProgramName (GH-26083)

commit ec7c09b
Author: Victor Stinner <[email protected]>
Date: Thu May 13 02:27:56 2021 +0200

bpo-44113: Fix compiler warning in PySys_AddWarnOption() (GH-26084)

Ignore Py_DEPRECATED() warning on purpose.

vstinner · 2021-05-14T11:57:21Z

commit 7565586
Author: Dong-hee Na <[email protected]>
Date: Thu May 13 10:19:46 2021 +0900

bpo-44113: Update fromzenmain not to use Py_SetProgramName (GH-26085)

vstinner · 2021-05-14T12:21:22Z

Well, Py_FrozenMain() should be rewritten with PyConfig, I want to do that for a long time, but I don't know how to test it. It would be nice to have a very basic test for Py_FrozenMain().

I created bpo-44131: [C API] Add tests on Py_FrozenMain().

vstinner · 2021-05-14T12:21:54Z

Thanks for the help Dong-hee ;-) I close the issue, functions are now deprecated.

vstinner · 2021-10-15T11:43:52Z

New changeset a7f8dfd by Victor Stinner in branch 'main':
bpo-44113: Move the What's New entry to Deprecate section (GH-28974)
a7f8dfd

Deprecate the following C functions: * PySys_SetArgv() * PySys_SetArgvEx() * PySys_SetPath()

…gvEx

…92428)

…gvEx (pythonGH-92428) (cherry picked from commit bd030b6) Co-authored-by: Serhiy Storchaka <[email protected]>

…H-92428) (cherry picked from commit bd030b6) Co-authored-by: Serhiy Storchaka <[email protected]>

Python 3.11 deprecates PySys_SetPath and Py_SetProgramName. The PyConfig API replaces these and other functions. This commit uses the PyConfig API to provide equivalent functionality while also preserving support for older versions of Python, i.e. those before Python 3.8. A beta version of Python 3.11 is available in Fedora Rawhide. Both Fedora 35 and Fedora 36 use Python 3.10, while Fedora 34 still used Python 3.9. I've tested these changes on Fedora 34, Fedora 36, and rawhide, though complete testing was not possible on rawhide due to a kernel bug. That being the case, I decided to enable the newer PyConfig API by testing PY_VERSION_HEX against 0x030a0000. This corresponds to Python 3.10. We could try to use the PyConfig API for Python versions as early as 3.8, but I'm reluctant to do this as there may have been PyConfig related bugs in earlier versions which have since been fixed. Recent linux distributions should have support for Python 3.10. This should be more than adequate for testing the new Python initialization code in GDB. Information about the PyConfig API as well as the motivation behind deprecating the old interface can be found at these links: python/cpython#88279 https://peps.python.org/pep-0587/ https://docs.python.org/3.11/c-api/init_config.html The v2 commit also addresses several problems that Simon found in the v1 version. In v1, I had used Py_DontWriteBytecodeFlag in the new initialization code, but Simon pointed out that this global configuration variable will be deprecated in Python 3.12. This version of the patch no longer uses Py_DontWriteBytecodeFlag in the new initialization code. Additionally, both Py_DontWriteBytecodeFlag and Py_IgnoreEnvironmentFlag will no longer be used when building GDB against Python 3.10 or higher. While it's true that both of these global configuration variables are deprecated in Python 3.12, it makes sense to disable their use for gdb builds against 3.10 and higher since those are the versions for which the PyConfig API is now being used by GDB. (The PyConfig API includes different mechanisms for making the same settings afforded by use of the soon-to-be deprecated global configuration variables.) Simon also noted that PyConfig_Clear() would not have be called for one of the failure paths. I've fixed that problem and also made the rest of the "bail out" code more direct. In particular, PyConfig_Clear() will always be called, both for success and failure. The v3 patch addresses some rebase conflicts related to module initialization . Commit 3acd9a6 ("Make 'import gdb.events' work") uses PyImport_ExtendInittab instead of PyImport_AppendInittab. That commit also initializes a struct for each module to import. Both the initialization and the call to were moved ahead of the ifdefs to avoid having to replicate (at least some of) the code three times in various portions of the ifdefs. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28668 Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=29287

milianw · 2024-01-10T18:54:44Z

Hey there, I'm unsure if this is the right place to ask - if not, please guide me elsewhere.

How are projects using the limited python API supposed to port away from the deprecated API? The "new" API in cpython/initconfig.h is guarded behind a #ifndef Py_LIMITED_API and thus not accessible...

Can you please make this API available through the limited API, or revert the decision to deprecate this API?

Thanks

encukou · 2024-01-16T14:14:03Z

The proper place is here: #105145
Apparently, the new API is available widely enough.

vstinner · 2024-01-16T14:27:29Z

How are projects using the limited python API supposed to port away from the deprecated API?

I suggest to continue in the dedicated issue: #107954 "No limited C API to customize Python initialization (PyConfig, PEP 587)"

I worked on two projects for the limited C API:

Stable ABI to configure Python initialization: [PEP 741] gh-107954: Add PyInitConfig C API #110176
Stable ABI to get the current Python configuration: [PEP 741] gh-107954: Add PyConfig_Get() function #112609 => Add PyConfig_Get() and PyConfig_GetInt() functions capi-workgroup/decisions#3

So far, I had to put this work aside since I had too many things on my plate.

vstinner added the 3.11 only security fixes label May 12, 2021

vstinner assigned docspython May 12, 2021

vstinner added docs Documentation in the Doc dir topic-C-API 3.11 only security fixes labels May 12, 2021

vstinner assigned docspython May 12, 2021

vstinner added docs Documentation in the Doc dir topic-C-API labels May 12, 2021

vstinner closed this as completed May 14, 2021

ezio-melotti transferred this issue from another repository Apr 10, 2022

vstinner reopened this May 6, 2022

vstinner added a commit that referenced this issue May 6, 2022

gh-88279: Deprecate PySys_SetArgvEx() (#92363)

299692a

Deprecate the following C functions: * PySys_SetArgv() * PySys_SetArgvEx() * PySys_SetPath()

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue May 7, 2022

pythongh-88279: Fix compiler warning for using deprecated PySys_SetAr…

16953b4

…gvEx

serhiy-storchaka mentioned this issue May 7, 2022

gh-88279: Fix compiler warning for using deprecated PySys_SetArgvEx #92428

Merged

pablogsal pushed a commit that referenced this issue May 7, 2022

gh-88279: Fix compiler warning for using deprecated PySys_SetArgvEx (#…

bd030b6

…92428)

miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 8, 2022

pythongh-88279: Fix compiler warning for using deprecated PySys_SetAr…

ecaf642

…gvEx (pythonGH-92428) (cherry picked from commit bd030b6) Co-authored-by: Serhiy Storchaka <[email protected]>

miss-islington added a commit that referenced this issue May 8, 2022

gh-88279: Fix compiler warning for using deprecated PySys_SetArgvEx (G…

14bd6df

…H-92428) (cherry picked from commit bd030b6) Co-authored-by: Serhiy Storchaka <[email protected]>

vstinner closed this as completed Nov 3, 2022

Csega mentioned this issue May 18, 2023

Deprecated use of Py_Initialize in gdlpython.cpp gnudatalanguage/gdl#1593

Open

vstinner mentioned this issue May 31, 2023

C API: Remove legacy API functions to configure Python initialization #105145

Closed

rokm mentioned this issue Aug 16, 2023

Configure and initialize python via PEP 587 API pyinstaller/pyinstaller#7847

Merged

jkohnert mentioned this issue Mar 29, 2024

Use new python initialisation, if old one is deprecated gnudatalanguage/gdl#1793

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[C API] Deprecate legacy API for configure Python initialization #88279

[C API] Deprecate legacy API for configure Python initialization #88279

vstinner commented May 12, 2021

vstinner commented May 12, 2021

vstinner commented May 12, 2021

vstinner commented May 12, 2021

vstinner commented May 13, 2021

vstinner commented May 14, 2021

vstinner commented May 14, 2021

vstinner commented May 14, 2021

vstinner commented Oct 15, 2021

milianw commented Jan 10, 2024

encukou commented Jan 16, 2024

vstinner commented Jan 16, 2024

[C API] Deprecate legacy API for configure Python initialization #88279

[C API] Deprecate legacy API for configure Python initialization #88279

Comments

vstinner commented May 12, 2021

vstinner commented May 12, 2021

vstinner commented May 12, 2021

vstinner commented May 12, 2021

vstinner commented May 13, 2021

vstinner commented May 14, 2021

vstinner commented May 14, 2021

vstinner commented May 14, 2021

vstinner commented Oct 15, 2021

milianw commented Jan 10, 2024

encukou commented Jan 16, 2024

vstinner commented Jan 16, 2024