Merge remote-tracking branch 'upstream/main' into gh-111140

.github/workflows/build.yml

@@ -250,7 +250,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        openssl_ver: [1.1.1w, 3.0.11, 3.1.3]
+        openssl_ver: [1.1.1w, 3.0.13, 3.1.5, 3.2.1]
     env:
       OPENSSL_VER: ${{ matrix.openssl_ver }}
       MULTISSL_DIR: ${{ github.workspace }}/multissl
@@ -304,7 +304,7 @@ jobs:
     needs: check_source
     if: needs.check_source.outputs.run_tests == 'true' && needs.check_source.outputs.run_hypothesis == 'true'
     env:
-      OPENSSL_VER: 3.0.11
+      OPENSSL_VER: 3.0.13
       PYTHONSTRICTEXTENSIONBUILD: 1
     steps:
     - uses: actions/checkout@v4
@@ -415,7 +415,7 @@ jobs:
     needs: check_source
     if: needs.check_source.outputs.run_tests == 'true'
     env:
-      OPENSSL_VER: 3.0.11
+      OPENSSL_VER: 3.0.13
       PYTHONSTRICTEXTENSIONBUILD: 1
       ASAN_OPTIONS: detect_leaks=0:allocator_may_return_null=1:handle_segv=0
     steps:

.github/workflows/reusable-ubuntu.yml

@@ -14,7 +14,7 @@ jobs:
     timeout-minutes: 60
     runs-on: ubuntu-20.04
     env:
-      OPENSSL_VER: 3.0.11
+      OPENSSL_VER: 3.0.13
       PYTHONSTRICTEXTENSIONBUILD: 1
     steps:
     - uses: actions/checkout@v4

Doc/c-api/dict.rst

@@ -174,6 +174,26 @@ Dictionary Objects
    .. versionadded:: 3.4
 
 
+.. c:function:: int PyDict_SetDefaultRef(PyObject *p, PyObject *key, PyObject *default_value, PyObject **result)
+
+   Inserts *default_value* into the dictionary *p* with a key of *key* if the
+   key is not already present in the dictionary. If *result* is not ``NULL``,
+   then *\*result* is set to a :term:`strong reference` to either
+   *default_value*, if the key was not present, or the existing value, if *key*
+   was already present in the dictionary.
+   Returns ``1`` if the key was present and *default_value* was not inserted,
+   or ``0`` if the key was not present and *default_value* was inserted.
+   On failure, returns ``-1``, sets an exception, and sets ``*result``
+   to ``NULL``.
+
+   For clarity: if you have a strong reference to *default_value* before
+   calling this function, then after it returns, you hold a strong reference
+   to both *default_value* and *\*result* (if it's not ``NULL``).
+   These may refer to the same object: in that case you hold two separate
+   references to it.
+   .. versionadded:: 3.13
+
+
 .. c:function:: int PyDict_Pop(PyObject *p, PyObject *key, PyObject **result)
 
    Remove *key* from dictionary *p* and optionally return the removed value.

Doc/c-api/intro.rst

@@ -148,7 +148,7 @@ complete listing.
    worse performances (due to increased code size for example). The compiler is
    usually smarter than the developer for the cost/benefit analysis.
 
-   If Python is :ref:`built in debug mode <debug-build>` (if the ``Py_DEBUG``
+   If Python is :ref:`built in debug mode <debug-build>` (if the :c:macro:`Py_DEBUG`
    macro is defined), the :c:macro:`Py_ALWAYS_INLINE` macro does nothing.
 
    It must be specified before the function return type. Usage::
@@ -812,12 +812,14 @@ available that support tracing of reference counts, debugging the memory
 allocator, or low-level profiling of the main interpreter loop.  Only the most
 frequently used builds will be described in the remainder of this section.
 
-Compiling the interpreter with the :c:macro:`Py_DEBUG` macro defined produces
+.. c:macro:: Py_DEBUG
+
+Compiling the interpreter with the :c:macro:`!Py_DEBUG` macro defined produces
 what is generally meant by :ref:`a debug build of Python <debug-build>`.
-:c:macro:`Py_DEBUG` is enabled in the Unix build by adding
+:c:macro:`!Py_DEBUG` is enabled in the Unix build by adding
 :option:`--with-pydebug` to the :file:`./configure` command.
 It is also implied by the presence of the
-not-Python-specific :c:macro:`_DEBUG` macro.  When :c:macro:`Py_DEBUG` is enabled
+not-Python-specific :c:macro:`!_DEBUG` macro.  When :c:macro:`!Py_DEBUG` is enabled
 in the Unix build, compiler optimization is disabled.
 
 In addition to the reference count debugging described below, extra checks are
@@ -832,4 +834,3 @@ after every statement run by the interpreter.)
 
 Please refer to :file:`Misc/SpecialBuilds.txt` in the Python source distribution
 for more detailed information.
-

Doc/faq/library.rst

@@ -405,22 +405,37 @@ lists.  When in doubt, use a mutex!
 Can't we get rid of the Global Interpreter Lock?
 ------------------------------------------------
 
-.. XXX link to dbeazley's talk about GIL?
-
 The :term:`global interpreter lock` (GIL) is often seen as a hindrance to Python's
 deployment on high-end multiprocessor server machines, because a multi-threaded
 Python program effectively only uses one CPU, due to the insistence that
 (almost) all Python code can only run while the GIL is held.
 
-Back in the days of Python 1.5, Greg Stein actually implemented a comprehensive
+With the approval of :pep:`703` work is now underway to remove the GIL from the
+CPython implementation of Python.  Initially it will be implemented as an
+optional compiler flag when building the interpreter, and so separate
+builds will be available with and without the GIL.  Long-term, the hope is
+to settle on a single build, once the performance implications of removing the
+GIL are fully understood.  Python 3.13 is likely to be the first release
+containing this work, although it may not be completely functional in this
+release.
+
+The current work to remove the GIL is based on a
+`fork of Python 3.9 with the GIL removed <https://github.com/colesbury/nogil>`_
+by Sam Gross.
+Prior to that,
+in the days of Python 1.5, Greg Stein actually implemented a comprehensive
 patch set (the "free threading" patches) that removed the GIL and replaced it
-with fine-grained locking.  Adam Olsen recently did a similar experiment
+with fine-grained locking.  Adam Olsen did a similar experiment
 in his `python-safethread <https://code.google.com/archive/p/python-safethread>`_
-project.  Unfortunately, both experiments exhibited a sharp drop in single-thread
+project.  Unfortunately, both of these earlier experiments exhibited a sharp
+drop in single-thread
 performance (at least 30% slower), due to the amount of fine-grained locking
-necessary to compensate for the removal of the GIL.
+necessary to compensate for the removal of the GIL.  The Python 3.9 fork
+is the first attempt at removing the GIL with an acceptable performance
+impact.
 
-This doesn't mean that you can't make good use of Python on multi-CPU machines!
+The presence of the GIL in current Python releases
+doesn't mean that you can't make good use of Python on multi-CPU machines!
 You just have to be creative with dividing the work up between multiple
 *processes* rather than multiple *threads*.  The
 :class:`~concurrent.futures.ProcessPoolExecutor` class in the new
@@ -434,22 +449,13 @@ thread of execution is in the C code and allow other threads to get some work
 done.  Some standard library modules such as :mod:`zlib` and :mod:`hashlib`
 already do this.
 
-It has been suggested that the GIL should be a per-interpreter-state lock rather
-than truly global; interpreters then wouldn't be able to share objects.
-Unfortunately, this isn't likely to happen either.  It would be a tremendous
-amount of work, because many object implementations currently have global state.
-For example, small integers and short strings are cached; these caches would
-have to be moved to the interpreter state.  Other object types have their own
-free list; these free lists would have to be moved to the interpreter state.
-And so on.
-
-And I doubt that it can even be done in finite time, because the same problem
-exists for 3rd party extensions.  It is likely that 3rd party extensions are
-being written at a faster rate than you can convert them to store all their
-global state in the interpreter state.
-
-And finally, once you have multiple interpreters not sharing any state, what
-have you gained over running each interpreter in a separate process?
+An alternative approach to reducing the impact of the GIL is
+to make the GIL a per-interpreter-state lock rather than truly global.
+This was :ref:`first implemented in Python 3.12 <whatsnew312-pep684>` and is
+available in the C API. A Python interface to it is expected in Python 3.13.
+The main limitation to it at the moment is likely to be 3rd party extension
+modules, since these must be written with multiple interpreters in mind in
+order to be usable, so many older extension modules will not be usable.
 
 
 Input and Output

Doc/library/argparse.rst

@@ -777,6 +777,8 @@ The add_argument() method
    * dest_ - The name of the attribute to be added to the object returned by
      :meth:`parse_args`.
 
+   * deprecated_ - Whether or not use of the argument is deprecated.
+
 The following sections describe how each of these are used.
 
 
@@ -1439,6 +1441,34 @@ behavior::
    >>> parser.parse_args('--foo XXX'.split())
    Namespace(bar='XXX')
 
+
+.. _deprecated:
+
+deprecated
+^^^^^^^^^^
+
+During a project's lifetime, some arguments may need to be removed from the
+command line. Before removing them, you should inform
+your users that the arguments are deprecated and will be removed.
+The ``deprecated`` keyword argument of
+:meth:`~ArgumentParser.add_argument`, which defaults to ``False``,
+specifies if the argument is deprecated and will be removed
+in the future.
+For arguments, if ``deprecated`` is ``True``, then a warning will be
+printed to standard error when the argument is used::
+
+   >>> import argparse
+   >>> parser = argparse.ArgumentParser(prog='snake.py')
+   >>> parser.add_argument('--legs', default=0, type=int, deprecated=True)
+   >>> parser.parse_args([])
+   Namespace(legs=0)
+   >>> parser.parse_args(['--legs', '4'])  # doctest: +SKIP
+   snake.py: warning: option '--legs' is deprecated
+   Namespace(legs=4)
+
+.. versionchanged:: 3.13
+
+
 Action classes
 ^^^^^^^^^^^^^^
 
@@ -1842,7 +1872,8 @@ Sub-commands
 
        {foo,bar}   additional help
 
-   Furthermore, ``add_parser`` supports an additional ``aliases`` argument,
+   Furthermore, :meth:`~_SubParsersAction.add_parser` supports an additional
+   *aliases* argument,
    which allows multiple strings to refer to the same subparser. This example,
    like ``svn``, aliases ``co`` as a shorthand for ``checkout``::
 
@@ -1853,6 +1884,20 @@ Sub-commands
      >>> parser.parse_args(['co', 'bar'])
      Namespace(foo='bar')
 
+   :meth:`~_SubParsersAction.add_parser` supports also an additional
+   *deprecated* argument, which allows to deprecate the subparser.
+
+      >>> import argparse
+      >>> parser = argparse.ArgumentParser(prog='chicken.py')
+      >>> subparsers = parser.add_subparsers()
+      >>> run = subparsers.add_parser('run')
+      >>> fly = subparsers.add_parser('fly', deprecated=True)
+      >>> parser.parse_args(['fly'])  # doctest: +SKIP
+      chicken.py: warning: command 'fly' is deprecated
+      Namespace()
+
+   .. versionadded:: 3.13
+
    One particularly effective way of handling sub-commands is to combine the use
    of the :meth:`add_subparsers` method with calls to :meth:`set_defaults` so
    that each subparser knows which Python function it should execute.  For

Doc/library/enum.rst

@@ -534,9 +534,7 @@ Data Types
          >>> list(purple)
          [<Color.RED: 1>, <Color.BLUE: 4>]
 
-      .. versionchanged:: 3.11
-
-         Aliases are no longer returned during iteration.
+      .. versionadded:: 3.11
 
    .. method:: __len__(self):
 

Doc/library/exceptions.rst

@@ -16,7 +16,7 @@ equivalent, even if they have the same name.
 
 .. index:: pair: statement; raise
 
-The built-in exceptions listed below can be generated by the interpreter or
+The built-in exceptions listed in this chapter can be generated by the interpreter or
 built-in functions.  Except where mentioned, they have an "associated value"
 indicating the detailed cause of the error.  This may be a string or a tuple of
 several items of information (e.g., an error code and a string explaining the

Doc/library/sqlite3.rst

@@ -1137,12 +1137,19 @@ Connection objects
 
    .. _Loading an Extension: https://www.sqlite.org/loadext.html#loading_an_extension_
 
-   .. method:: iterdump
+   .. method:: iterdump(*, filter=None)
 
       Return an :term:`iterator` to dump the database as SQL source code.
       Useful when saving an in-memory database for later restoration.
       Similar to the ``.dump`` command in the :program:`sqlite3` shell.
 
+      :param filter:
+
+        An optional ``LIKE`` pattern for database objects to dump, e.g. ``prefix_%``.
+        If ``None`` (the default), all database objects will be included.
+
+      :type filter: str | None
+
       Example:
 
       .. testcode::
@@ -1158,6 +1165,8 @@ Connection objects
 
          :ref:`sqlite3-howto-encoding`
 
+      .. versionchanged:: 3.13
+         Added the *filter* parameter.
 
    .. method:: backup(target, *, pages=-1, progress=None, name="main", sleep=0.250)
 

Doc/library/test.rst

@@ -324,9 +324,9 @@ The :mod:`test.support` module defines the following constants:
 
 .. data:: Py_DEBUG
 
-   True if Python is built with the :c:macro:`Py_DEBUG` macro defined: if
-   Python is :ref:`built in debug mode <debug-build>`
-   (:option:`./configure --with-pydebug <--with-pydebug>`).
+   True if Python was built with the :c:macro:`Py_DEBUG` macro
+   defined, that is, if
+   Python was :ref:`built in debug mode <debug-build>`.
 
    .. versionadded:: 3.12
 

Doc/whatsnew/3.13.rst

@@ -134,6 +134,11 @@ Other Language Changes
   is rejected when the global is used in the :keyword:`else` block.
   (Contributed by Irit Katriel in :gh:`111123`.)
 
+* Many functions now emit a warning if a boolean value is passed as
+  a file descriptor argument.
+  This can help catch some errors earlier.
+  (Contributed by Serhiy Storchaka in :gh:`82626`.)
+
 * Added a new environment variable :envvar:`PYTHON_FROZEN_MODULES`. It
   determines whether or not frozen modules are ignored by the import machinery,
   equivalent of the :option:`-X frozen_modules <-X>` command-line option.
@@ -153,6 +158,15 @@ New Modules
 Improved Modules
 ================
 
+argparse
+--------
+
+* Add parameter *deprecated* in methods
+  :meth:`~argparse.ArgumentParser.add_argument` and :meth:`!add_parser`
+  which allows to deprecate command-line options, positional arguments and
+  subcommands.
+  (Contributed by Serhiy Storchaka in :gh:`83648`).
+
 array
 -----
 
@@ -343,7 +357,7 @@ pathlib
   (Contributed by Barney Gale in :gh:`89812`.)
 
 * Add :meth:`pathlib.Path.from_uri`, a new constructor to create a :class:`pathlib.Path`
-  object from a 'file' URI (``file:/``).
+  object from a 'file' URI (``file://``).
   (Contributed by Barney Gale in :gh:`107465`.)
 
 * Add :meth:`pathlib.PurePath.full_match` for matching paths with
@@ -390,6 +404,10 @@ sqlite3
   object is not :meth:`closed <sqlite3.Connection.close>` explicitly.
   (Contributed by Erlend E. Aasland in :gh:`105539`.)
 
+* Add *filter* keyword-only parameter to :meth:`sqlite3.Connection.iterdump`
+  for filtering database objects to dump.
+  (Contributed by Mariusz Felisiak in :gh:`91602`.)
+
 subprocess
 ----------
 
@@ -1388,6 +1406,12 @@ New Features
   not needed.
   (Contributed by Victor Stinner in :gh:`106004`.)
 
+* Added :c:func:`PyDict_SetDefaultRef`, which is similar to
+  :c:func:`PyDict_SetDefault` but returns a :term:`strong reference` instead of
+  a :term:`borrowed reference`. This function returns ``-1`` on error, ``0`` on
+  insertion, and ``1`` if the key was already present in the dictionary.
+  (Contributed by Sam Gross in :gh:`112066`.)
+
 * Add :c:func:`PyDict_ContainsString` function: same as
   :c:func:`PyDict_Contains`, but *key* is specified as a :c:expr:`const char*`
   UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`.

Include/cpython/dictobject.h

@@ -41,6 +41,16 @@ PyAPI_FUNC(PyObject *) _PyDict_GetItemStringWithError(PyObject *, const char *);
 PyAPI_FUNC(PyObject *) PyDict_SetDefault(
     PyObject *mp, PyObject *key, PyObject *defaultobj);
 
+// Inserts `key` with a value `default_value`, if `key` is not already present
+// in the dictionary.  If `result` is not NULL, then the value associated
+// with `key` is returned in `*result` (either the existing value, or the now
+// inserted `default_value`).
+// Returns:
+//   -1 on error
+//    0 if `key` was not present and `default_value` was inserted
+//    1 if `key` was present and `default_value` was not inserted
+PyAPI_FUNC(int) PyDict_SetDefaultRef(PyObject *mp, PyObject *key, PyObject *default_value, PyObject **result);
+
 /* Get the number of items of a dictionary. */
 static inline Py_ssize_t PyDict_GET_SIZE(PyObject *op) {
     PyDictObject *mp;