Merge branch 'main' into pythongh-115103-qsbr

.gitattributes

@@ -94,7 +94,7 @@ Programs/test_frozenmain.h                          generated
 Python/Python-ast.c                                 generated
 Python/executor_cases.c.h                           generated
 Python/generated_cases.c.h                          generated
-Python/tier2_redundancy_eliminator_bytecodes.c.h    generated
+Python/tier2_redundancy_eliminator_cases.c.h        generated
 Python/opcode_targets.h                             generated
 Python/stdlib_module_names.h                        generated
 Tools/peg_generator/pegen/grammar_parser.py         generated

.github/workflows/jit.yml

@@ -4,10 +4,14 @@ on:
     paths:
       - '**jit**'
       - 'Python/bytecodes.c'
+      - 'Python/optimizer*.c'
+      - 'Python/tier2_redundancy_eliminator_bytecodes.c'
   push:
     paths:
       - '**jit**'
       - 'Python/bytecodes.c'
+      - 'Python/optimizer*.c'
+      - 'Python/tier2_redundancy_eliminator_bytecodes.c'
   workflow_dispatch:
 
 concurrency:

Doc/c-api/structures.rst

@@ -187,26 +187,26 @@ Implementing functions and methods
                                         PyObject *kwargs);
 
 
-.. c:type:: _PyCFunctionFast
+.. c:type:: PyCFunctionFast
 
    Type of the functions used to implement Python callables in C
    with signature :c:macro:`METH_FASTCALL`.
    The function signature is::
 
-      PyObject *_PyCFunctionFast(PyObject *self,
-                                 PyObject *const *args,
-                                 Py_ssize_t nargs);
+      PyObject *PyCFunctionFast(PyObject *self,
+                                PyObject *const *args,
+                                Py_ssize_t nargs);
 
-.. c:type:: _PyCFunctionFastWithKeywords
+.. c:type:: PyCFunctionFastWithKeywords
 
    Type of the functions used to implement Python callables in C
    with signature :ref:`METH_FASTCALL | METH_KEYWORDS <METH_FASTCALL-METH_KEYWORDS>`.
    The function signature is::
 
-      PyObject *_PyCFunctionFastWithKeywords(PyObject *self,
-                                             PyObject *const *args,
-                                             Py_ssize_t nargs,
-                                             PyObject *kwnames);
+      PyObject *PyCFunctionFastWithKeywords(PyObject *self,
+                                            PyObject *const *args,
+                                            Py_ssize_t nargs,
+                                            PyObject *kwnames);
 
 .. c:type:: PyCMethod
 
@@ -290,7 +290,7 @@ There are these calling conventions:
 .. c:macro:: METH_FASTCALL
 
    Fast calling convention supporting only positional arguments.
-   The methods have the type :c:type:`_PyCFunctionFast`.
+   The methods have the type :c:type:`PyCFunctionFast`.
    The first parameter is *self*, the second parameter is a C array
    of :c:expr:`PyObject*` values indicating the arguments and the third
    parameter is the number of arguments (the length of the array).
@@ -306,7 +306,7 @@ There are these calling conventions:
 
 :c:expr:`METH_FASTCALL | METH_KEYWORDS`
    Extension of :c:macro:`METH_FASTCALL` supporting also keyword arguments,
-   with methods of type :c:type:`_PyCFunctionFastWithKeywords`.
+   with methods of type :c:type:`PyCFunctionFastWithKeywords`.
    Keyword arguments are passed the same way as in the
    :ref:`vectorcall protocol <vectorcall>`:
    there is an additional fourth :c:expr:`PyObject*` parameter

Doc/library/ctypes.rst

@@ -670,6 +670,10 @@ compiler does it. It is possible to override this behavior by specifying a
 :attr:`~Structure._pack_` class attribute in the subclass definition.
 This must be set to a positive integer and specifies the maximum alignment for the fields.
 This is what ``#pragma pack(n)`` also does in MSVC.
+It is also possible to set a minimum alignment for how the subclass itself is packed in the
+same way ``#pragma align(n)`` works in MSVC.
+This can be achieved by specifying a ::attr:`~Structure._align_` class attribute
+in the subclass definition.
 
 :mod:`ctypes` uses the native byte order for Structures and Unions.  To build
 structures with non-native byte order, you can use one of the
@@ -2534,6 +2538,12 @@ fields, or any other data types containing pointer type fields.
       Setting this attribute to 0 is the same as not setting it at all.
 
 
+   .. attribute:: _align_
+
+      An optional small integer that allows overriding the alignment of
+      the structure when being packed or unpacked to/from memory.
+      Setting this attribute to 0 is the same as not setting it at all.
+
    .. attribute:: _anonymous_
 
       An optional sequence that lists the names of unnamed (anonymous) fields.

Doc/library/dis.rst

@@ -1606,7 +1606,7 @@ iterations of the loop.
 
       value = STACK.pop()
       result = func(value)
-      STACK.push(result)
+      STACK.append(result)
 
    * ``oparg == 1``: call :func:`str` on *value*
    * ``oparg == 2``: call :func:`repr` on *value*
@@ -1623,7 +1623,7 @@ iterations of the loop.
 
       value = STACK.pop()
       result = value.__format__("")
-      STACK.push(result)
+      STACK.append(result)
 
    Used for implementing formatted literal strings (f-strings).
 
@@ -1636,7 +1636,7 @@ iterations of the loop.
       spec = STACK.pop()
       value = STACK.pop()
       result = value.__format__(spec)
-      STACK.push(result)
+      STACK.append(result)
 
    Used for implementing formatted literal strings (f-strings).
 
@@ -1783,7 +1783,7 @@ iterations of the loop.
       arg2 = STACK.pop()
       arg1 = STACK.pop()
       result = intrinsic2(arg1, arg2)
-      STACK.push(result)
+      STACK.append(result)
 
    The operand determines which intrinsic function is called:
 

Doc/library/exceptions.rst

@@ -416,6 +416,24 @@ The following exceptions are the exceptions that are usually raised.
    handling in C, most floating point operations are not checked.
 
 
+.. exception:: PythonFinalizationError
+
+   This exception is derived from :exc:`RuntimeError`.  It is raised when
+   an operation is blocked during interpreter shutdown also known as
+   :term:`Python finalization <interpreter shutdown>`.
+
+   Examples of operations which can be blocked with a
+   :exc:`PythonFinalizationError` during the Python finalization:
+
+   * Creating a new Python thread.
+   * :func:`os.fork`.
+
+   See also the :func:`sys.is_finalizing` function.
+
+   .. versionadded:: 3.13
+      Previously, a plain :exc:`RuntimeError` was raised.
+
+
 .. exception:: RecursionError
 
    This exception is derived from :exc:`RuntimeError`.  It is raised when the

Doc/library/inspect.rst

@@ -340,6 +340,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
       Functions wrapped in :func:`functools.partial` now return ``True`` if the
       wrapped function is a Python generator function.
 
+   .. versionchanged:: 3.13
+      Functions wrapped in :func:`functools.partialmethod` now return ``True``
+      if the wrapped function is a Python generator function.
 
 .. function:: isgenerator(object)
 
@@ -363,6 +366,10 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
       Sync functions marked with :func:`markcoroutinefunction` now return
       ``True``.
 
+   .. versionchanged:: 3.13
+      Functions wrapped in :func:`functools.partialmethod` now return ``True``
+      if the wrapped function is a :term:`coroutine function`.
+
 
 .. function:: markcoroutinefunction(func)
 
@@ -429,6 +436,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
       Functions wrapped in :func:`functools.partial` now return ``True`` if the
       wrapped function is a :term:`asynchronous generator` function.
 
+   .. versionchanged:: 3.13
+      Functions wrapped in :func:`functools.partialmethod` now return ``True``
+      if the wrapped function is a :term:`coroutine function`.
 
 .. function:: isasyncgen(object)
 

Doc/library/profile.rst

@@ -299,6 +299,13 @@ functions:
       Create a :class:`~pstats.Stats` object based on the current
       profile and print the results to stdout.
 
+      The *sort* parameter specifies the sorting order of the displayed
+      statistics. It accepts a single key or a tuple of keys to enable
+      multi-level sorting, as in :func:`Stats.sort_stats <pstats.Stats.sort_stats>`.
+
+      .. versionadded:: 3.13
+         :meth:`~Profile.print_stats` now accepts a tuple of keys.
+
    .. method:: dump_stats(filename)
 
       Write the results of the current profile to *filename*.

Doc/library/sys.rst

@@ -1202,6 +1202,8 @@ always available.
    Return :const:`True` if the main Python interpreter is
    :term:`shutting down <interpreter shutdown>`. Return :const:`False` otherwise.
 
+   See also the :exc:`PythonFinalizationError` exception.
+
    .. versionadded:: 3.5
 
 .. data:: last_exc

Doc/whatsnew/2.6.rst

@@ -2992,6 +2992,33 @@ Changes to Python's build process and to the C API include:
   architectures (x86, PowerPC), 64-bit (x86-64 and PPC-64), or both.
   (Contributed by Ronald Oussoren.)
 
+* A new function added in Python 2.6.6, :c:func:`!PySys_SetArgvEx`, sets
+  the value of ``sys.argv`` and can optionally update ``sys.path`` to
+  include the directory containing the script named by ``sys.argv[0]``
+  depending on the value of an *updatepath* parameter.
+
+  This function was added to close a security hole for applications
+  that embed Python.  The old function, :c:func:`!PySys_SetArgv`, would
+  always update ``sys.path``, and sometimes it would add the current
+  directory.  This meant that, if you ran an application embedding
+  Python in a directory controlled by someone else, attackers could
+  put a Trojan-horse module in the directory (say, a file named
+  :file:`os.py`) that your application would then import and run.
+
+  If you maintain a C/C++ application that embeds Python, check
+  whether you're calling :c:func:`!PySys_SetArgv` and carefully consider
+  whether the application should be using :c:func:`!PySys_SetArgvEx`
+  with *updatepath* set to false.  Note that using this function will
+  break compatibility with Python versions 2.6.5 and earlier; if you
+  have to continue working with earlier versions, you can leave
+  the call to :c:func:`!PySys_SetArgv` alone and call
+  ``PyRun_SimpleString("sys.path.pop(0)\n")`` afterwards to discard
+  the first ``sys.path`` component.
+
+  Security issue reported as `CVE-2008-5983
+  <http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983>`_;
+  discussed in :gh:`50003`, and fixed by Antoine Pitrou.
+
 * The BerkeleyDB module now has a C API object, available as
   ``bsddb.db.api``.   This object can be used by other C extensions
   that wish to use the :mod:`bsddb` module for their own purposes.
@@ -3294,6 +3321,15 @@ that may require changes to your code:
   scoping rules, also cause warnings because such comparisons are forbidden
   entirely in 3.0.
 
+For applications that embed Python:
+
+* The :c:func:`!PySys_SetArgvEx` function was added in Python 2.6.6,
+  letting applications close a security hole when the existing
+  :c:func:`!PySys_SetArgv` function was used.  Check whether you're
+  calling :c:func:`!PySys_SetArgv` and carefully consider whether the
+  application should be using :c:func:`!PySys_SetArgvEx` with
+  *updatepath* set to false.
+
 .. ======================================================================
 
 

Doc/whatsnew/3.1.rst

@@ -80,6 +80,28 @@ Support was also added for third-party tools like `PyYAML <https://pyyaml.org/>`
       PEP written by Armin Ronacher and Raymond Hettinger.  Implementation
       written by Raymond Hettinger.
 
+Since an ordered dictionary remembers its insertion order, it can be used
+in conjuction with sorting to make a sorted dictionary::
+
+    >>> # regular unsorted dictionary
+    >>> d = {'banana': 3, 'apple':4, 'pear': 1, 'orange': 2}
+
+    >>> # dictionary sorted by key
+    >>> OrderedDict(sorted(d.items(), key=lambda t: t[0]))
+    OrderedDict([('apple', 4), ('banana', 3), ('orange', 2), ('pear', 1)])
+
+    >>> # dictionary sorted by value
+    >>> OrderedDict(sorted(d.items(), key=lambda t: t[1]))
+    OrderedDict([('pear', 1), ('orange', 2), ('banana', 3), ('apple', 4)])
+
+    >>> # dictionary sorted by length of the key string
+    >>> OrderedDict(sorted(d.items(), key=lambda t: len(t[0])))
+    OrderedDict([('pear', 1), ('apple', 4), ('orange', 2), ('banana', 3)])
+
+The new sorted dictionaries maintain their sort order when entries
+are deleted.  But when new keys are added, the keys are appended
+to the end and the sort is not maintained.
+
 
 PEP 378: Format Specifier for Thousands Separator
 =================================================

Doc/whatsnew/3.10.rst

@@ -1517,6 +1517,13 @@ functions internally.  For more details, please see their respective
 documentation.
 (Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
 
+The presence of newline or tab characters in parts of a URL allows for some
+forms of attacks. Following the WHATWG specification that updates :rfc:`3986`,
+ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the
+URL by the parser in :mod:`urllib.parse` preventing such attacks. The removal
+characters are controlled by a new module level variable
+``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :gh:`88048`)
+
 xml
 ---
 
@@ -2315,3 +2322,43 @@ Removed
 
 * The ``PyThreadState.use_tracing`` member has been removed to optimize Python.
   (Contributed by Mark Shannon in :issue:`43760`.)
+
+
+Notable security feature in 3.10.7
+==================================
+
+Converting between :class:`int` and :class:`str` in bases other than 2
+(binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)
+now raises a :exc:`ValueError` if the number of digits in string form is
+above a limit to avoid potential denial of service attacks due to the
+algorithmic complexity. This is a mitigation for `CVE-2020-10735
+<https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10735>`_.
+This limit can be configured or disabled by environment variable, command
+line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion
+length limitation <int_max_str_digits>` documentation.  The default limit
+is 4300 digits in string form.
+
+Notable security feature in 3.10.8
+==================================
+
+The deprecated :mod:`!mailcap` module now refuses to inject unsafe text
+(filenames, MIME types, parameters) into shell commands. Instead of using such
+text, it will warn and act as if a match was not found (or for test commands,
+as if the test failed).
+(Contributed by Petr Viktorin in :gh:`98966`.)
+
+Notable changes in 3.10.12
+==========================
+
+tarfile
+-------
+
+* The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`,
+  have a new a *filter* argument that allows limiting tar features than may be
+  surprising or dangerous, such as creating files outside the destination
+  directory.
+  See :ref:`tarfile-extraction-filter` for details.
+  In Python 3.12, use without the *filter* argument will show a
+  :exc:`DeprecationWarning`.
+  In Python 3.14, the default will switch to ``'data'``.
+  (Contributed by Petr Viktorin in :pep:`706`.)

Doc/whatsnew/3.12.rst

@@ -1956,6 +1956,8 @@ Build Changes
   :file:`!configure`.
   (Contributed by Christian Heimes in :gh:`89886`.)
 
+* Windows builds and macOS installers from python.org now use OpenSSL 3.0.
+
 
 C API Changes
 =============

Doc/whatsnew/3.13.rst

@@ -160,6 +160,21 @@ Other Language Changes
   (Contributed by Levi Sabah, Zackery Spytz and Hugo van Kemenade in
   :gh:`73965`.)
 
+* Add :exc:`PythonFinalizationError` exception. This exception derived from
+  :exc:`RuntimeError` is raised when an operation is blocked during
+  the :term:`Python finalization <interpreter shutdown>`.
+
+  The following functions now raise PythonFinalizationError, instead of
+  :exc:`RuntimeError`:
+
+  * :func:`_thread.start_new_thread`.
+  * :class:`subprocess.Popen`.
+  * :func:`os.fork`.
+  * :func:`os.forkpty`.
+
+  (Contributed by Victor Stinner in :gh:`114570`.)
+
+
 New Modules
 ===========