gh-123678: Upgrade libexpat 2.6.3

[sethmlarson]

• Issue: Please upgrade bundled Expat to 2.6.3 (e.g. for the fixes to CVE-2024-45490, CVE-2024-45491 and CVE-2024-45492) #123678

[hartwork]

@sethmlarson verified using the Dockerfile below 👍

# Copyright (c) 2022-2024 Sebastian Pipping <[email protected]>
# Licensed under the Apache License version 2.0

FROM alpine
RUN apk add --update \
            diffutils \
            git \
            sed \
        && \
    git clone --depth 1 https://github.com/python/cpython cpython-main \
        && \
    ( cd cpython-main && git rev-parse HEAD ) \
        && \
    git clone --depth 1 --branch expat-2.6.3 https://github.com/sethmlarson/cpython cpython-pr \
        && \
    ( cd cpython-pr && git rev-parse HEAD ) \
        && \
    git config --global advice.detachedHead false \
        && \
    git clone --depth 1 --branch R_2_6_2 https://github.com/libexpat/libexpat libexpat_2_6_2 \
        && \
    git clone --depth 1 --branch R_2_6_3 https://github.com/libexpat/libexpat libexpat_2_6_3 \
        && \
    diff -r -u libexpat_2_6_2/expat/lib/ cpython-main/Modules/expat/ | tee 2-6-2.diff \
        && \
    diff -r -u libexpat_2_6_3/expat/lib/ cpython-pr/Modules/expat/ | tee 2-6-3.diff \
        && \
    sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-6-2.diff 2-6-3.diff \
        && \
    diff -u 2-6-2.diff 2-6-3.diff \
        && \
    echo 'Diff is good.'

# docker build .
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  3.072kB
Step 1/2 : FROM alpine
 ---> 05455a08881e
Step 2/2 : RUN apk add --update             diffutils             git             sed         &&     git clone --depth 1 https://github.com/python/cpython cpython-main         &&     ( cd cpython-main && git rev-parse HEAD )         &&     git clone --depth 1 --branch expat-2.6.3 https://github.com/sethmlarson/cpython cpython-pr         &&     ( cd cpython-pr && git rev-parse HEAD )         &&     git config --global advice.detachedHead false         &&     git clone --depth 1 --branch R_2_6_2 https://github.com/libexpat/libexpat libexpat_2_6_2         &&     git clone --depth 1 --branch R_2_6_3 https://github.com/libexpat/libexpat libexpat_2_6_3         &&     diff -r -u libexpat_2_6_2/expat/lib/ cpython-main/Modules/expat/ | tee 2-6-2.diff         &&     diff -r -u libexpat_2_6_3/expat/lib/ cpython-pr/Modules/expat/ | tee 2-6-3.diff         &&     sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-6-2.diff 2-6-3.diff         &&     diff -u 2-6-2.diff 2-6-3.diff         &&     echo 'Diff is good.'
 ---> Running in 03fb2eed1a29
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/12) Installing diffutils (3.10-r0)
(2/12) Installing ca-certificates (20240226-r0)
(3/12) Installing brotli-libs (1.1.0-r1)
(4/12) Installing c-ares (1.27.0-r0)
(5/12) Installing libunistring (1.1-r2)
(6/12) Installing libidn2 (2.3.4-r4)
(7/12) Installing nghttp2-libs (1.58.0-r0)
(8/12) Installing libcurl (8.9.0-r0)
(9/12) Installing libexpat (2.6.2-r0)
(10/12) Installing pcre2 (10.42-r2)
(11/12) Installing git (2.43.4-r0)
(12/12) Installing sed (4.9-r2)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20240226-r0.trigger
OK: 19 MiB in 27 packages
Cloning into 'cpython-main'...
Updating files: 100% (5350/5350), done.
d2eafe2f48aac31aa8a152620bdfd0f2a274ee1d
Cloning into 'cpython-pr'...
Updating files: 100% (5350/5350), done.
e6fe0ac014e8f5b17064c20d1ae937dbd4c761a2
Cloning into 'libexpat_2_6_2'...
Cloning into 'libexpat_2_6_3'...
Only in libexpat_2_6_2/expat/lib/: .gitignore
Only in cpython-main/Modules/expat/: COPYING
Only in libexpat_2_6_2/expat/lib/: Makefile.am
Only in cpython-main/Modules/expat/: expat_config.h
diff -r -u libexpat_2_6_2/expat/lib/expat_external.h cpython-main/Modules/expat/expat_external.h
--- libexpat_2_6_2/expat/lib/expat_external.h   2024-09-04 18:04:32.429990554 +0000
+++ cpython-main/Modules/expat/expat_external.h 2024-09-04 18:04:21.161886969 +0000
@@ -64,6 +64,11 @@
    compiled with the cdecl calling convention as the default since
    system headers may assume the cdecl convention.
 */
+
+/* Namespace external symbols to allow multiple libexpat version to
+   co-exist. */
+#include "pyexpatns.h"
+
 #ifndef XMLCALL
 #  if defined(_MSC_VER)
 #    define XMLCALL __cdecl
Only in libexpat_2_6_2/expat/lib/: libexpat.def.cmake
Only in cpython-main/Modules/expat/: pyexpatns.h
Only in libexpat_2_6_3/expat/lib/: .gitignore
Only in cpython-pr/Modules/expat/: COPYING
Only in libexpat_2_6_3/expat/lib/: Makefile.am
Only in cpython-pr/Modules/expat/: expat_config.h
diff -r -u libexpat_2_6_3/expat/lib/expat_external.h cpython-pr/Modules/expat/expat_external.h
--- libexpat_2_6_3/expat/lib/expat_external.h   2024-09-04 18:04:35.764021202 +0000
+++ cpython-pr/Modules/expat/expat_external.h   2024-09-04 18:04:29.352962267 +0000
@@ -64,6 +64,11 @@
    compiled with the cdecl calling convention as the default since
    system headers may assume the cdecl convention.
 */
+
+/* Namespace external symbols to allow multiple libexpat version to
+   co-exist. */
+#include "pyexpatns.h"
+
 #ifndef XMLCALL
 #  if defined(_MSC_VER)
 #    define XMLCALL __cdecl
Only in libexpat_2_6_3/expat/lib/: libexpat.def.cmake
Only in cpython-pr/Modules/expat/: pyexpatns.h
Diff is good.
 ---> Removed intermediate container 03fb2eed1a29
 ---> e85a4065bbe4
Successfully built e85a4065bbe4

[miss-islington-app]

Thanks @sethmlarson for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-123707 is a backport of this pull request to the 3.13 branch.

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.11

[bedevere-app]

GH-123708 is a backport of this pull request to the 3.12 branch.

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.10

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.9

[sethmlarson]

Ah I'll close the backports that succeeded and handle the rest, thanks @gpshead!

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 40bdb0deee746e51c71c56329df21e5172fd8aa0 3.8

[gpshead]

I'm not sure if this needs further backporting or not, i didn't look to see what versions those used. the specific int vs size_t change for the bugfix could likely be manually applied if relevant though. leaving that up to you.

[sethmlarson]

@gpshead Looks like every branch is upgradeable, so I went ahead and backported to every branch. Those PRs can be closed if we decide not to move forward with them.

[hartwork]

@gpshead Looks like every branch is upgradeable, so I went ahead and backported to every branch.

@sethmlarson 👍 👍