-
-
Notifications
You must be signed in to change notification settings - Fork 30.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gh-115133: test_xml_etree.py: Fix for Expat >=2.6.0 with reparse deferral (fixes #115133) #115138
gh-115133: test_xml_etree.py: Fix for Expat >=2.6.0 with reparse deferral (fixes #115133) #115138
Conversation
3c711d7
to
a147a3e
Compare
It rather defeats the purpose of the test. The purpose of the incremental parsing is that you can process the XML file without reading all data and building the full tree. This test tests that you get the parsed data as fast as the corresponding input data is available. The modified test only tests that it is available after feeding all input data and closing the parser. But you do not need the incremental parsing for this. |
I wonder how this affects stuff like XMPP where you actually need incremental parsing for things to work at all. |
I'm not sure how to guarantee that one gets the data "as fast as the input", but it should at least be possible to assert that some parsing happens before
Probably not well, at least in general. With XMPP specifically, you may get lucky and always send a whole packet for parsing at once (or just generally have small tokens), in which case there will be no incomplete tokens and therefore no need for reparse deferral. But it will be very implementation-dependent. For such a special usecase, this should work as a "flush" operation:
|
In case it's relevant to anyone: Python 3.11.8, built with the patch from this pull request against expat 2.6.0, and also using expat 2.6.0 after installing said Python package, is working just fine for me, both for running Gajim (an XMPP client written in Python) and for running a matrix-synapse server. |
Thank you for your PR, but it was fixed by #115164 which keeps some of more strict testing. |
…GH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from #115138 (comment) . ### Notes - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka.
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka.
…-52425) (GH-115623) (GH-116248) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from #115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. (cherry picked from commit 6a95676) (cherry picked from commit 73807eb) (cherry picked from commit eda2963) --------- Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. Co-authored-by: Gregory P. Smith <[email protected]>
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
…-52425) (GH-115623) (GH-116248) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python/cpython#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. (cherry picked from commit 6a95676bb526261434dd068d6c49927c44d24a9b) (cherry picked from commit 73807eb634315f70a464a18feaae33d9e065de09) (cherry picked from commit eda2963378a3c292cf6bb202bb00e94e46ee6d90) --------- Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. Co-authored-by: Gregory P. Smith <[email protected]> CPython-sync-commit-latest: 0a01ed6c2a116bd3e174fce33c21d84d650de569
…-52425) (GH-115623) (#116268) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from #115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. (cherry picked from commit 6a95676) (cherry picked from commit 73807eb) (cherry picked from commit eda2963) --------- Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. Co-authored-by: Gregory P. Smith <[email protected]>
…-52425) (GH-115623) (GH-116270) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from #115138 (comment) . Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. Co-authored-by: Gregory P. Smith <[email protected]>
…52425) (GH-115623) (GH-116272) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from #115138 (comment) . Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. Co-authored-by: Gregory P. Smith <[email protected]>
…52425) (GH-115623) (GH-116275) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from #115138 (comment) . Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. Co-authored-by: Gregory P. Smith <[email protected]>
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . ### Notes - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka.
…52425) (pythonGH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python#115138 (comment) . ### Notes - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka.
… (GH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from python/cpython#115138 (comment) . - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <[email protected]> and by core dev Serhiy Storchaka. (cherry picked from commit 6a95676)
Fixes #115133