GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-139400: Make sure that parent parsers outlive their subparsers in `pyexpat` by hartwork · Pull Request #139403 · python/cpython

Conversation

@hartwork

@hartwork hartwork commented Sep 28, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

CC @picnixz

  • Issue: Heap-use-after-free issue in pyexpat related to .ExternalEntityParserCreate #139400

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
hartwork added 3 commits September 29, 2025 00:17
@hartwork
pyexpat.c: Disallow collection of in-use parent parsers
e1c0470
Within libexpat, a parser created via XML_ExternalEntityParserCreate
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser
to be garbage-collected by CPython. This fixes related reference
counting, to stop that from happening.
@hartwork
Add a regression test for issue 139400
3b0afda
@hartwork
Add news item for issue 139400
8f70991
@bedevere-app bedevere-app Bot added the awaiting review label Sep 28, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Sep 28, 2025
Heap-use-after-free issue in pyexpat related to .ExternalEntityParserCreate #139400
Closed
@picnixz
picnixz self-requested a review September 28, 2025 22:45
@hartwork
Improve test's docstring
a63bd34
picnixz
picnixz reviewed Sep 28, 2025
View reviewed changes

@picnixz picnixz left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, this looks like Tcl/Tk and their callbacks. If you want to check for reference leaks, you should also run the tests with -R (python -m test -R the_test)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

hartwork added 2 commits September 29, 2025 01:04
@hartwork
Simplify xmlparseobject_ to xmlparseobject
22417ec
Idea by @picnixz
@hartwork
Leverage Py_CLEAR
20caacd
Idea by @picnixz
@hartwork

hartwork commented Sep 28, 2025

Copy link
Copy Markdown
Contributor Author

Now, this looks like Tcl/Tk and their callbacks.

@picnixz I'm not sure what that means 😃

If you want to check for reference leaks, you should also run the tests with -R (python -m test -R the_test)

I cannot figure out why or what to do with python -m test -R the_test. Could you spell it out for me? 🙏

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hartwork
hartwork requested a review from picnixz September 28, 2025 23:13
@picnixz

picnixz commented Sep 28, 2025

Copy link
Copy Markdown
Member

I cannot figure out why or what to do with python -m test -R the_test. Could you spell it out for me? 🙏

Sorry, to be precise the -R : option checks if there are reference leaks. In your case run:

$ ./python -m test test_pyexpat -m test_parent_parser_outlives_its_subparsers -R :
hartwork reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Sep 28, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

I'm not sure what that means

Tcl/Tk is a graphic library that allows to have delayed callbacks and the callback data is the Python object. Here, for Expat, the handler data is the XML parser itself as well. I wanted to make a parallel between those two constructions.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz
picnixz reviewed Sep 28, 2025
View reviewed changes

@picnixz picnixz left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit tired (it's 01:33 AM here) so the review might not be perfect. I'll have another look tomorrow.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hartwork reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_pyexpat.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_pyexpat.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@hartwork

hartwork commented Sep 28, 2025

Copy link
Copy Markdown
Contributor Author

I cannot figure out why or what to do with python -m test -R the_test. Could you spell it out for me? 🙏

Sorry, to be precise the -R : option checks if there are reference leaks. In your case run:

$ ./python -m test test_pyexpat -m test_parent_parser_outlives_its_subparsers -R :

@picnixz after a recompile with --with-pydebug that worked — thank you! 🙏

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hartwork added 3 commits September 29, 2025 01:45
@hartwork
Simplify call to Py_CLEAR
f8c58a3
Idea by @picnixz
@hartwork
Streamline docsting of ParentParserLifetimeTest
f1e3e36
Idea by @picnixz
@hartwork
pyexpat.c: Adjust whitespace around pointer
1b2937d
As requested.
@hartwork

hartwork commented Sep 28, 2025

Copy link
Copy Markdown
Contributor Author

I'm a bit tired (it's 01:33 AM here) so the review might not be perfect. I'll have another look tomorrow.

@picnixz no worries, same UTC+2 here. Thank you! 👍 👍

yihong0618 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hartwork

hartwork commented Sep 28, 2025

Copy link
Copy Markdown
Contributor Author

Related: libexpat/libexpat#1066

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hartwork
hartwork requested a review from picnixz September 29, 2025 14:06
picnixz
picnixz reviewed Sep 29, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-09-29-00-01-28.gh-issue-139400.X2T-jO.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@hartwork
Re-write news item
8c5a4e8
As suggested by @picnixz
@hartwork
hartwork requested a review from picnixz September 29, 2025 21:04
@hartwork hartwork mentioned this pull request Sep 30, 2025
[3.14] gh-90949: add Expat API to prevent XML deadly allocations (CVE-2025-59375) (GH-139234) #139359
Merged
@hartwork
Turn parent into a PyObject pointer
a6f5520
Suggested by @picnixz
picnixz
picnixz reviewed Sep 30, 2025
View reviewed changes
Comment thread Modules/pyexpat.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

hartwork added 2 commits September 30, 2025 15:49
@hartwork
Add missing parent visit to xmlparse_traverse
4470cea
Kudos to @picnixz
@hartwork
Re-write comment about mission of "PyObject *parent"
3759ac7
As suggested by @picnixz
@picnixz

picnixz commented Oct 5, 2025

Copy link
Copy Markdown
Member

Lucky me, 3.14 has a clean backport :)

hugovk reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Oct 5, 2025

Copy link
Copy Markdown
Member

@hartwork Could you make the backports for 3.10-3.13 please? I'm not on my dev session now (sorry for bothering you with this).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hartwork

hartwork commented Oct 5, 2025

Copy link
Copy Markdown
Contributor Author

@picnixz thanks for your help getting this over the finish line 🙏 🎉 I can look into the backports for this and report back if I hit any roadblocks.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Oct 5, 2025

Copy link
Copy Markdown
Member

Thank you for writing libexpat :')

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hartwork added a commit to hartwork/cpython that referenced this pull request Oct 5, 2025
@hartwork
pythongh-139400: Make sure that parent parsers outlive their subparse…
ae0a3c3
…rs in `pyexpat` (python#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139608 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Oct 5, 2025
hugovk pushed a commit that referenced this pull request Oct 5, 2025
@miss-islington @hartwork
[3.14] gh-139400: Make sure that parent parsers outlive their subpars…
66ca0d9
…ers in `pyexpat` (GH-139403) (#139606)

Co-authored-by: Sebastian Pipping <[email protected]>
hartwork added a commit to hartwork/cpython that referenced this pull request Oct 5, 2025
@hartwork
pythongh-139400: Make sure that parent parsers outlive their subparse…
65730f3
…rs in `pyexpat` (python#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139609 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Oct 5, 2025
hartwork added a commit to hartwork/cpython that referenced this pull request Oct 5, 2025
@hartwork
pythongh-139400: Make sure that parent parsers outlive their subparse…
37f71cb
…rs in `pyexpat` (python#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139612 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.11 only security fixes label Oct 5, 2025
@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139608 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139609 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139612 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hartwork added a commit to hartwork/cpython that referenced this pull request Oct 5, 2025
@hartwork
pythongh-139400: Make sure that parent parsers outlive their subparse…
135a66e
…rs in `pyexpat` (python#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139613 is a backport of this pull request to the 3.10 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.10 label Oct 5, 2025
hartwork added a commit to hartwork/cpython that referenced this pull request Oct 5, 2025
@hartwork
pythongh-139400: Make sure that parent parsers outlive their subparse…
3334e29
…rs in `pyexpat` (python#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
@bedevere-app

bedevere-app Bot commented Oct 5, 2025

Copy link
Copy Markdown

GH-139614 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou pushed a commit that referenced this pull request Oct 6, 2025
@hartwork
[3.13] gh-139400: Make sure that parent parsers outlive their subpars…
69ab8fb
…ers in `pyexpat` (GH-139403) (GH-139608)

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
@hartwork hartwork mentioned this pull request Oct 6, 2025
[3.9] gh-139400: Make sure that parent parsers outlive their subparsers in pyexpat (GH-139403) #139614
Merged
pablogsal pushed a commit that referenced this pull request Oct 6, 2025
@hartwork
[3.11] gh-139400: Make sure that parent parsers outlive their subpars…
1459d1f
…ers in `pyexpat` (GH-139403) (#139612)

* gh-139400: Make sure that parent parsers outlive their subparsers in `pyexpat` (#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)

* Move news item from section "Core and Builtins" to section "Security"
pablogsal pushed a commit that referenced this pull request Oct 6, 2025
@hartwork
[3.10] gh-139400: Make sure that parent parsers outlive their subpars…
8ea678d
…ers in `pyexpat` (GH-139403) (#139613)

* gh-139400: Make sure that parent parsers outlive their subparsers in `pyexpat` (#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)

* Move news item to from section "Core and Builtins" to section "Security"
@bedevere-bot

bedevere-bot commented Oct 7, 2025

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Refleaks 3.11 (tier-1) has failed when building commit 1459d1f.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1027/builds/823) and take a look at the build logs.
  4. Check if the failure is related to this commit (1459d1f) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1027/builds/823

Failed tests:

  • test_socket

Failed subtests:

  • test_aead_aes_gcm - test.test_socket.LinuxKernelCryptoAPI.test_aead_aes_gcm

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64.refleak/build/Lib/test/test_socket.py", line 6557, in test_aead_aes_gcm
    op.sendall(plain)
OSError: [Errno 22] Invalid argument
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

danigm pushed a commit to openSUSE-Python/cpython that referenced this pull request Oct 7, 2025
@hartwork @danigm
pythongh-139400: Make sure that parent parsers outlive their subparse…
4e450d1
…rs in `pyexpat` (python#139403)

* Modules/pyexpat.c: Disallow collection of in-use parent parsers.

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.
ambv pushed a commit that referenced this pull request Oct 7, 2025
@hartwork
[3.9] gh-139400: Make sure that parent parsers outlive their subparse…
598165e
…rs in `pyexpat` (GH-139403) (GH-139614)

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
ambv pushed a commit that referenced this pull request Oct 7, 2025
@hartwork
[3.12] gh-139400: Make sure that parent parsers outlive their subpars…
dea7e3d
…ers in `pyexpat` (GH-139403) (GH-139609)

Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.

(cherry picked from commit 6edb2dd)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@picnixz picnixz picnixz approved these changes
@ZeroIntensity ZeroIntensity ZeroIntensity left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@picnixz picnixz

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@hartwork @picnixz @hugovk @bedevere-bot @ZeroIntensity
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.