GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy__` by picnixz · Pull Request #133010 · python/cpython

Conversation

@picnixz

@picnixz picnixz commented Apr 26, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: UAF: xml.etree.ElementTree.Element.__deepcopy__ when concurrent mutations happen #133009

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@picnixz
fix UAF in xml.etree.ElementTree.Element.__deepcopy__
2b5be5f
@bedevere-app bedevere-app Bot added the awaiting core review label Apr 26, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Apr 26, 2025
UAF: xml.etree.ElementTree.Element.__deepcopy__ when concurrent mutations happen #133009
Closed
@picnixz picnixz added the topic-XML label Apr 26, 2025
@picnixz
picnixz requested a review from vstinner April 26, 2025 13:58
@picnixz picnixz added the needs backport to 3.13 only security fixes label Apr 26, 2025
@picnixz
picnixz requested a review from serhiy-storchaka April 26, 2025 14:35
serhiy-storchaka
serhiy-storchaka reviewed Apr 26, 2025
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not enough. "tag", "text" and "tail" can be set to different value during deepcopying. If it was the last reference, the deepcopy() argument can be destroyed, and the reference can became handling.

The safe way is to increase the reference count of the deepcopy() argument before calling any code that can release the GIL.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@picnixz

picnixz commented Apr 26, 2025

Copy link
Copy Markdown
Member Author

I actually tried to do something with tag etc, but I wasn't able to crash the interpreter.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented Apr 26, 2025

Copy link
Copy Markdown
Member

String literals are interned and saved in the constants list. You need to use something having a single reference. Try 'tag'.upper().

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Apr 26, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

The safe way is to increase the reference count of the deepcopy() argument before calling any code that can release the GIL.

I've actually removed this code because I thought it wasn't needed, but I'll check with an evil non-interned string tomorrow.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz added 2 commits April 27, 2025 11:48
@picnixz
always protect against possible mutations in __deepcopy__
721481d
@picnixz
remove bad assertion
cef5146
@picnixz
picnixz requested a review from serhiy-storchaka April 27, 2025 10:16
@picnixz

picnixz commented Apr 27, 2025

Copy link
Copy Markdown
Member Author

I'll try to add more tests later as well.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz
picnixz commented Apr 27, 2025
View reviewed changes
Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
Update Modules/_elementtree.c
8297c8a
JelleZijlstra
JelleZijlstra reviewed Apr 28, 2025
View reviewed changes
Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

JelleZijlstra
JelleZijlstra reviewed Apr 28, 2025
View reviewed changes
Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
use Py_CLEAR to be on the safe side
7f8ed28
JelleZijlstra
JelleZijlstra approved these changes Apr 28, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Apr 28, 2025
serhiy-storchaka
serhiy-storchaka reviewed May 1, 2025
View reviewed changes
Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
avoid killing deepcopy() optimization
c5badac
@picnixz
picnixz requested a review from serhiy-storchaka May 7, 2025 12:36
picnixz
picnixz commented May 7, 2025
View reviewed changes
Comment thread Modules/_elementtree.c Outdated

if (Py_REFCNT(object) == 1) {
if (PyDict_CheckExact(object)) {
// Exact dictionaries do not execute arbitrary code as it's

@picnixz picnixz May 7, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@serhiy-storchaka is this assumption correct? namely, here I don't need to incref object temporarily right?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@serhiy-storchaka serhiy-storchaka May 9, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyDict_Next() does not use __iter__, so this comment is redundant. PyDict_Next() does not call any user code.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@picnixz picnixz added the needs backport to 3.14 bugs and security fixes label May 7, 2025
serhiy-storchaka
serhiy-storchaka reviewed May 9, 2025
View reviewed changes
Comment thread Modules/_elementtree.c Outdated

if (Py_REFCNT(object) == 1) {
if (PyDict_CheckExact(object)) {
// Exact dictionaries do not execute arbitrary code as it's

@serhiy-storchaka serhiy-storchaka May 9, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyDict_Next() does not use __iter__, so this comment is redundant. PyDict_Next() does not call any user code.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
Merge remote-tracking branch 'upstream/main' into fix/xml/uaf-deepcop…
f56a368
…y-133009
@picnixz

picnixz commented May 9, 2025

Copy link
Copy Markdown
Member Author

Actually, it's much more tricky than what I thought. Even with the INCREF/DECREF and the additional checks, the following still crashes, but not during deepcopy() but during de-allocation:

class Evil(ET.Element):
    def __deepcopy__(self, memo):
        root.append(ET.Element('y'))
        root.append(ET.Element('z'))
        return self

Y = Evil('y')
root = ET.Element('a')
root.extend([Evil('x'), ET.Element('t'), Y])
c = deepcopy(root)
print(list(c))
print("ok")
assert 0

So I'll need a bit more work.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented May 9, 2025

Copy link
Copy Markdown
Member

#133010 (comment) should help. It is good that you already have a reproducer.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented May 9, 2025

Copy link
Copy Markdown
Member

Ah, you have a different issue -- growing children, not attributes. The solution should be the same -- either ignore new items or resize the array in process.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented May 9, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

The solution should be the same -- either ignore new items or resize the array in process.

Yes, actually I found a way to fix it in the meantime (but the same can be said)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
handle more evil mutations
d585b09
serhiy-storchaka
serhiy-storchaka approved these changes May 9, 2025
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_xml_etree.py
root = ET.Element('a')
evil = X('x')
root.extend([evil, ET.Element('y')])
if is_python_implementation():

@serhiy-storchaka serhiy-storchaka May 9, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also make the C implementation raising RuntimeError. It is fine either way.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@picnixz picnixz May 10, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll postpone this for a future PR as I want to backport this one to 3.13 and 3.14 first.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@picnixz
picnixz merged commit 116a9f9 into python:main May 10, 2025
@miss-islington-app

miss-islington-app Bot commented May 10, 2025

Copy link
Copy Markdown

Thanks @picnixz for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
picnixz deleted the fix/xml/uaf-deepcopy-133009 branch May 10, 2025 07:32
@bedevere-app bedevere-app Bot removed the awaiting merge label May 10, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 10, 2025
@picnixz @miss-islington
pythongh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy…
32edeb3
…__` (pythonGH-133010)

(cherry picked from commit 116a9f9)

Co-authored-by: Bénédikt Tran <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 10, 2025
@picnixz @miss-islington
pythongh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy…
94eeb29
…__` (pythonGH-133010)

(cherry picked from commit 116a9f9)

Co-authored-by: Bénédikt Tran <[email protected]>
@bedevere-app

bedevere-app Bot commented May 10, 2025

Copy link
Copy Markdown

GH-133805 is a backport of this pull request to the 3.14 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label May 10, 2025
@bedevere-app

bedevere-app Bot commented May 10, 2025

Copy link
Copy Markdown

GH-133806 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label May 10, 2025
picnixz added a commit that referenced this pull request May 10, 2025
@miss-islington @picnixz
[3.13] gh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcop…
9718880
…y__` (GH-133010) (#133806)

gh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy__` (GH-133010)
(cherry picked from commit 116a9f9)

Co-authored-by: Bénédikt Tran <[email protected]>
picnixz added a commit that referenced this pull request May 10, 2025
@miss-islington @picnixz
[3.14] gh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcop…
62109c4
…y__` (GH-133010) (#133805)

gh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy__` (GH-133010)
(cherry picked from commit 116a9f9)

Co-authored-by: Bénédikt Tran <[email protected]>
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
@picnixz @Pranjal095
pythongh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy…
5d826c5
…__` (python#133010)
taegyunkim pushed a commit to taegyunkim/cpython that referenced this pull request Aug 4, 2025
@picnixz @taegyunkim
pythongh-133009: fix UAF in `xml.etree.ElementTree.Element.__deepcopy…
86a762f
…__` (python#133010)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JelleZijlstra JelleZijlstra JelleZijlstra approved these changes
@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes
@vstinner vstinner Awaiting requested review from vstinner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-XML

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@picnixz @serhiy-storchaka @JelleZijlstra
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.