GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-149816: Fix race conditions in `Modules/_elementtree.c` with free-threaded by IvyXu420 · Pull Request #149918 · python/cpython

Conversation

@IvyXu420

@IvyXu420 IvyXu420 commented May 16, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This PR fixes (69) Unsynchronized extra pointer dereference in len in Modules/_elementtree.c mentioned in #149816.
It also fixes race condition in getting attrib.

  • Issue: 22 free-threading race conditions #149816

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
IvyXu420 added 3 commits May 16, 2026 18:14
@IvyXu420
fix: Fix race condition in elementtree with free-threading
94a0e28
@IvyXu420
clear comments
b3cad8b
@IvyXu420
fix
1dc4f36
@bedevere-app bedevere-app Bot added the awaiting review label May 16, 2026
@bedevere-app bedevere-app Bot mentioned this pull request May 16, 2026
22 free-threading race conditions #149816
Open
13 of 24 tasks
IvyXu420 added 3 commits May 16, 2026 19:09
@IvyXu420
fix CI tests
4e16057
@IvyXu420
add news
6d8b559
@IvyXu420
Improve naming of free-threading race test
de9bbf6
dpdani
dpdani reviewed Jun 8, 2026
View reviewed changes
Comment thread Lib/test/test_xml_etree_c.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_xml_etree_c.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Misc/NEWS.d/next/Library/2026-05-16-19-13-58.gh-issue-149816.Ht-jC5.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_elementtree.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@IvyXu420
address review
2f71c50
@IvyXu420

IvyXu420 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@dpdani Thanks for you review!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

IvyXu420 added 2 commits June 10, 2026 22:44
@IvyXu420
PyErr_SetString in element_get_attrib() is removed
bb270e7
@IvyXu420
Merge branch 'main' into fix-149816
8b88271
@devdanzin

devdanzin commented Jul 19, 2026

Copy link
Copy Markdown
Member

Independent confirmation of item (69) "Unsynchronized extra pointer dereference in len in Modules/_elementtree.c", found by ThreadSanitizer fuzzing (fusil --tsan). Worth noting the race is a bit broader than a read-deref: it's a write/write on the lazily-allocated self->extra, because the if (!self->extra) create_extra(...) guard in the extra accessors isn't atomic. Two threads first-touching a shared Element both take the !self->extra branch and both run create_extra, which does self->extra = PyMem_Malloc(...) (_elementtree.c:274) with no critical section — so one allocation is overwritten (leaked) and readers can observe a torn pointer.

Minimal deterministic reproducer (exit 66 under TSan; create_extra:274 as the write, reached via element_attrib_getter / element_length):

import threading
import xml.etree.ElementTree as ET

NTHREADS = 8
barrier = threading.Barrier(NTHREADS)

def worker(elem):
    barrier.wait()
    for _ in range(4000):
        _ = elem.attrib          # if (!self->extra) create_extra(...) -- unlocked lazy init
        _ = len(elem)            # element_length reads self->extra

for _ in range(200):
    shared = ET.Element("tag")   # extra == NULL until first attrib/child touch
    ts = [threading.Thread(target=worker, args=(shared,)) for _ in range(NTHREADS)]
    for t in ts: t.start()
    for t in ts: t.join()

Confirmed still present on current main (3.16.0a0), on both a debug and a release --with-thread-sanitizer build. PR #149918's approach — taking Py_BEGIN_CRITICAL_SECTION(self) around the if (!self->extra) create_extra(...) check-and-create and the other extra accessors — covers this (both the read-deref and the write/write faces). Faces the fuzzer also hit: create_extra | element_length and clear_extra | create_extra.

(Found by fusil --tsan, a ThreadSanitizer fuzzer; draft and reproducer by Claude Code, minimized and reviewed by hand.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

dpdani
dpdani reviewed Jul 27, 2026
View reviewed changes

@dpdani dpdani left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this approach is not going to be effective. While it does resolve the sharpest edges, it still leaves some race conditions that need to be resolved.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_elementtree.c
return -1;
}

Py_BEGIN_CRITICAL_SECTION(self);

@dpdani dpdani Jul 27, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do locking here. There are a few callers of this function, and locking should be moved to the callers. For instance, this code in element_resize would not be thread safe with an inner critical section:

    if (!self->extra) {
        if (create_extra(self, NULL) < 0)
            return -1;
    }

Allocations and init functions need not be thread safe.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_elementtree.c

if (!self->extra)
return;
Py_BEGIN_CRITICAL_SECTION(self);

@dpdani dpdani Jul 27, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not add locking here. The only code path that may call this code concurrently is in element_setstate_from_attributes, and locking should be moved there. It is probably needed anyway by the looks of it.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_elementtree.c
Py_VISIT(JOIN_OBJ(self->text));
Py_VISIT(JOIN_OBJ(self->tail));

Py_BEGIN_CRITICAL_SECTION(self);

@dpdani dpdani Jul 27, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GC runs during a stop-the-world pause, so there's no need for locking here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@github-actions

github-actions Bot commented Oct 27, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 90 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@dpdani dpdani dpdani left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@IvyXu420 @devdanzin @dpdani
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.