GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-135557: Use atomic writes on heapq operations by xuantengh · Pull Request #135601 · python/cpython

Conversation

@xuantengh

@xuantengh xuantengh commented Jun 17, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Use atomic writes on heapq operation.

  • Issue: data race between lock free list reads and heapq #135557

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@xuantengh
xuantengh requested a review from rhettinger as a code owner June 17, 2025 05:22
@bedevere-app bedevere-app Bot added the awaiting review label Jun 17, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Jun 17, 2025
data race between lock free list reads and heapq #135557
Closed
@xuantengh xuantengh changed the title gh-135557: Fix lock-free list access on heapq gh-135557: Use atomic writes on heapq operations Jun 17, 2025
@xuantengh

xuantengh commented Jun 17, 2025

Copy link
Copy Markdown
Contributor Author

I think this could be "skip news"?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@rhettinger
rhettinger removed their request for review June 17, 2025 12:51
@ZeroIntensity

ZeroIntensity commented Jun 17, 2025

Copy link
Copy Markdown
Member

I think this could be "skip news"?

This is user-facing, so this does need a news. Something like "Fix race with lists and heapq on the free threaded build" would be fine. It would also be good to get a test. (Sorry for pushing back on both your PRs!)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ZeroIntensity ZeroIntensity added the needs backport to 3.14 bugs and security fixes label Jun 17, 2025
ZeroIntensity
ZeroIntensity reviewed Jun 17, 2025
View reviewed changes
Comment thread Modules/_heapqmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@xuantengh
xuantengh force-pushed the heap-list-lock-free-read branch from f7fbb68 to 08770ec Compare June 17, 2025 16:16
ZeroIntensity
ZeroIntensity reviewed Jun 17, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2025-06-17-23-13-56.gh-issue-135557.Bfcy4v.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303
kumaraditya303 reviewed Jun 17, 2025
View reviewed changes
Comment thread Modules/_heapqmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@python-cla-bot

python-cla-bot Bot commented Jun 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@xuantengh
xuantengh force-pushed the heap-list-lock-free-read branch from 0184b21 to 463963d Compare June 18, 2025 06:38
kumaraditya303
kumaraditya303 reviewed Jun 18, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303
kumaraditya303 reviewed Jun 18, 2025
View reviewed changes
Comment thread Modules/_heapqmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@xuantengh
xuantengh force-pushed the heap-list-lock-free-read branch from 463963d to c49ff7d Compare June 19, 2025 00:48
kumaraditya303
kumaraditya303 reviewed Jun 19, 2025
View reviewed changes
Comment thread Modules/_heapqmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
xuantengh and others added 10 commits June 19, 2025 18:30
@xuantengh
fix lock-free list access on heapq
86d3fbb
@xuantengh
use FT_ATOMIC macro to store ptr
5f24742
@xuantengh
add news entry
06328a1
@xuantengh
add test
3b8ea80
@xuantengh
add missing header
481ef22
@xuantengh @ZeroIntensity
Update Misc/NEWS.d/next/Library/2025-06-17-23-13-56.gh-issue-135557.B…
d00d698 
…fcy4v.rst

Co-authored-by: Peter Bierma <[email protected]>
@xuantengh
use relaxed atomic store
dd8f5ce
@xuantengh
update tests
b7528bd
@xuantengh
reduce test num elements
925a093
@xuantengh
use ft store in heappushpop and heapreplace as well
27d6cb6
@xuantengh
xuantengh force-pushed the heap-list-lock-free-read branch from c49ff7d to 27d6cb6 Compare June 19, 2025 10:31
ZeroIntensity
ZeroIntensity reviewed Jun 19, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_free_threading/test_heapq.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@xuantengh
xuantengh force-pushed the heap-list-lock-free-read branch from 2cc859d to 8cf4794 Compare June 19, 2025 13:17
@xuantengh
use run_concurrent to spawn threads
32d767f
@xuantengh
xuantengh force-pushed the heap-list-lock-free-read branch from 8cf4794 to 32d767f Compare June 19, 2025 13:42
@xuantengh
fix
4fb9389
kumaraditya303
kumaraditya303 reviewed Jun 19, 2025
View reviewed changes
Comment thread Modules/_heapqmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@xuantengh
fix style
a10560e
kumaraditya303
kumaraditya303 approved these changes Jun 20, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Jun 20, 2025
@ZeroIntensity

ZeroIntensity commented Jun 20, 2025

Copy link
Copy Markdown
Member

!buildbot nogil
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Jun 20, 2025

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @ZeroIntensity for commit a10560e 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F135601%2Fmerge

The command will test the builders whose names match following regular expression: nogil

The builders matched are:

  • ARM64 MacOS M1 NoGIL PR
  • AMD64 Windows Server 2022 NoGIL PR
  • aarch64 Fedora Rawhide NoGIL refleaks PR
  • AMD64 Fedora Rawhide NoGIL PR
  • aarch64 Fedora Rawhide NoGIL PR
  • ARM64 MacOS M1 Refleaks NoGIL PR
  • AMD64 CentOS9 NoGIL Refleaks PR
  • AMD64 Fedora Rawhide NoGIL refleaks PR
  • s390x Fedora Rawhide NoGIL refleaks PR
  • PPC64LE Fedora Rawhide NoGIL PR
  • PPC64LE Fedora Rawhide NoGIL refleaks PR
  • x86-64 MacOS Intel NoGIL PR
  • AMD64 CentOS9 NoGIL PR
  • x86-64 MacOS Intel ASAN NoGIL PR
  • AMD64 Windows PGO NoGIL PR
  • s390x Fedora Rawhide NoGIL PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
ZeroIntensity
ZeroIntensity approved these changes Jun 20, 2025
View reviewed changes

@ZeroIntensity ZeroIntensity left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well, assuming buildbots pass.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@kumaraditya303
kumaraditya303 merged commit 13cac83 into python:main Jun 21, 2025
@miss-islington-app

miss-islington-app Bot commented Jun 21, 2025

Copy link
Copy Markdown

Thanks @xuantengh for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting merge label Jun 21, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 21, 2025
@xuantengh @miss-islington
pythongh-135557: use atomic stores in heapq operations in free-thre…
753cec2 
…ading (pythonGH-135601)

(cherry picked from commit 13cac83)

Co-authored-by: Xuanteng Huang <[email protected]>
@bedevere-app

bedevere-app Bot commented Jun 21, 2025

Copy link
Copy Markdown

GH-135787 is a backport of this pull request to the 3.14 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jun 21, 2025
YvesDup
YvesDup reviewed Jun 21, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_heapq.py
except IndexError:
pass

self.run_concurrently(worker, (), n_threads * 2)

@YvesDup YvesDup Jun 21, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test is successful here because the worker function has no arguments.
But shouldn't the second argument args be a tuple?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@kumaraditya303 kumaraditya303 Jun 21, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't it an empty tuple here?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
ZeroIntensity reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
kumaraditya303 pushed a commit that referenced this pull request Jun 21, 2025
@miss-islington @xuantengh
[3.14] gh-135557: use atomic stores in heapq operations in free-thr…
e388b29 
…eading (GH-135601) (#135787)

gh-135557: use atomic stores in `heapq` operations in free-threading (GH-135601)
(cherry picked from commit 13cac83)

Co-authored-by: Xuanteng Huang <[email protected]>
@YvesDup

YvesDup commented Jun 23, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

isn't it an empty tuple here?

Oh yes it is.
Reading an empty tuple for arguments when creating new threads destabilized me (and then silly). This is because when creating a new Thread with no arguments, the args argument is always omitted (the default is an empty tuple).
Sorry for the noise.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@xuantengh
xuantengh deleted the heap-list-lock-free-read branch June 23, 2025 16:26
@kumaraditya303 kumaraditya303 added the topic-free-threading label Jun 24, 2025
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
@xuantengh @Pranjal095
pythongh-135557: use atomic stores in heapq operations in free-thre…
442de3b 
…ading (python#135601)
taegyunkim pushed a commit to taegyunkim/cpython that referenced this pull request Aug 4, 2025
@xuantengh @taegyunkim
pythongh-135557: use atomic stores in heapq operations in free-thre…
bb95a55 
…ading (python#135601)
Agent-Hellboy pushed a commit to Agent-Hellboy/cpython that referenced this pull request Aug 19, 2025
@xuantengh @Agent-Hellboy
pythongh-135557: use atomic stores in heapq operations in free-thre…
a25c15e 
…ading (python#135601)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mpage mpage mpage left review comments
@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes
@ZeroIntensity ZeroIntensity ZeroIntensity approved these changes
+1 more reviewer
@YvesDup YvesDup YvesDup left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@xuantengh @ZeroIntensity @bedevere-bot @YvesDup @mpage @kumaraditya303
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.