GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-129069: Fix listobject.c data races due to memmove by colesbury · Pull Request #142957 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Dec 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The use of memmove and _Py_memory_repeat were not thread-safe in the free threading build in some cases. In theory, memmove and _Py_memory_repeat can copy byte-by-byte instead of pointer-by-pointer, so concurrent readers could see uninitialized data or tearing.

Additionally, we should be using "release" (or stronger) ordering to be compliant with the C11 memory model when copying objects within a list.

I've also updated list_resize so it definitively doesn't fail when shrinking a list. This simplifies some of the call sites.

  • Issue: Race in concurrent list mutation and item retrieval #129069

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@colesbury
pythongh-129069: Fix listobject.c data races due to memmove
97e8d13
The use of memmove and _Py_memory_repeat were not thread-safe in the
free threading build in some cases. In theory, memmove and
_Py_memory_repeat can copy byte-by-byte instead of pointer-by-pointer,
so concurrent readers could see uninitialized data or tearing.

Additionally, we should be using "release" (or stronger) ordering to be
compliant with the C11 memory model when copying objects within a list.
@colesbury colesbury added skip news topic-free-threading labels Dec 18, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Dec 18, 2025
Race in concurrent list mutation and item retrieval #129069
Closed
@colesbury colesbury added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Dec 18, 2025
@bedevere-bot

bedevere-bot commented Dec 18, 2025

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 97e8d13 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F142957%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Dec 18, 2025
@colesbury
Add _Py_IsOwnedByCurrentThread() special case for list_atomic_memmove
11c64a6
@colesbury
colesbury marked this pull request as ready for review December 19, 2025 18:42
@bedevere-app bedevere-app Bot added the awaiting core review label Dec 19, 2025
@colesbury colesbury mentioned this pull request Dec 19, 2025
gh-129069: make list ass_slice and memory_repeat safe in free-threading #131882
Closed
@colesbury
colesbury requested review from Yhg1s and kumaraditya303 December 19, 2025 18:44
Yhg1s
Yhg1s approved these changes Dec 19, 2025
View reviewed changes
Comment thread Objects/listobject.c Outdated
Comment on lines +1094 to +1100
PyObject **src = items;
PyObject **dst = items + input_size;
Py_ssize_t remaining = output_size - input_size;
while (remaining > 0) {
_Py_atomic_store_ptr_release(dst++, *src++);
remaining--;
}

@Yhg1s Yhg1s Dec 19, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't this just call list_atomic_memmove?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@colesbury colesbury Dec 19, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that will simplify this a bit and avoid the special case here (since list_atomic_memmove has it internally).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Objects/listobject.c Outdated
goto Error;
}
Py_ssize_t tail = Py_SIZE(a) - ihigh;
list_atomic_memmove(a, &item[ihigh+d], &item[ihigh], tail);

@Yhg1s Yhg1s Dec 19, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so much nicer, I wish we'd refactored this the first time round :P

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Objects/listobject.c Outdated
// Pointer-by-pointer memmove for PyObject** arrays that is safe
// for shared lists in Py_GIL_DISABLED builds.
static void
list_atomic_memmove(PyListObject *a, PyObject **dest, PyObject **src, Py_ssize_t n)

@Yhg1s Yhg1s Dec 19, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of the name (it's not the memmove as a whole that's atomic, just the individual writes) but I can't think of a better name and I think the comment explains it well enough.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@colesbury colesbury Dec 19, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's a bit misleading. I can name it ptr_wise_atomic_memmove (as in Byte-wise atomic memcpy)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Dec 19, 2025
colesbury added 2 commits December 19, 2025 16:26
@colesbury
Changes from review
44226b4
@colesbury
Merge branch 'main' into pythongh-129069-list-memmove
761cad1
@colesbury
colesbury merged commit e46f28c into python:main Dec 19, 2025
44 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@colesbury
colesbury deleted the gh-129069-list-memmove branch December 19, 2025 23:06
@bedevere-app bedevere-app Bot removed the awaiting merge label Dec 19, 2025
@kumaraditya303 kumaraditya303 mentioned this pull request Dec 20, 2025
gh-129069: fix more thread safety issues in list #143019
Merged
cocolato pushed a commit to cocolato/cpython that referenced this pull request Dec 22, 2025
@colesbury @cocolato
pythongh-129069: Fix listobject.c data races due to memmove (pythongh…
a3c229f
…-142957)

The use of memmove and _Py_memory_repeat were not thread-safe in the
free threading build in some cases. In theory, memmove and
_Py_memory_repeat can copy byte-by-byte instead of pointer-by-pointer,
so concurrent readers could see uninitialized data or tearing.

Additionally, we should be using "release" (or stronger) ordering to be
compliant with the C11 memory model when copying objects within a list.
@nascheme nascheme mentioned this pull request Jul 14, 2026
Backport free-threading memory-safety fixes to 3.14 and 3.15 #153714
Closed
@bedevere-app

bedevere-app Bot commented Jul 16, 2026

Copy link
Copy Markdown

GH-153791 is a backport of this pull request to the 3.14 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nascheme added a commit that referenced this pull request Jul 17, 2026
@nascheme @eendebakpt @kumaraditya303
[3.14] gh-123471: Free-threading fixes for itertools (GH-131212) (GH-…
45bfb43
…132814) (GH-135689) (GH-144402) (GH-146033) (GH-142957) (GH-153791)

Combined backport of PRs from main branch, fixing free-threading data-races in itertools:

* gh-123471: make concurrent iteration over `itertools.cycle` safe under free-threading (gh-131212)
* gh-123471: Make itertools.product and itertools.combinations thread-safe (GH-132814)
* gh-123471: Make itertools.chain thread-safe (gh-135689)
* gh-123471: Make concurrent iteration over `itertools.permutations` and `itertools.combinations_with_replacement` thread-safe (gh-144402)
* gh-123471: make concurrent iteration over itertools.accumulate thread-safe (gh-144486)
* gh-123471: Make `itertools.zip_longest` safe in the FT build (gh-146033)

(cherry picked from commit 26a1cd4)
(cherry picked from commit 847d1c2)
(cherry picked from commit 0533c1f)
(cherry picked from commit 009c8c0)
(cherry picked from commit 3a24856)
(cherry picked from commit 9214e3f)

Co-authored-by: Pieter Eendebak <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Yhg1s Yhg1s Yhg1s approved these changes
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@colesbury @bedevere-bot @Yhg1s
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.