Description

Yhg1s

opened on Jan 20, 2025

Issue body actions

Bug report

Bug description:

A data race exists in lists when they are concurrently updated and indexed (or otherwise accessed): for slice assignment and other ways of adding or removing multiple items (list.extend, in-place multiplication), memory writes to the newly allocated areas happen with memcpy/memmove, which have no write size/atomicity guarantees (and it's not unlikely that they will perform jagged writes). These reallocations happen with the lock held, but since we're relying on QSBR, concurrent reads of those memory areas are technically possible.

The tests in PR #128798 (test_free_threading.test_iteration), when run under ThreadSanitizer, expose those races. (They are suppressed in Tools/tsan/suppressions_free_threading.txt).

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-129069: make list ass_slice and memory_repeat safe in free-threading #131882
• gh-129069: Fix listobject.c data races due to memmove #142957
• gh-129069: fix more thread safety issues in list #143019
• gh-129069: fix TSAN-0013 from gh-153852 #154516
• [3.14] gh-129069: fix more thread safety issues in list (GH-143019) #158488

Activity

1. Yhg1s

   added

   topic-free-threading

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jan 20, 2025
2. colesbury commented on Jan 20, 2025

   

   colesbury

   on Jan 20, 2025

   Contributor

   More actions

   I'm a bit confused by the "memory writes to the newly allocated areas happen with memcpy/memmove" part of the issue description.

   The data races with memcpy I've seen involve writes to the existing item array. For example, in list_ass_slice_lock_held, we have memmove calls with self->ob_item and recycle as the destinations. The ones with the newly allocated recycle as the destination are okay, but the ones with self->ob_item are potentially a problem.
3. Yhg1s commented on Jan 20, 2025

   

   Yhg1s

   on Jan 20, 2025

   MemberAuthor

   More actions

   Yes, you're right, 'newly allocated' isn't really relevant. I just meant that the calls that move items around before/after list_resize() are potentially problematic. I have to admit my head hurts every time I try to trace through list_ass_slice_lock_held.
4. picnixz

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Jan 31, 2025
5. tom-pytel

   added a commit that references this issue on Mar 29, 2025

   pythongh-129069: make list ass_slice and memory_repeat safe

   0c8dcfc
6. colesbury

   added 3 commits that reference this issue on Dec 18, 2025

   pythongh-129069: Fix listobject.c data races due to memmove

   97e8d13

   Merge branch 'main' into pythongh-129069-list-memmove

   761cad1

   gh-129069: Fix listobject.c data races due to memmove (gh-142957)

   e46f28c
7. colesbury

   closed this as completedon Dec 19, 2025
8. cocolato

   added a commit that references this issue on Dec 22, 2025

   pythongh-129069: Fix listobject.c data races due to memmove (pythongh…

   a3c229f
9. kumaraditya303

   added a commit that references this issue on Dec 22, 2025

   gh-129069: fix more thread safety issues in `list` (#143019)

   487e91c

Issue actions