Description

Yhg1s

opened on Jan 20, 2025

Issue body actions

Bug report

Bug description:

Concurrent iteration over range iterators (not range objects themselves) is not thread-safe, because range iterators are currently implemented as two pieces of state (the next value and the number of values left) that aren't updated atomically. ThreadSanitizer correctly identifies these races as well. See tests in PR #128637, and an early attempt at fixing the issues in 1533d1d.

This also affects bytecode specialisation (FOR_ITER_RANGE), although in PR #128798 we're only specialising if the iterator is uniquely referenced (and so we don't care about other threads).

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-129068: allow concurrent iteration over range iterators #131921
• gh-129068: Make range iterators thread-safe #142886

Activity

1. Yhg1s

   added

   topic-free-threading

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jan 20, 2025
2. picnixz

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Jan 31, 2025
3. colesbury commented on Mar 14, 2025

   

   colesbury

   on Mar 14, 2025

   Contributor

   More actions

   I wonder if we should apply the same technique to fast_range_iter that we do in the bytecode, where we don't need to care about other threads if _PyObject_IsUniquelyReferenced(iter) is true. We could fall back to a locking code path if that's not the case. (Apologies, if this is something you've already suggested previously).

   I'd want to wait for Matt's LOAD_FAST optimizations to land first, because otherwise I think we'll slow down cases like:

   range_it = range(N):
   for v in range_it:
     ...

   That pattern exists a lot in the pyperformance benchmarks and probably in some real world code too.
4. colesbury

   marked iter(range) shared by threads exceeds max range value #131199 as a duplicate of this issue on Mar 14, 2025
5. tom-pytel

   added a commit that references this issue on Mar 30, 2025

   pythongh-129068: allow concurrent iteration over range iterators

   579bd08
6. colesbury

   added 5 commits that reference this issue on Dec 16, 2025

   pythongh-129068: Make range iterators thread-safe

   45f1ff8

   pythongh-129068: Make range iterators thread-safe

   afd3eff

   Merge branch 'main' into pythongh-129068-rangeiter-next

   30bbb31

   Merge branch 'main' into pythongh-129068-rangeiter-next

   b43cf0d

   gh-129068: Make range iterators thread-safe (gh-142886)

   f54d44d
7. colesbury

   closed this as completedon Dec 18, 2025
8. colesbury commented on Dec 18, 2025

   

   colesbury

   on Dec 18, 2025

   Contributor

   More actions

   Fixed in main (3.15). I'm not planning to backport it to 3.14 unless someone asks for it.
9. cocolato

   added a commit that references this issue on Dec 22, 2025

   pythongh-129068: Make range iterators thread-safe (pythongh-142886)

   8192cda

Issue actions