GeezerVison is showing github.com — Strategy for Iterators in Free Threading · Issue #124397 · python/cpython
Strategy for Iterators in Free Threading #124397DescriptionThis is an umbrella issue to capture planning and strategy discussions at the sprints.
Our draft plan has three points:
-
Add a new itertool, serialize(), which will take a non-threadsafe iterator as input and produce a new iterator that is thread-safe. Multiple threads can access the new iterator which is guaranteed to make serial (one-at-a-time) calls to the upstream iterators. This will be implemented with locks that block __next__ calls while others are pending. The implementation will not buffer calls; instead, it implements blocking to achieve serialization. If applicable, send() and throw() method calls will be forwarded as well.
-
The itertools.tee() code will have guaranteed semantics. It can take a non-threadsafe iterator from one thread as an input and provide tee objects for other threads to get reliable independent copies of the data stream. The new iterators are only thread-safe if consumed within a single thread. Internally, it buffers data to fulfill this contract.
-
Other iterators implemented in C will get only the minimal changes necessary to cause them to not crash in a free-threaded build. The edits should be made in a way that does not impact existing semantics or performance (i.e. do not damage the standard GIL build). Concurrent access is allowed to return duplicate values, skip values, or raise an exception.
Linked PRs
Reacted by Gregory P. Smith Reacted by Pieter Eendebak Activityeendebakpt commented on Oct 10, 2024 @rhettinger Thanks for the update. Will there be more updates (more sprints)? I have two questions about the strategy above:
i) Is the part about not impacting performance only about the GIL build, or also about single-threaded iteration in the free-threading builds?
ii) What about the following cases:
- In the free-threaded build an iterator is created in thread A, but iteration only takes places in thread B
- In the free-threaded build an iterator is created in thread A and iteration is performed
k times. After completion of the k-th call to next in thread A, iteration continues in thread B
Should we guarantee "correct" iteration for these case? serhiy-storchaka commented on Oct 11, 2024 +1 for serialize(). I've been wanting to implement this for a while, I just wasn't sure about the name and what module it should be in (itertools or threading).
Note that generator objects are not thread-safe. You cannot use the same generator objects concurrently in different threads -- you will get a RuntimeError if other thread already executes the generator code. So such wrapper was needed long before free threading. rhettinger commented on Oct 12, 2024 ContributorAuthor More actions [pieter]
i) Is the part about not impacting performance only about the GIL build, or also about single-threaded iteration in the free-threading builds?
Just the first one. There isn't much we can do for the second one because some anti-race logic needs to replace the current reliance on the GIL.
[pieter]
ii) What about the following cases:
The only requirement is to not crash if an application makes concurrent __next__ calls.
Iterators aren't limited to one thread. They can be created in one, used in another, and later used in another. If needed, a user can (and should) manage contention by adding their own locks or some higher level threading API just like they would with any other shared resource.
[serhiy]
I just wasn't sure about the name and what module it should be in (itertools or threading).
Either module would be a reasonable choice. Are you happy with the name, serialize()? serhiy-storchaka commented on Oct 13, 2024 At that time there was only C API for non-reentrant lock, and I was not sure that this is enough. Reentrant lock would make itertools depending on threading, and this does not look good to me. But I see that there is more private C API for locking, so this may be not needed.
As for the name, serialize does not tell anything about its behavior to me. I think you have reasons for this name. I only though about something ugly like thread_safe_iter.
The implementation is trivial:
class serialize(Iterator):
def __init__(self, it):
self._it = it
self._lock = Lock() # or RLock()?
def __next__(self):
with self._lock:
return next(self._it)a-reich commented on Apr 8, 2025 Just curious - is the plan still to add a serialize helper for general iterators? eendebakpt commented on Apr 9, 2025 It is still in the plan I believe, but needs someone to pick up the task.
@rhettinger
- Since itertools is implemented in C, do we want the
serialize to be implemented in C as well?
- The name
serialize feels a bit off. Do you perhaps have other suggestions or can we open a thread on discourse to do some bike shedding?
- In the suggested implementation by serhiy: should the iterator
_it be public so users can see the underlying iterator?
a-reich commented on Apr 9, 2025 If folks think a Python implementation is sufficient (for now at least), is this task suitable for a first time CPython contributor such as myself? rhettinger commented on Apr 9, 2025 ContributorAuthor More actions @a-reich I do not think this one is suitable. I would definitely start on other issues. serhiy-storchaka commented on Apr 9, 2025 The threading module may be a better place for such utility. Different variant can be added in the asyncio module. eendebakpt commented on Apr 29, 2025 I am looking at the implementation of serialize. Would it make sense to implement this as the C equivalent of the following?
from typing import Callable, Iterator
from threading import Lock
class locked_operation(Callable):
def __init__(self, callable):
self._callable = callable
self._lock = Lock() # or RLock()?
def __call__(self, *args, **kwargs):
with self._lock: # in C would be implemented using Py_BEGIN_CRITICAL_SECTION(self._callable);
return self._callable(*args, **kwargs)
class serialize(Iterator):
def __init__(self, it):
self._locked_next = locked_operation(iter(it).__next__)
def __next__(self):
return self._locked_next()
The locked_operation might be valuable in itself (but does not seem to fit in itertools, so not sure where it would go).
@serhiy-storchaka @colesbury serhiy-storchaka commented on Apr 29, 2025 eendebakpt commented on Apr 29, 2025 No real advantage, except that if we would add the locked_operation for other reasons it is nice to re-use. But since the implementation (also in C) would be quite small, the gain might be small.
So maybe the question is: do we need the locked_operation in addition to the serialize. serhiy-storchaka commented on Apr 30, 2025 I do not think that locked_operation would be so useful, because we usually need to synchronize several operations with the same lock. For example, the above example is a simplification, the wrapper should wrap also close(), send() and throw() (if we want to support generators).
The main problem -- naming. serialize does not look good at all. But we will need also other name for the decorator which makes the generator function synchronized.
def make_gen_func_thread_safe(func):
@functools.wraps(func)
def wrapped(*args, **kwargs):
return make_generator_thread_safe(func(*args, **kwargs))
return wrappedWe could even add a general decorator factory that wraps the generator with the specified function:
def apply_to_result(postprocess):
def deco(func):
@functools.wraps(func)
def wrapped(*args, **kwargs):
return postprocess(func(*args, **kwargs))
return wrapped
return deco
make_gen_func_thread_safe = apply_to_result(make_generator_thread_safe)But the hardest part is to find good names for all this. ZeroIntensity commented on Apr 23, 2026 Some bikeshedding:
- I have already expressed before that I think
threading is a bad place for this. threading should be for concurrency primitives, like locks and events, not necessarily utilities. I don't find the "asyncio might need this too" argument to be very convincing, because asyncio already copies all sorts of things from various stdlib modules (ex time.sleep -> asyncio.sleep, concurrent.futures.wait -> asyncio.wait, asyncio.subprocess).
- If we put it in
threading anyway, threading.serialize is a bad name for something that's solely about iteration. In terms of parallelism, "serialize" is a blanket term for ensuring that only a single thread can interact with something. (In contrast, I think itertools.serialize would be a fine name, because the itertools module implies that it's about iteration.)
devdanzin commented on Jul 21, 2026 Re point (3) — "Other iterators implemented in C will get only the minimal changes necessary to cause them to not crash in a free-threaded build … Concurrent access is allowed to return duplicate values, skip values, or raise an exception."
memoryiter_next (Objects/memoryobject.c) currently crashes under concurrent iteration on a free-threaded build — it doesn't merely duplicate/skip. It takes no critical section anywhere in the function, and on exhaustion runs:
seq = it->it_seq; /* unguarded read */
...
it->it_seq = NULL; /* non-atomic clear */
Py_DECREF(seq); /* drops the iterator's one owning ref to the memoryview */Two threads advancing the same iterator to exhaustion both read the same non-NULL seq and both Py_DECREF(seq) → the memoryview's refcount underflows → double-free / use-after-free. (Lines 3623 / 3641–3642 as of main@a1d580430c8.)
Reproducer (pure Python, stdlib only)
import threading
NT = 8; ITERS = 200000
def newit(): return iter(memoryview(bytearray(range(32))))
cell = [newit()]
def worker():
for _ in range(ITERS):
it = cell[0]
try: next(it)
except StopIteration: cell[0] = newit()
except Exception: pass
ts = [threading.Thread(target=worker) for _ in range(NT)]
for t in ts: t.start()
for t in ts: t.join()
print("done, no crash")Run with PYTHON_GIL=0:
- debug build (
--disable-gil --with-pydebug): aborts within seconds —
Objects/memoryobject.c: _Py_NegativeRefcount: Assertion failed: object has negative ref count / <object … is freed>.
- release + ASan (
--disable-gil): AddressSanitizer: SEGV … in memoryiter_next dereferencing the freed memoryview — a genuine use-after-free, not a debug-only assertion.
- Plain release without a sanitizer doesn't reliably fault (as usual for a UAF where the freed block isn't immediately reused).
Two races in the one function — only the second violates point (3)
- Cursor (
it->it_index < it->it_length read vs it->it_index++ write): value-benign — duplicate/skip an element. Fine under point (3).
- Exhaustion (
it->it_seq = NULL; Py_DECREF(seq)): the double-free above. Not benign — it crosses the "must not crash" bar.
This is the same it_seq = NULL; Py_DECREF(seq)-on-exhaustion shape already confirmed as a crash for the dict iterator (gh-154130) and the set iterator (fix PR gh-144357, for gh-144356); str/bytes/tuple/list/seqiter share it too. memoryiter_next is the most exposed of them — no critical section at all. The set PR's fix pattern (make exhaustion sticky and drop the owning reference in tp_dealloc rather than in tp_iternext) applies here directly.
Question
Would you prefer I open a standalone issue for the memoryiter_next double-free (mirroring gh-154130 for the dict iterator, so it has its own fix PR to hang off), or fold it into the class-wide handling under this strategy? Happy to do whichever fits the plan.
Found with the fusil fuzzer under ThreadSanitizer; reproducers and backtraces drafted with the help of Claude Code and verified by hand. weixlu commented on Jul 28, 2026 @devdanzin Do you mind if I take the issue you mentioned here? It looks similar to a preivous memoryview issue, and I have submit a fix for that :) devdanzin commented on Jul 28, 2026 Feel free to tackle it, but I still don't know whether a standalone issue is best. Probably is. weixlu commented on Jul 28, 2026
but I still don't know whether a standalone issue is best. Probably is.
I think it worth a new issue. Because it aborts on debug build, kind of serious
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|