Description

KowalskiThomas

opened on Jun 2, 2026

Issue body actions

Bug report

Bug description:

Two threads calling next on the same groupby object concurrently in a free-threaded build race on the currgrouper field, corrupting the iterator's internal state and producing AttributeError on slot accesses of live objects.

groupby_next has no Py_BEGIN_CRITICAL_SECTION guard. The first thing it does is write gbo->currgrouper = NULL:

cpython/Modules/itertoolsmodule.c

Lines 531 to 540 in c5516e7

	static PyObject *
	groupby_next(PyObject *op)
	{
	PyObject *grouper;
	groupbyobject *gbo = groupbyobject_CAST(op);
	gbo->currgrouper = NULL;
	/* skip to next iteration group */
	for (;;) {
	if (gbo->currkey == NULL)

Later, after the loop exits, it calls _grouper_create, which writes parent->currgrouper = igo:

cpython/Modules/itertoolsmodule.c

Lines 624 to 637 in c5516e7

	static PyObject *
	_grouper_create(groupbyobject *parent, PyObject *tgtkey)
	{
	itertools_state *state = parent->state;
	_grouperobject *igo = PyObject_GC_New(_grouperobject, state->_grouper_type);
	if (igo == NULL)
	return NULL;
	igo->parent = Py_NewRef(parent);
	igo->tgtkey = Py_NewRef(tgtkey);
	parent->currgrouper = igo; /* borrowed reference */
	PyObject_GC_Track(igo);
	return (PyObject *)igo;
	}

If Thread A is past line 633 (just stored the new grouper pointer) while Thread B is at line 537 (about to store NULL), Thread B overwrites the pointer Thread A just wrote. Both are plain pointer stores with no synchronisation.

---

The following script reproduces the condition under which this happens.

import itertools, threading

class K:
    __slots__ = ("v",)
    def __init__(self, v): self.v = v
    def __eq__(self, o): return isinstance(o, K) and self.v == o.v
    def __hash__(self): return hash(self.v)

def consume(g):
    try:
        while True:
            _, _ = next(g)
    except StopIteration:
        pass

keys = [K(i) for i in range(500_000)]
g = itertools.groupby(keys)
threads = [threading.Thread(target=consume, args=(g,)) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()

On a free-threaded build, this results in the following Python logs...

Exception in Thread-15 (consume):
  File "<python-input-2>", line 6, in __eq__
    def __eq__(self, o): return isinstance(o, K) and self.v == o.v
                                                     ^^^^^^
AttributeError: 'K' object has no attribute 'v'

... and the following TSan logs.

WARNING: ThreadSanitizer: data race (pid=20464)
  Write of size 8 at 0x0003026e4a88 by thread T2:
    #0 groupby_next itertoolsmodule.c:537 (python.exe:arm64+0x10042a8b8)
    #1 builtin_next bltinmodule.c:1770

  Previous write of size 8 at 0x0003026e4a88 by thread T1:
    #0 _grouper_create itertoolsmodule.c:633 (python.exe:arm64+0x10042ac5c)
    #1 groupby_next itertoolsmodule.c:570 (python.exe:arm64+0x10042ac5c)
    #2 builtin_next bltinmodule.c:1770

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-150791: add critical section for groupby.next #150792

Activity

1. KowalskiThomas

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jun 2, 2026
2. eendebakpt

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   topic-free-threading

   on Jun 2, 2026
3. devdanzin commented on Jul 19, 2026

   

   devdanzin

   on Jul 19, 2026

   Member

   More actions

   Independent confirmation via fusil --tsan, with a reproducer that doesn't need a custom key type — just several threads consuming one shared groupby:

   import itertools, threading
   
   NTHREADS = 8
   barrier = threading.Barrier(NTHREADS)
   
   def worker(gb):
       barrier.wait()
       try:
           list(gb)                              # groupby_next on the shared gb
       except (RuntimeError, StopIteration, ValueError, TypeError):
           pass
   
   for _ in range(500):
       shared = itertools.groupby(range(8192))   # ONE shared groupby
       ts = [threading.Thread(target=worker, args=(shared,)) for _ in range(NTHREADS)]
       for t in ts: t.start()
       for t in ts: t.join()

   Exit 66 under TSAN; the script above races _grouper_create against groupby_next (creating the child grouper while another thread advances the parent), alongside the groupby_next-vs-groupby_next race on currgrouper described in the issue. Also iterating the yielded sub-groups races _grouper_next against the parent's currkey/currvalue — consistent with PR #150792 guarding _grouper_next (via Py_BEGIN_CRITICAL_SECTION(parent)) as well as groupby_next. Confirmed on current main (3.16.0a0), debug + release TSan builds.

   (Found by fusil --tsan, a ThreadSanitizer fuzzer; draft and reproducer by Claude Code, minimized and reviewed by hand.)

Issue actions