GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-153981: Fix use after free bug itertools.count() by johng · Pull Request #153983 · python/cpython

Conversation

@johng

@johng johng commented Jul 18, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Obtain a reference to lz->long_cnt, lz->long_step
such that we don't hit a use after free.

  • Issue: itertoolsmodule: free-threading use after free bug in counting slow mode #153981

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@johng
johng requested a review from rhettinger as a code owner July 18, 2026 14:39
@bedevere-app bedevere-app Bot added the awaiting review label Jul 18, 2026
@johng johng changed the title gr-153981 Fix use after free bug itertools.count() gr-153981: Fix use after free bug itertools.count() Jul 18, 2026
@johng
johng force-pushed the use-after-free-iter-tools branch from 8623456 to 765d7d5 Compare July 18, 2026 14:40
@johng
Fix use after free bug itertools.count()
8580b6a
Obtain a reference to lz->long_cnt, lz->long_step
such that we don't hit a use after free
@johng
johng force-pushed the use-after-free-iter-tools branch from 765d7d5 to 8580b6a Compare July 18, 2026 14:42
@johng johng changed the title gr-153981: Fix use after free bug itertools.count() gh-153981: Fix use after free bug itertools.count() Jul 18, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jul 18, 2026
itertoolsmodule: free-threading use after free bug in counting slow mode #153981
Open
@blurb-it
📜🤖 Added by blurb_it.
31f322f
@johng

johng commented Jul 18, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

This appears to add some very slight overhead to repr in slow mode, no impact to the likely more heavily used fast mode

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@johng johng mentioned this pull request Jul 19, 2026
tsan-006: itertoolsmodule.c: count.__repr__ plain-reads cnt #153908
Closed
picnixz
picnixz reviewed Jul 24, 2026
View reviewed changes
Comment thread Modules/itertoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Misc/NEWS.d/next/Library/2026-07-18-14-47-52.gh-issue-153981.yZcmKQ.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Jul 24, 2026

Copy link
Copy Markdown
Member

Performances are not really inportant for __repr__. You usually don't print a count in a hot path.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

johng and others added 2 commits July 25, 2026 21:59
@johng @picnixz
Apply suggestion from @picnixz
1068168
Co-authored-by: Bénédikt Tran <[email protected]>
@johng
fixup! Fix use after free bug itertools.count()
f4b7e9f
@johng
johng requested a review from picnixz July 25, 2026 21:01
@johng

johng commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

@picnixz would you be able to kindly review again, thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@johng

johng commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

@picnixz kindly pining for a re-review, thank you!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz
picnixz reviewed Oct 3, 2026
View reviewed changes

@picnixz picnixz left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may just seem dumb but since __repr__ is fairly used just for debugging, I don't think we really need to worry about performances; so why not just having everything in a critical section? would that work?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@johng

johng commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Would that not risk locking other threads executing more latency sensitive functions that are holding the lock?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@johng
johng requested a review from picnixz October 12, 2026 10:48
@picnixz

picnixz commented Oct 12, 2026

Copy link
Copy Markdown
Member

We don't care here IMO. If you print the counter, just expect it to possbly be slow. And can you simply benchmark this please?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@johng

johng commented Oct 12, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

If I add a lock around the whole of repr I can significantly slow down the actual counting module if another thread is printing. I'm sure this example is a bit contrived however I think a real possibility another thread printing could slow down the counting module.

Build next() thread time for 1M calls (median of 5)
PR as-is (lock only while taking references) 108 / 116 / 112 ms
Whole repr in a critical section 1479 / 2095 / 1783 ms

simple_bench.py (see below)

Details
import statistics
import threading
import time
from itertools import count

N = 1_000_000


def bench():
    it = count(10**18, 2)  # slow mode: next() takes the critical section
    done = threading.Event()
    elapsed = []

    def counter():
        start = time.perf_counter()
        for _ in range(N):
            next(it)
        elapsed.append(time.perf_counter() - start)

    def reprer():
        while not done.is_set():
            repr(it)

    r = threading.Thread(target=reprer)
    c = threading.Thread(target=counter)
    r.start()
    c.start()
    c.join()
    done.set()
    r.join()
    return elapsed[0]


times = [bench() for _ in range(5)]
print(f"next() thread: {statistics.median(times) * 1000:.1f} ms "
      f"for {N:,} calls while another thread calls repr() (median of 5)")

And just to confirm a lock held across repr would look something like this

static PyObject *
count_repr_lock_held(countobject *lz)
{
    if (lz->long_cnt == NULL) {
        Py_ssize_t cnt = FT_ATOMIC_LOAD_SSIZE_RELAXED(lz->cnt);
        return PyUnicode_FromFormat("%s(%zd)",
                                    _PyType_Name(Py_TYPE(lz)), cnt);
    }

    if (PyLong_Check(lz->long_step)) {
        long step = PyLong_AsLong(lz->long_step);
        if (step == -1 && PyErr_Occurred()) {
            PyErr_Clear();
        }
        if (step == 1) {
            /* Don't display step when it is an integer equal to 1 */
            return PyUnicode_FromFormat("%s(%R)",
                                        _PyType_Name(Py_TYPE(lz)),
                                        lz->long_cnt);
        }
    }
    return PyUnicode_FromFormat("%s(%R, %R)",
                                _PyType_Name(Py_TYPE(lz)),
                                lz->long_cnt, lz->long_step);
}

static PyObject *
count_repr(PyObject *op)
{
    PyObject *result;
    Py_BEGIN_CRITICAL_SECTION(op);
    result = count_repr_lock_held(countobject_CAST(op));
    Py_END_CRITICAL_SECTION();
    return result;
}
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Oct 12, 2026

Copy link
Copy Markdown
Member

Two possibilities:

  • I think it's expected that printing continuously may slow down the rest of the access. So there should be NO hot paths being affected (if you decide to repr() the count, you are no longer in a hot path).
  • OTOH, the reason why I suggested the whole critical section is because the code either has a lots of ifs or a UnicodeWriter. Would it be possible to actually have a minimal change with neither (maybe just storing the message and then returning it, like msg = PyUnicode... as we have, and then goto finally where finally would free the resources?)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

johng added 2 commits October 12, 2026 21:08
@johng
Remove the if/else's
60b7012
@johng
Simplify more
129177c
@johng

johng commented Oct 12, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

I'm not sure if this was what you were after but I think this reduces the branches considerably if the if/else branches were the main sticking point. Agree looking back it was rather messy for all the approaches!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz
picnixz reviewed Oct 12, 2026
View reviewed changes
Comment thread Modules/itertoolsmodule.c Outdated
lz->long_cnt, lz->long_step);

/* Don't display step when it is an integer equal to 1 */
msg = PyUnicode_FromFormat(step == 1 ? "%s(%R)" : "%s(%R, %R)",

@picnixz picnixz Oct 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can put PyObject *msg = here instead.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

johng reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
picnixz
picnixz reviewed Oct 12, 2026
View reviewed changes
Comment thread Modules/itertoolsmodule.c Outdated
Comment on lines +3698 to +3701
/* Don't display step when it is an integer equal to 1 */
msg = PyUnicode_FromFormat(step == 1 ? "%s(%R)" : "%s(%R, %R)",
_PyType_Name(Py_TYPE(lz)),
long_cnt, lz->long_step);

@picnixz picnixz Oct 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have the if check out of the format call? I don't exactly remember but I think passing more va args than what the format expects is UB>

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

johng reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@johng
review comments
7803734
@johng
johng requested a review from picnixz October 14, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rhettinger rhettinger Awaiting requested review from rhettinger rhettinger is a code owner
@picnixz picnixz Awaiting requested review from picnixz

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@johng @picnixz
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.