GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-161242: Add a freelist for traceback objects by eendebakpt · Pull Request #161243 · python/cpython

Conversation

@eendebakpt

@eendebakpt eendebakpt commented Nov 3, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

A traceback object is created for every exception that is raised. This PR adds a freelist for traceback objects and removes some redundant checks.

benchmark main this PR
raise ValueError / except 454 ns 418 ns (1.09x faster)
raise ValueError(x) / except 495 ns 476 ns (1.04x faster)
raise through one frame 792 ns 764 ns (1.04x faster)
d[key] KeyError 630 ns 604 ns (1.04x faster)
obj.x AttributeError 1.53 us 1.45 us (1.05x faster)

Exception-heavy code in the style of real programs:

benchmark main this PR
parse mixed column (int/float/str) 953 us 938 us (1.02x faster)
word count via KeyError 411 us 403 us (1.02x faster)
template variable resolve 368 us 356 us (1.03x faster)
duck typing AttributeError 464 us 439 us (1.06x faster)
pyperformance chameleon 7.04 ms 6.86 ms (1.03x faster)

A freelist for the exception instances themselves is possible as well, but needs more care (allocator slots on the static exception types) and is left for a separate PR.

Micro-benchmark script (bench_exc_pyperf.py)
"""pyperf micro-benchmarks for raising and catching exceptions.

Usage: python bench_exc.py [pyperf args...]
"""
import sys
import pyperf


def keyerror(d, key, fallback):
    try:
        return d[key]
    except KeyError:
        return fallback(key, d)


def attrerror(obj, name, default):
    try:
        return obj.x
    except AttributeError:
        return default + name


def valueerror_raise(x):
    try:
        raise ValueError
    except ValueError:
        return x


def valueerror_raise_inst(x):
    try:
        raise ValueError(x)
    except ValueError:
        return x


def stopiter(it):
    try:
        return next(it)
    except StopIteration:
        return it


def through_frame(x):
    try:
        raiser(x)
    except ValueError:
        return x


def raiser(x):
    raise ValueError(x)


class NoX:
    pass


def make_bench(fn, *args):
    def bench(loops):
        r = range(loops)
        t0 = pyperf.perf_counter()
        for _ in r:
            fn(*args); fn(*args); fn(*args); fn(*args)
        return pyperf.perf_counter() - t0
    return bench


if __name__ == "__main__":
    a = float(len(sys.argv)) * 2.5
    runner = pyperf.Runner()
    runner.bench_time_func("raise ValueError", make_bench(valueerror_raise, a))
    runner.bench_time_func("raise ValueError(x)", make_bench(valueerror_raise_inst, a))
    runner.bench_time_func("raise through frame", make_bench(through_frame, a))
    runner.bench_time_func("d[key] KeyError", make_bench(keyerror, {}, a, lambda k, d: k))
    runner.bench_time_func("next() StopIteration", make_bench(stopiter, iter(())))
    runner.bench_time_func("obj.x AttributeError", make_bench(attrerror, NoX(), "n" * 2, "d" * 2))
Macro-benchmark script (bench_exc_macro.py)
"""Exception-heavy workloads in the style of real code, for pyperf.

Each benchmark raises and catches exceptions through Python frames as a
side effect of ordinary work, which is the case the traceback and
exception freelists target.  Usage: python bench_exc_macro.py [pyperf args]
"""
import random
import pyperf

random.seed(12345)


# 1. Data cleaning: parse a column of mixed strings with try/except fallbacks
#    (int -> float -> keep string), the usual idiom for CSV cleanup.
_RAW_COLUMN = []
for _i in range(2000):
    _r = random.random()
    if _r < 0.5:
        _RAW_COLUMN.append(str(random.randint(-10**6, 10**6)))
    elif _r < 0.8:
        _RAW_COLUMN.append(repr(random.uniform(-1e6, 1e6)))
    else:
        _RAW_COLUMN.append(random.choice(["n/a", "", "NULL", "12,5", "1e", "x"]))


def to_number(s):
    try:
        return int(s)
    except ValueError:
        pass
    try:
        return float(s)
    except ValueError:
        return s


def bench_parse_column(loops):
    column = _RAW_COLUMN
    t0 = pyperf.perf_counter()
    for _ in range(loops):
        total = 0
        for s in column:
            v = to_number(s)
            if type(v) is not str:
                total += 1
    return pyperf.perf_counter() - t0


# 2. Counting with the classic try/except KeyError idiom on a text with a
#    long tail of rare words (every first occurrence raises).
_WORDS = ([f"w{i}" for i in range(50) for _ in range(20)]
          + [f"rare{i}" for i in range(3000)])
random.shuffle(_WORDS)


def bench_count_keyerror(loops):
    words = _WORDS
    t0 = pyperf.perf_counter()
    for _ in range(loops):
        counts = {}
        for w in words:
            try:
                counts[w] += 1
            except KeyError:
                counts[w] = 1
    return pyperf.perf_counter() - t0


# 3. Template-engine style variable resolution: try a dict lookup, then an
#    attribute, then an index, like Django's Variable._resolve_lookup().
class _Ctx:
    def __init__(self, **kw):
        self.__dict__.update(kw)


def resolve(obj, bit):
    try:
        return obj[bit]
    except (TypeError, KeyError, IndexError):
        pass
    try:
        return getattr(obj, bit)
    except AttributeError:
        pass
    try:
        return obj[int(bit)]
    except (ValueError, TypeError, IndexError, KeyError):
        raise LookupError(bit)


_CONTEXT = {"user": _Ctx(name="Ada", tags=["a", "b", "c"], profile={"city": "London"}),
            "items": [_Ctx(title=f"t{i}", price=i) for i in range(20)]}
_LOOKUPS = ["user.name", "user.tags.1", "user.profile.city", "items.3.title",
            "items.7.price", "user.missing", "items.25.title"]


def render_var(context, path):
    obj = context
    for bit in path.split("."):
        try:
            obj = resolve(obj, bit)
        except LookupError:
            return ""
    return obj


def bench_template_resolve(loops):
    ctx = _CONTEXT
    lookups = _LOOKUPS
    t0 = pyperf.perf_counter()
    for _ in range(loops):
        for _r in range(50):
            for p in lookups:
                render_var(ctx, p)
    return pyperf.perf_counter() - t0


# 4. Duck typing: optional protocol methods probed with try/except
#    AttributeError in Python code, over a mix of object kinds.
class _WithClose:
    def close(self):
        return 1


class _WithFlush:
    def flush(self):
        return 2


class _Plain:
    pass


_OBJECTS = [random.choice([_WithClose, _WithFlush, _Plain])() for _ in range(1000)]


def finish(obj):
    n = 0
    try:
        obj.flush()
        n += 1
    except AttributeError:
        pass
    try:
        obj.close()
        n += 1
    except AttributeError:
        pass
    return n


def bench_duck_typing(loops):
    objs = _OBJECTS
    t0 = pyperf.perf_counter()
    for _ in range(loops):
        for o in objs:
            finish(o)
    return pyperf.perf_counter() - t0


# 5. Python-level iterator classes: StopIteration raised from __next__
#    through a Python frame once per (short) iteration.
class Countdown:
    def __init__(self, n):
        self.n = n

    def __iter__(self):
        return self

    def __next__(self):
        if self.n <= 0:
            raise StopIteration
        self.n -= 1
        return self.n


def bench_iterator_class(loops):
    t0 = pyperf.perf_counter()
    for _ in range(loops):
        for _r in range(500):
            for _x in Countdown(3):
                pass
    return pyperf.perf_counter() - t0


if __name__ == "__main__":
    runner = pyperf.Runner()
    runner.bench_time_func("parse mixed column (int/float/str)", bench_parse_column)
    runner.bench_time_func("word count via KeyError", bench_count_keyerror)
    runner.bench_time_func("template variable resolve", bench_template_resolve)
    runner.bench_time_func("duck typing AttributeError", bench_duck_typing)
    runner.bench_time_func("iterator class StopIteration", bench_iterator_class)

Many thanks to Claude Code

  • Issue: Improve performance of exceptions #161242

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@eendebakpt @claude
Add a freelist for traceback objects
0251311
A traceback object is created for every raised exception. Keep up to 8
of them on a per-thread freelist, and replace the argument check in
tb_create_raw() with asserts since all callers validate their arguments.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
@eendebakpt
eendebakpt requested review from ZeroIntensity and ericsnowcurrently as code owners November 3, 2026 09:04
@bedevere-app bedevere-app Bot added awaiting core review type-feature A feature request or enhancement labels Nov 3, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Nov 3, 2026
Improve performance of exceptions #161242
Open
@eendebakpt eendebakpt added the skip news label Nov 3, 2026
sobolevn
sobolevn reviewed Nov 3, 2026
View reviewed changes
Comment thread Python/traceback.c
PyTracebackObject *tb;
if ((next != NULL && !PyTraceBack_Check(next)) ||
frame == NULL || !PyFrame_Check(frame)) {
PyErr_BadInternalCall();

@sobolevn sobolevn Nov 3, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it safe to remove this call? tb_create_raw is used in several C-API calls that can in theory reach this point.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@eendebakpt eendebakpt Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All callers have their own guards: tb_new_impl is checked via AC, _PyTraceBack_FromFrame checks with asserts as well (callers of _PyTraceBack_FromFrame do perform actual checks, _PyTraceBack_FromFrame is internal api). Maybe I missed a path?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sobolevn reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
picnixz
picnixz reviewed Nov 3, 2026
View reviewed changes
Comment thread Python/traceback.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

eendebakpt
eendebakpt commented Nov 3, 2026
View reviewed changes
Comment thread Python/traceback.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@eendebakpt
Apply suggestion from @eendebakpt
6765b38
cmaloney
cmaloney approved these changes Nov 5, 2026
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Nov 5, 2026
sobolevn
sobolevn reviewed Nov 6, 2026
View reviewed changes
Comment thread Python/traceback.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

eendebakpt
eendebakpt commented Nov 6, 2026
View reviewed changes
Comment thread Python/traceback.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

eendebakpt added 2 commits November 6, 2026 09:25
@eendebakpt
Apply suggestion from @eendebakpt
ed4db4c
@eendebakpt
Merge branch 'main' into traceback_freelist_only
81dd418
sobolevn
sobolevn approved these changes Nov 6, 2026
View reviewed changes
@eendebakpt
eendebakpt merged commit 3feccd1 into python:main Nov 6, 2026
53 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@picnixz picnixz picnixz left review comments
@cmaloney cmaloney cmaloney approved these changes
@sobolevn sobolevn sobolevn approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ZeroIntensity ZeroIntensity Awaiting requested review from ZeroIntensity ZeroIntensity is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news type-feature A feature request or enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@eendebakpt @cmaloney @sobolevn @picnixz
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.