GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-161242: Add a freelist for traceback objects by eendebakpt · Pull Request #161243 · python/cpython Conversation
|
| benchmark | main | this PR |
|---|---|---|
raise ValueError / except |
454 ns | 418 ns (1.09x faster) |
raise ValueError(x) / except |
495 ns | 476 ns (1.04x faster) |
| raise through one frame | 792 ns | 764 ns (1.04x faster) |
d[key] KeyError |
630 ns | 604 ns (1.04x faster) |
obj.x AttributeError |
1.53 us | 1.45 us (1.05x faster) |
Exception-heavy code in the style of real programs:
| benchmark | main | this PR |
|---|---|---|
| parse mixed column (int/float/str) | 953 us | 938 us (1.02x faster) |
| word count via KeyError | 411 us | 403 us (1.02x faster) |
| template variable resolve | 368 us | 356 us (1.03x faster) |
| duck typing AttributeError | 464 us | 439 us (1.06x faster) |
pyperformance chameleon |
7.04 ms | 6.86 ms (1.03x faster) |
A freelist for the exception instances themselves is possible as well, but needs more care (allocator slots on the static exception types) and is left for a separate PR.
"""pyperf micro-benchmarks for raising and catching exceptions. Usage: python bench_exc.py [pyperf args...] """ import sys import pyperf def keyerror(d, key, fallback): try: return d[key] except KeyError: return fallback(key, d) def attrerror(obj, name, default): try: return obj.x except AttributeError: return default + name def valueerror_raise(x): try: raise ValueError except ValueError: return x def valueerror_raise_inst(x): try: raise ValueError(x) except ValueError: return x def stopiter(it): try: return next(it) except StopIteration: return it def through_frame(x): try: raiser(x) except ValueError: return x def raiser(x): raise ValueError(x) class NoX: pass def make_bench(fn, *args): def bench(loops): r = range(loops) t0 = pyperf.perf_counter() for _ in r: fn(*args); fn(*args); fn(*args); fn(*args) return pyperf.perf_counter() - t0 return bench if __name__ == "__main__": a = float(len(sys.argv)) * 2.5 runner = pyperf.Runner() runner.bench_time_func("raise ValueError", make_bench(valueerror_raise, a)) runner.bench_time_func("raise ValueError(x)", make_bench(valueerror_raise_inst, a)) runner.bench_time_func("raise through frame", make_bench(through_frame, a)) runner.bench_time_func("d[key] KeyError", make_bench(keyerror, {}, a, lambda k, d: k)) runner.bench_time_func("next() StopIteration", make_bench(stopiter, iter(()))) runner.bench_time_func("obj.x AttributeError", make_bench(attrerror, NoX(), "n" * 2, "d" * 2))
"""Exception-heavy workloads in the style of real code, for pyperf. Each benchmark raises and catches exceptions through Python frames as a side effect of ordinary work, which is the case the traceback and exception freelists target. Usage: python bench_exc_macro.py [pyperf args] """ import random import pyperf random.seed(12345) # 1. Data cleaning: parse a column of mixed strings with try/except fallbacks # (int -> float -> keep string), the usual idiom for CSV cleanup. _RAW_COLUMN = [] for _i in range(2000): _r = random.random() if _r < 0.5: _RAW_COLUMN.append(str(random.randint(-10**6, 10**6))) elif _r < 0.8: _RAW_COLUMN.append(repr(random.uniform(-1e6, 1e6))) else: _RAW_COLUMN.append(random.choice(["n/a", "", "NULL", "12,5", "1e", "x"])) def to_number(s): try: return int(s) except ValueError: pass try: return float(s) except ValueError: return s def bench_parse_column(loops): column = _RAW_COLUMN t0 = pyperf.perf_counter() for _ in range(loops): total = 0 for s in column: v = to_number(s) if type(v) is not str: total += 1 return pyperf.perf_counter() - t0 # 2. Counting with the classic try/except KeyError idiom on a text with a # long tail of rare words (every first occurrence raises). _WORDS = ([f"w{i}" for i in range(50) for _ in range(20)] + [f"rare{i}" for i in range(3000)]) random.shuffle(_WORDS) def bench_count_keyerror(loops): words = _WORDS t0 = pyperf.perf_counter() for _ in range(loops): counts = {} for w in words: try: counts[w] += 1 except KeyError: counts[w] = 1 return pyperf.perf_counter() - t0 # 3. Template-engine style variable resolution: try a dict lookup, then an # attribute, then an index, like Django's Variable._resolve_lookup(). class _Ctx: def __init__(self, **kw): self.__dict__.update(kw) def resolve(obj, bit): try: return obj[bit] except (TypeError, KeyError, IndexError): pass try: return getattr(obj, bit) except AttributeError: pass try: return obj[int(bit)] except (ValueError, TypeError, IndexError, KeyError): raise LookupError(bit) _CONTEXT = {"user": _Ctx(name="Ada", tags=["a", "b", "c"], profile={"city": "London"}), "items": [_Ctx(title=f"t{i}", price=i) for i in range(20)]} _LOOKUPS = ["user.name", "user.tags.1", "user.profile.city", "items.3.title", "items.7.price", "user.missing", "items.25.title"] def render_var(context, path): obj = context for bit in path.split("."): try: obj = resolve(obj, bit) except LookupError: return "" return obj def bench_template_resolve(loops): ctx = _CONTEXT lookups = _LOOKUPS t0 = pyperf.perf_counter() for _ in range(loops): for _r in range(50): for p in lookups: render_var(ctx, p) return pyperf.perf_counter() - t0 # 4. Duck typing: optional protocol methods probed with try/except # AttributeError in Python code, over a mix of object kinds. class _WithClose: def close(self): return 1 class _WithFlush: def flush(self): return 2 class _Plain: pass _OBJECTS = [random.choice([_WithClose, _WithFlush, _Plain])() for _ in range(1000)] def finish(obj): n = 0 try: obj.flush() n += 1 except AttributeError: pass try: obj.close() n += 1 except AttributeError: pass return n def bench_duck_typing(loops): objs = _OBJECTS t0 = pyperf.perf_counter() for _ in range(loops): for o in objs: finish(o) return pyperf.perf_counter() - t0 # 5. Python-level iterator classes: StopIteration raised from __next__ # through a Python frame once per (short) iteration. class Countdown: def __init__(self, n): self.n = n def __iter__(self): return self def __next__(self): if self.n <= 0: raise StopIteration self.n -= 1 return self.n def bench_iterator_class(loops): t0 = pyperf.perf_counter() for _ in range(loops): for _r in range(500): for _x in Countdown(3): pass return pyperf.perf_counter() - t0 if __name__ == "__main__": runner = pyperf.Runner() runner.bench_time_func("parse mixed column (int/float/str)", bench_parse_column) runner.bench_time_func("word count via KeyError", bench_count_keyerror) runner.bench_time_func("template variable resolve", bench_template_resolve) runner.bench_time_func("duck typing AttributeError", bench_duck_typing) runner.bench_time_func("iterator class StopIteration", bench_iterator_class)
Many thanks to Claude Code
Sorry, something went wrong.
There was an error while loading. Please reload this page.
A traceback object is created for every raised exception. Keep up to 8 of them on a per-thread freelist, and replace the argument check in tb_create_raw() with asserts since all callers validate their arguments. Co-Authored-By: Claude Fable 5.1 <[email protected]>
| PyTracebackObject *tb; | ||
| if ((next != NULL && !PyTraceBack_Check(next)) || | ||
| frame == NULL || !PyFrame_Check(frame)) { | ||
| PyErr_BadInternalCall(); |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Is it safe to remove this call? tb_create_raw is used in several C-API calls that can in theory reach this point.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
All callers have their own guards: tb_new_impl is checked via AC, _PyTraceBack_FromFrame checks with asserts as well (callers of _PyTraceBack_FromFrame do perform actual checks, _PyTraceBack_FromFrame is internal api). Maybe I missed a path?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.