GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-153568: Cache repeated identifiers while parsing by pablogsal · Pull Request #153577 · python/cpython

Conversation

@pablogsal

@pablogsal pablogsal commented Jul 11, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Every occurrence of a name paid a UTF-8 decode, a hash and an intern lookup, even though identifiers repeat constantly in real code. A small per-parse table now reuses the interned name created the first time a given spelling is seen, and everything it stores lives in the parse arena so no lifetime management is needed.

Benchmark (parsing 8 of the largest stdlib files, 1.3 MB, 20 times per run with _PyParser_ASTFromString — parser only, no AST-to-Python conversion; pyperf, interleaved runs):

build time per run speedup
main 1.70 s ± 0.02
this PR 1.45 s ± 0.01 1.17x faster
  • Issue: Speed up the parser #153568

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@pablogsal
pythongh-153568: Cache repeated identifiers while parsing
da479ac 
Identifiers repeat constantly in real code, so the parser now reuses
the interned name created the first time it sees a given spelling.
@pablogsal
pablogsal requested a review from lysnikolaou as a code owner July 11, 2026 14:08
@bedevere-app bedevere-app Bot added the awaiting core review label Jul 11, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jul 11, 2026
Speed up the parser #153568
Open
@StanFromIreland

StanFromIreland commented Jul 12, 2026

Copy link
Copy Markdown
Member

Oh nice, a 17% speed up is quite good!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal
pablogsal requested a review from Fidget-Spinner July 18, 2026 11:31
Fidget-Spinner
Fidget-Spinner approved these changes Jul 18, 2026
View reviewed changes
Comment thread Parser/pegen.c Outdated
// so borrowed references are valid for the lifetime of the parse
// (including the second error pass, which reuses parser and arena).
Py_ssize_t len = PyBytes_GET_SIZE(t->bytes);
uint32_t hash = 2166136261u;

@Fidget-Spinner Fidget-Spinner Jul 18, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this to global+const.

Image

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
savannahostrowski, pablogsal, StanFromIreland, and maurycy reacted with heart emoji
All reactions
  • ❤️ 4 reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jul 18, 2026
pablogsal added 2 commits July 18, 2026 13:28
@pablogsal
fixup! pythongh-153568: Cache repeated identifiers while parsing
837625b
@pablogsal
fixup! pythongh-153568: Cache repeated identifiers while parsing
c2e00bb
@pablogsal
pablogsal enabled auto-merge (squash) July 18, 2026 12:55
@pablogsal
pablogsal merged commit a0c6c4c into python:main Jul 18, 2026
54 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal
pablogsal deleted the gh-153568-ident-cache branch July 18, 2026 13:25
@bedevere-app bedevere-app Bot removed the awaiting merge label Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Fidget-Spinner Fidget-Spinner Fidget-Spinner approved these changes
@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@pablogsal @StanFromIreland @Fidget-Spinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.