Description

pablogsal

opened on Jul 11, 2026

Issue body actions

This is a catch-all issue to track a series of small, independent speedups to the parser

Linked PRs

• gh-153568: Skip parser memo lookups that cannot match #153571
• gh-153568: Make the parser's per-rule stack check an inline comparison #153573
• gh-153568: Stop re-walking memo lists when growing left-recursive rules #153574
• gh-153568: Skip the expression precedence chain for single-token atoms #153575
• gh-153568: Don't materialize parser token text that is never read #153576
• gh-153568: Cache repeated identifiers while parsing #153577
• gh-153568: Pause the GC while converting the AST to Python objects #153583
• gh-153568: Skip newline translation for source without carriage returns #153584
• gh-153568: Use FIRST sets to speed up parser choices #156209
• gh-153568: Reduce parser allocations #157374
• gh-153568: Parse arithmetic with operator loops #157413
• gh-153568: Factor shared prefixes in expression rules #157437
• gh-153568: Share expression parsing with assignment targets #157450
• gh-153568: Reuse shared prefixes in generated parser rules #157465
• gh-153568: Skip prefix tests when peg_generator is unavailable #157534
• gh-153568: Skip PEG alternatives that cannot match the next token #160458
• gh-153568: Use a managed dict for AST nodes #160616

Activity

1. pablogsal

   added

   type-featureA feature request or enhancementA feature request or enhancement

   performancePerformance or resource usagePerformance or resource usage

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Jul 11, 2026
2. pablogsal

   added a commit that references this issue on Jul 11, 2026

   pythongh-153568: Skip parser memo lookups that cannot match

   a638dad
3. StanFromIreland commented on Jul 11, 2026

   

   StanFromIreland

   on Jul 11, 2026

   Member

   More actions

   @pablogsal I have a little suggestion in #148931.

   Reacted by Pablo Galindo Salgado
4. pablogsal commented on Jul 11, 2026

   

   pablogsal

   on Jul 11, 2026

   MemberAuthor

   More actions

   @pablogsal I have a little suggestion in #148931.

   Will take a look soon. Thanks a lot!
5. pablogsal

   added 4 commits that reference this issue on Jul 11, 2026

   pythongh-153568: Skip the expression precedence chain for single-toke…

   f5d126f

   pythongh-153568: Skip the expression precedence chain for single-toke…

   758e344

   pythongh-153568: Skip the expression precedence chain for single-toke…

   f3cc7e4

   pythongh-153568: Skip the expression precedence chain for single-toke…

   24e869c
6. pablogsal

   added a sub-issue on Jul 11, 2026

   • Speed up parser by caching normalized identifiers? #148931
7. pablogsal

   added 4 commits that reference this issue on Jul 11, 2026

   pythongh-153568: Skip the expression precedence chain for single-toke…

   b3a8d8f

   pythongh-153568: Add fast paths for single-token atoms in the parser

   605c6d7

   pythongh-153568: Add fast paths for single-token atoms in the parser

   26282a7

   pythongh-153568: Skip the expression precedence chain for single-toke…

   61277c2
8. 70 remaining items
9. pablogsal

   added 6 commits that reference this issue on Sep 15, 2026

   gh-153568: Reuse shared prefixes in generated parser rules (#157465)

   89fe985

   pythongh-153568: Use FIRST sets to speed up parser choices

   456eb6f

   pythongh-153568: Fix FIRST-set dispatch typing

   5646f86

   pythongh-153568: Make FIRST-set dispatch conservative

   306ec7e

   pythongh-153568: Make FIRST-set dispatch conservative

   6409262

   gh-153568: Skip prefix tests when peg_generator is unavailable (#157534)

   e03ed46
10. eendebakpt commented on Sep 15, 2026

    

    eendebakpt

    on Sep 15, 2026

    Contributor

    More actions

    Improvements of the parser might help with dataclass creation. See https://discuss.python.org/t/faster-lazier-dataclasses/107744 for a discussion. Profiling showed a significant amount of time spend in the parser, so this might be a nice testcase.

    For

    def create():
        @dataclasses.dataclass
        class C:
            a: int
            b: str
            c: float
            d: list = dataclasses.field(default_factory=list)
            e: int = 0
        return C
    

    performance increased by about 20% (tested on a53b5b1 vs 2fcb0e2 )
11. pablogsal

    added a commit that references this issue on Sep 16, 2026

    gh-153568: Reduce parser allocations (#157374)

    d8a1072
12. pablogsal

    added 5 commits that reference this issue on Oct 23, 2026

    pythongh-153568: Skip the expression precedence chain for single-toke…

    5eb058a

    pythongh-153568: Parse rule flags in the Rule constructor

    0136cf5

    pythongh-153568: Skip the expression precedence chain for single-toke…

    86b00a5

    pythongh-153568: Parse rule flags in the Rule constructor

    a431447

    gh-153568: Skip the expression precedence chain for single-token atoms (

    b77cb89
13. lysnikolaou

    added a commit that references this issue on Oct 25, 2026

    gh-153568: Use a managed dict for AST nodes (#160616)

    aa0e410
14. cmaloney

    added a related issue on Nov 1, 2026

    • Speed up UTF-8 validation in tokenizer and ASCII scanning in UTF-8 decoding #161145

Issue actions