Description

encukou

opened on Jun 18, 2025

Issue body actions

Documentation

Continuing with #127833, I'd like to reword and restructure the Lexical analysis section of the Reference documentation.

I think I found a style that works: each section will start with the most obvious things, expand toward a complete & precise description, with examples along the way, and end with the formal grammar. (Currently the grammar is usually listed first, and the author of the prose is tempted to assume that the reader has parsed and understood it.)

PRs merged for this before opening this issue:

• gh-127833: Add links to token types to the lexical analysis intro #131468
• gh-127833: lexical analysis: Improve section on Names #131474
• gh-127833: lexical analysis: Add backticks to BOM example #132407

Linked PRs

• gh-135676: lexical analysis: Improve section on Numeric literals #134850
• [3.14] gh-135676: lexical analysis: Improve section on Numeric literals (GH-134850) #135677
• gh-135676: Lexical analysis: Reword String literals and related sections #135942
• [3.14] gh-135676: Lexical analysis: Reword String literals and related sections (GH-135942) #137048
• gh-135676: Reword the f-string (and t-string) section #137469
• gh-135676: Reword the Operators & Delimiters section(s) #137713
• gh-135676: Add a summary of source characters #138194
• [3.14] gh-135676: Reword the Operators & Delimiters section(s) (GH-137713) #138457
• [3.14] gh-135676: Add a summary of source characters (GH-138194) #139781
• gh-135676: Simplify docs on lexing names #140464
• [3.14] gh-135676: Simplify docs on lexing names (GH-140464) #142015
• [3.14] gh-135676: Reword the f-string (and t-string) section (GH-137469) #142227

Activity

1. encukou

   self-assigned this

   on Jun 18, 2025
2. encukou

   added

   docsDocumentation in the Doc dirDocumentation in the Doc dir

   on Jun 18, 2025
3. github-project-automation

   added this to docs issueson Jun 18, 2025
4. github-project-automation

   moved this to Todo in docs issueson Jun 18, 2025
5. encukou

   added a commit that references this issue on Jun 18, 2025

   gh-135676: lexical analysis: Improve section on Numeric literals (GH-…

   21f3d15
6. miss-islington

   added a commit that references this issue on Jun 18, 2025

   pythongh-135676: lexical analysis: Improve section on Numeric literals (

   c8065a9
7. encukou

   added a commit that references this issue on Jun 18, 2025

   [3.14] gh-135676: lexical analysis: Improve section on Numeric litera…

   bb99270
8. lkollar

   added a commit that references this issue on Jun 19, 2025

   pythongh-135676: lexical analysis: Improve section on Numeric literals (

   bcfb5f7
9. Pranjal095

   added a commit that references this issue on Jul 12, 2025

   pythongh-135676: lexical analysis: Improve section on Numeric literals (

   d99a682
10. encukou

    added a commit that references this issue on Jul 23, 2025

    gh-135676: Lexical analysis: Reword String literals and related secti…

    777159f
11. miss-islington

    added a commit that references this issue on Jul 23, 2025

    pythongh-135676: Lexical analysis: Reword String literals and related…

    7a1a8cd
12. hugovk

    added a commit that references this issue on Jul 23, 2025

    [3.14] gh-135676: Lexical analysis: Reword String literals and relate…

    4832cea
13. taegyunkim

    added 2 commits that reference this issue on Aug 4, 2025

    pythongh-135676: lexical analysis: Improve section on Numeric literals (

    3e0a74a

    pythongh-135676: Lexical analysis: Reword String literals and related…

    8835e82
14. Agent-Hellboy

    added a commit that references this issue on Aug 19, 2025

    pythongh-135676: lexical analysis: Improve section on Numeric literals (

    02787e9
15. 9 remaining items
16. encukou

    added a commit that references this issue on Oct 8, 2025

    gh-135676: Add a summary of source characters (GH-138194)

    59a6f9d
17. miss-islington

    added a commit that references this issue on Oct 8, 2025

    pythongh-135676: Add a summary of source characters (pythonGH-138194)

    adc96b2
18. encukou

    added a commit that references this issue on Oct 8, 2025

    [3.14] gh-135676: Add a summary of source characters (GH-138194) (GH-…

    1c1a0bd
19. StanFromIreland

    marked Lexical analysis - Section 2.4.1 - More accurate description for long[string/bytes]char #110942 as a duplicate of this issue on Oct 30, 2025
20. encukou

    added a commit that references this issue on Nov 26, 2025

    gh-135676: Simplify docs on lexing names (GH-140464)

    2ff8608
21. StanFromIreland

    added a commit that references this issue on Nov 27, 2025

    [3.14] pythongh-135676: Simplify docs on lexing names (pythonGH-140464)

    69312e4
22. encukou

    added 2 commits that reference this issue on Dec 3, 2025

    [3.14] gh-135676: Simplify docs on lexing names (GH-140464) (GH-142015)

    79245a4

    gh-135676: Reword the f-string (and t-string) section (GH-137469)

    aea5531
23. miss-islington

    added 2 commits that reference this issue on Dec 3, 2025

    pythongh-135676: Reword the f-string (and t-string) section (pythonGH…

    1886791

    pythongh-135676: Reword the f-string (and t-string) section (pythonGH…

    baa16b9
24. StanFromIreland

    added a commit that references this issue on Dec 3, 2025

    [3.14] pythongh-135676: Reword the f-string (and t-string) section (p…

    dab6cb5
25. encukou

    added a commit that references this issue on Dec 4, 2025

    [3.14] gh-135676: Reword the f-string (and t-string) section (GH-137469…

    5611e08
26. StanFromIreland

    added 2 commits that reference this issue on Dec 6, 2025

    pythongh-135676: Simplify docs on lexing names (pythonGH-140464)

    9a421ea

    pythongh-135676: Reword the f-string (and t-string) section (pythonGH…

    eb4688e
27. encukou

    added a parent issue on Feb 11, 2026

    • Align the grammar documentation with Python's actual grammar #127833

Issue actions