GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-41690: Use a loop to collect args in the parser instead of recursion by pablogsal · Pull Request #22053 · python/cpython

Conversation

@pablogsal

@pablogsal pablogsal commented Sep 2, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

https://bugs.python.org/issue41690

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Sep 2, 2020
@bedevere-bot bedevere-bot added the awaiting core review label Sep 2, 2020
@pablogsal
pablogsal force-pushed the pegen_args branch from bd6c7c5 to b1100ae Compare September 2, 2020 02:08
@pablogsal pablogsal mentioned this pull request Sep 2, 2020
RecursionError in generated Python parser we-like-parsers/pegen_experiments#245
Open
@pablogsal
Use a loop to collect args in the parser instead of recursion
b66524a
@pablogsal
pablogsal force-pushed the pegen_args branch from b1100ae to b66524a Compare September 2, 2020 02:14
gvanrossum
gvanrossum reviewed Sep 2, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just style nits -- should I file a bpo bug to track this?

It's awkward that this is happening so close to rc2. But it's technically a regression from 3.8, and a legit way to cause a segfault without even running code. Plus for N=20000 you can really feel the quadratic time.

@ambv We'd like this to be adopted into rc2 (in about 10 days).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Parser/pegen.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Parser/pegen.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal
Fix style issues
15632cd
@pablogsal
pablogsal marked this pull request as ready for review September 2, 2020 10:57
@pablogsal
pablogsal requested a review from lysnikolaou as a code owner September 2, 2020 10:57
@pablogsal pablogsal changed the title Use a loop to collect args in the parser instead of recursion bpo-41690: Use a loop to collect args in the parser instead of recursion Sep 2, 2020
@pablogsal

pablogsal commented Sep 2, 2020

Copy link
Copy Markdown
Member Author

Opened https://bugs.python.org/issue41690 to track this

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal
Add NEWS entry
df2b506
gvanrossum
gvanrossum approved these changes Sep 2, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Sep 2, 2020
@gvanrossum gvanrossum added the needs backport to 3.9 label Sep 2, 2020
@pablogsal
pablogsal merged commit 4a97b15 into python:master Sep 2, 2020
@miss-islington

miss-islington commented Sep 2, 2020

Copy link
Copy Markdown
Contributor

Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Sep 2, 2020
@pablogsal
pablogsal deleted the pegen_args branch September 2, 2020 16:44
@miss-islington

miss-islington commented Sep 2, 2020

Copy link
Copy Markdown
Contributor

Sorry, @pablogsal, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 4a97b1517a6b5ff22e2984b677a680b07ff0ce11 3.9

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington miss-islington assigned pablogsal Sep 2, 2020
pablogsal added a commit to pablogsal/cpython that referenced this pull request Sep 2, 2020
@pablogsal
[3.9] bpo-41690: Use a loop to collect args in the parser instead of …
461b264
…recursion (pythonGH-22053)

This program can segfault the parser by stack overflow:

```
import ast

code = "f(" + ",".join(['a' for _ in range(100000)]) + ")"
print("Ready!")
ast.parse(code)
```

the reason is that the rule for arguments has a simple recursion when collecting args:

args[expr_ty]:
    [...]
    | a=named_expression b=[',' c=args { c }] {
        [...] }.
(cherry picked from commit 4a97b15)

Co-authored-by: Pablo Galindo <[email protected]>
@bedevere-bot

bedevere-bot commented Sep 2, 2020

Copy link
Copy Markdown

GH-22067 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.9 label Sep 2, 2020
pablogsal added a commit that referenced this pull request Sep 2, 2020
@pablogsal
[3.9] bpo-41690: Use a loop to collect args in the parser instead of …
8de34cd
…recursion (GH-22053) (GH-22067)

This program can segfault the parser by stack overflow:

```
import ast

code = "f(" + ",".join(['a' for _ in range(100000)]) + ")"
print("Ready!")
ast.parse(code)
```

the reason is that the rule for arguments has a simple recursion when collecting args:

args[expr_ty]:
    [...]
    | a=named_expression b=[',' c=args { c }] {
        [...] }.
(cherry picked from commit 4a97b15)

Co-authored-by: Pablo Galindo <[email protected]>
shihai1991 added a commit to shihai1991/cpython that referenced this pull request Sep 5, 2020
@shihai1991
Merge remote-tracking branch 'origin/master' into bpo_41073_add_types…
7537c86
…lots1

* origin/master: (63 commits)
  bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (pythonGH-22098)
  bpo-38585: Remove references to defusedexpat (pythonGH-22095)
  bpo-41721: Add xlc options (pythonGH-22096)
  bpo-40486: Specify what happens if directory content change diring iteration (pythonGH-22025)
  bpo-41638: Improve ProgrammingError message for absent parameter. (pythonGH-21999)
  bpo-41713: _signal doesn't use multi-phase init (pythonGH-22087)
  bpo-41700: Skip test if the locale is not supported (pythonGH-22081)
  [doc] Update documentation on logging optimization. (pythonGH-22075)
  Fix 'gather' rules in the python parser generator (pythonGH-22021)
  bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (pythonGH-22077)
  [doc] Fix a typo in the graphlib docs (python#22030)
  bpo-1635741: Port _signal module to multi-phase init (PEP 489) (pythonGH-22049)
  bpo-39883: Use BSD0 license for code in docs (pythonGH-17635)
  bpo-39010: Improve test shutdown (python#22066)
  bpo-41696: Fix handling of debug mode in asyncio.run (python#22069)
  bpo-41690: Use a loop to collect args in the parser instead of recursion (pythonGH-22053)
  closes bpo-41689: Preserve text signature from tp_doc in C heap type creation. (pythonGH-22058)
  Fix invalid escape sequences in the peg_highlight Sphinx extension (pythonGH-22047)
  bpo-41675: Modernize siginterrupt calls (pythonGH-22028)
  bpo-41685: Don't pin setuptools version anymore in Doc/Makefile (pythonGH-22062)
  ...
xzy3 pushed a commit to xzy3/cpython that referenced this pull request Oct 18, 2020
@pablogsal @xzy3-cdc
bpo-41690: Use a loop to collect args in the parser instead of recurs…
de11d22
…ion (pythonGH-22053)

This program can segfault the parser by stack overflow:

```
import ast

code = "f(" + ",".join(['a' for _ in range(100000)]) + ")"
print("Ready!")
ast.parse(code)
```

the reason is that the rule for arguments has a simple recursion when collecting args:

args[expr_ty]:
    [...]
    | a=named_expression b=[',' c=args { c }] {
        [...] }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes
@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@pablogsal pablogsal

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@pablogsal @miss-islington @bedevere-bot @gvanrossum @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.