GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Replace Python parser with ruff parser by etaloof · Pull Request #5453 · RustPython/RustPython

Conversation

@etaloof

@etaloof etaloof commented Dec 2, 2024

Copy link
Copy Markdown
Contributor

This PR replaces the Python parser with the ruff Parser. It is based on the work of the earlier PR #5423 (thanks @qingshi163)! Code generation has been updated and should mostly work now. There is is still some work left to fix the built-in AST module, REPL (probably) and I haven't looked into WASM support.

Running the tests with cargo test --workspace --exclude rustpython_wasm --features freeze-stdlib gives me one remaining test failure: test_run_script in src/lib.rs. It seems fail when importing the extra_tests/snippets/dir_main but I am not sure why that happens.

Concerning the AST module: The previous implementation is using a lot of generated code (script). Not sure if we should continue updating this script, I'll look into it. @youknowone do you have any suggestions on this?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
qingshi163 and youknowone reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
qingshi163 and others added 21 commits September 10, 2024 19:09
@qingshi163
stage1
93b7d6b
@qingshi163
compiler pass build
e0a3f26
@qingshi163
introduce rustpython-compiler-source
38575bd
@qingshi163
stage2
dce0d94
@qingshi163
fixup
3162b9b
@qingshi163
pass compile
1f7024a
@etaloof
Fix hello world compiler test
e189ecf
@etaloof
Fix code generation for if-elif-else statement
0f100e9
@etaloof
Fix code generation for lambda expression
b42f88f
@etaloof
Fix code generation for integers
7c53857
@etaloof
Fix code generation for fstrings
424e05a
@etaloof
Fix code generation for if statement
2691cbe
@etaloof
Fix code generation for if statement
a98a623
@etaloof
Fix code generation for if statement
168aa01
@etaloof
Fix code generation for fstring
c25d958
@etaloof
Fix code generation for class definition
4e29dd4
@etaloof
Replace feature flags
89afad2
@etaloof
Initialize frozen core modules
e760ac7
@etaloof
Allow __future__ import after module doc comment
2987cbe
@etaloof
Disable ast module
c10df16
@etaloof
Commit remaining fixes for compile errors in examples
bcfbf88
@youknowone

youknowone commented Dec 3, 2024

Copy link
Copy Markdown
Member

Thank you so much! Since ruff already has its own ast coworking with same parser, how about starting from ruff side again about it too?
I couldnt look in details yet, but i feel like code generated ast module is not cost efficient anymore. We now can borrow lots of code from other repository easy
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@youknowone

youknowone commented Dec 4, 2024

Copy link
Copy Markdown
Member

@qingshi163 @coolreader18 do you also have advices?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@etaloof
Fix some warnings
def0e42
@youknowone

youknowone commented Dec 28, 2024

Copy link
Copy Markdown
Member

I have to look in this PR, sorry for delay
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
etaloof added 5 commits January 3, 2025 21:22
@etaloof
Update ast stdlib module
4bef13b
@etaloof
Update ast stdlib module
1fd48c7
@etaloof
Update ast stdlib module
ca2b302
@etaloof
Update ast stdlib module
e9bc377
@etaloof
Update ast stdlib module
c6ece03
etaloof added 10 commits January 6, 2025 14:01
@etaloof
Split ast stdlib module into files
5c22c39
@etaloof
Fix codegen for positional arguments with defaults
6b49cfb
@etaloof
Update ast stdlib module
4cdd4ae
@etaloof
Update ast stdlib module
f03d43f
@etaloof
Extract string and constant handling from expression.rs
60dc970
@etaloof
Always add required fields to AST nodes
5bd4d5b
@etaloof
Compile doc strings correctly again
85edb25
@etaloof
Enable "ast" Cargo feature by default
f4154a0
@etaloof
Refactor compilation of big integer literal
1bff3dc
@etaloof
Update ast stdlib module
a669803
@youknowone youknowone mentioned this pull request Jan 18, 2025
Recruiting maintainers #5387
Open
etaloof added 4 commits January 18, 2025 18:45
@etaloof
Update ast stdlib module
1bd4078
@etaloof
Update ast stdlib module
a054ca6
@etaloof
Reset barebones example
1476b49
@etaloof
Merge remote-tracking branch 'refs/remotes/origin/main' into ruff-parser
1a3c5bb 
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	compiler/codegen/src/compile.rs
#	compiler/codegen/src/symboltable.rs
#	wasm/lib/src/convert.rs
@etaloof

etaloof commented Jan 25, 2025

Copy link
Copy Markdown
Contributor Author

@youknowone I think this is ready to be reviewed now. I went ahead and merged main into this branch to resolve the merge conflicts (see commit 1a3c5bb. Not sure if this is the correct way to do things, I can rebase and merge again if needed.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
etaloof added 3 commits January 25, 2025 21:15
@etaloof
Fix some left-over warnings
595622b
@etaloof
Undo accidential change
c00d58f
@etaloof
Adapt shell to ruff parser
d08716f
@youknowone

youknowone commented Jan 27, 2025

Copy link
Copy Markdown
Member

@etaloof could you make your fork of RustPython not from qingshi163's but from RustPython org's? Then I can directly push my edits to your PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@youknowone

youknowone commented Jan 27, 2025

Copy link
Copy Markdown
Member

It looks going well. this commit will pin parser version to git version of Ruff c8acbf8
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
youknowone
youknowone reviewed Jan 27, 2025
View reviewed changes

@youknowone youknowone left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, I feel like everything is going to the right direction.
Is there any blocking issue?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread compiler/core/src/mode.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread vm/src/stdlib/ast/basic.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread compiler/codegen/src/compile.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.
@etaloof etaloof closed this by deleting the head repository Jan 27, 2025
@etaloof

etaloof commented Jan 27, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Thank you, I deleted my fork from qingshi163's repository and forked again from RustPython. Could you please open this PR again? Sorry for the confusion.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@youknowone

youknowone commented Jan 28, 2025

Copy link
Copy Markdown
Member

a PR is only can be re-opened when the branch is not changed. This one doesn't seem so. Please open a new one.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
This was referenced Jan 28, 2025
ruff-parser etaloof/RustPython#1
Closed
Replace Python parser with ruff parser #5494
Merged
@etaloof

etaloof commented Jan 28, 2025

Copy link
Copy Markdown
Contributor Author

Thanks, I have created a new PR #5494.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@etaloof @youknowone @qingshi163
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.