GeezerVison is showing github.com — Normalize source newlines before warnings, compilation and diagnostics by youknowdot · Pull Request #9793 · RustPython/RustPython
Conversation
youknowdot
commented
Nov 10, 2026
•
edited by youknowone
Loading
Uh oh!
There was an error while loading. Please reload this page.
Summary
Normalize CRLF and lone CR before parser warnings and compilation, including physical newlines inside string literals. This follows CPython 3.15's _PyTokenizer_translate_newlines / _PyTokenizer_FromUTF8 ordering.
This is an independent alternative to close #9762, not stacked on it. The original change fixes literal values but runs after VM parser warnings and leaves diagnostics reading the untranslated input.
- Share one universal-newline translator between source decoding, compiler entry points, symbol-table compilation, and syntax diagnostics.
- Translate before VM parser warning collection. Keep diagnostic line lookup consistent with the translated parser source.
- Borrow CR-free input; otherwise use a single output allocation instead of chained full-string replacements.
- Preserve the caller's original source bytes in
compile audit events.
- Cover CR/CRLF combinations, ordinary/raw/bytes/f-/t-string literals, docstrings, explicit escape preservation, AST compilation, warnings, error text consistency and audit input.
Verification
- CPython 3.15.1 and RustPython
builtin_compile.py / builtin_compile_module.py: pytest 4 passed.
- RustPython
test_compile: passed (187 tests run, 51 skipped).
- CLI
-c physical-CR literal checks: passed.
- Full workspace Rust tests, separate CAPI tests, workspace/CAPI clippy with
-D warnings: passed.
- Shared normalizer tests and pre-commit hooks: passed.
This does not claim full parser diagnostic parity: existing LF-input differences in trailing newlines in SyntaxError.text and warning-as-error precedence for invalid source also remain for CR input. This patch makes the newline forms consistent; it does not hide those independent differences.
AI assistance: OpenAI Codex.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
…tion
Share universal-newline translation across byte decoding, compiler entry points and syntax diagnostics. Keep LF-only input borrowed and preserve original compile audit bytes. Cover literal values, docstrings, warnings and diagnostic consistency.
Assisted-by: OpenAI Codex:GPT-6.1
coderabbitai
Bot
commented
Nov 10, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Assisted-by: OpenAI Codex:GPT-6.1
Codex Review Summary: ❌ Failed
This comment shows the latest Codex review activity on this pull request.
| Review |
Status |
Commit |
Review trigger |
| 📝 Code Review |
❌ Failed 2026-11-11T02:36:32.681578Z We couldn’t complete this review. Comment @codex review to try again. |
b658c0a |
Draft marked ready |
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review" or "@codex security review".
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|