GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Normalize source newlines before warnings, compilation and diagnostics by youknowdot · Pull Request #9793 · RustPython/RustPython

Conversation

@youknowdot

@youknowdot youknowdot commented Nov 10, 2026 •
edited by youknowone
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Summary

Normalize CRLF and lone CR before parser warnings and compilation, including physical newlines inside string literals. This follows CPython 3.15's _PyTokenizer_translate_newlines / _PyTokenizer_FromUTF8 ordering.

This is an independent alternative to close #9762, not stacked on it. The original change fixes literal values but runs after VM parser warnings and leaves diagnostics reading the untranslated input.

  • Share one universal-newline translator between source decoding, compiler entry points, symbol-table compilation, and syntax diagnostics.
  • Translate before VM parser warning collection. Keep diagnostic line lookup consistent with the translated parser source.
  • Borrow CR-free input; otherwise use a single output allocation instead of chained full-string replacements.
  • Preserve the caller's original source bytes in compile audit events.
  • Cover CR/CRLF combinations, ordinary/raw/bytes/f-/t-string literals, docstrings, explicit escape preservation, AST compilation, warnings, error text consistency and audit input.

Verification

  • CPython 3.15.1 and RustPython builtin_compile.py / builtin_compile_module.py: pytest 4 passed.
  • RustPython test_compile: passed (187 tests run, 51 skipped).
  • CLI -c physical-CR literal checks: passed.
  • Full workspace Rust tests, separate CAPI tests, workspace/CAPI clippy with -D warnings: passed.
  • Shared normalizer tests and pre-commit hooks: passed.

This does not claim full parser diagnostic parity: existing LF-input differences in trailing newlines in SyntaxError.text and warning-as-error precedence for invalid source also remain for CR input. This patch makes the newline forms consistent; it does not hide those independent differences.

AI assistance: OpenAI Codex.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@youknowdot
Normalize physical source newlines before parser warnings and compila…
b9def6a
…tion

Share universal-newline translation across byte decoding, compiler entry points and syntax diagnostics. Keep LF-only input borrowed and preserve original compile audit bytes. Cover literal values, docstrings, warnings and diagnostic consistency.

Assisted-by: OpenAI Codex:GPT-6.1
@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Explain Change →

Pre-merge checks: Passed 1 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Passed Passed checks: 1 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

  • X
  • Mastodon
  • Reddit
  • LinkedIn
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowdot
Mark normalized source result as must-use
b658c0a
Assisted-by: OpenAI Codex:GPT-6.1
@youknowone
youknowone marked this pull request as ready for review November 11, 2026 02:36
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-11T02:36:32.681578Z
We couldn’t complete this review. Comment @codex review to try again.
b658c0a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

youknowone
youknowone approved these changes Nov 11, 2026
View reviewed changes
@youknowone
youknowone merged commit 5bfbb3b into RustPython:main Nov 11, 2026
28 checks passed

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@youknowdot @youknowone
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.