GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Guard raw and inspected signature observations in CI by youknowdot · Pull Request #9628 · RustPython/RustPython

Conversation

@youknowdot

@youknowdot youknowdot commented Nov 3, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Purpose

Turn the existing raw text-signature/result-persistence follow-up into a real CI regression gate, without another RustPython build or scanner pass. This remains separate from the merged non-raw reporting prerequisite (#9726).

Rebased onto main b66534d448. Current head 9671f56cee.

Changes

  • Keep native raw __text_signature__ observations independent of inspect.signature comparisons; preserve type-document and unknown distinctions.
  • Reuse the existing three-OS scan. Compare exact individual observed failures and selected-scope fingerprints against platform-specific baselines.
  • Reject new/changed failures, unknown/coverage regressions, missing required observation metadata, and stale resolved exceptions. Requiring resolved exceptions to disappear prevents later reintroduction under an old allowance.
  • Preserve full scan/context artifacts even if the gate fails. Pin the CPython oracle through .python-version; check exact OS/architecture/version/build-feature context.
  • Keep the baseline generation command explicit and outside CI.

Baseline provenance and remaining work

The initial three baselines were generated from the existing verified Linux/macOS/Windows artifacts of CI run 42421287760, merge 5da7d5b1d1d56467fb60997bbdb66676e0c6c33d, CPython 3.15.0. Downloaded ZIP SHA256 values match GitHub's artifact metadata.

Baseline refresh completed: exact-head run 42491705325, head 4a6ddde862, merge 3b4735987efcc7ba59d454a1f6cf9d667a631778, produced all three platform artifacts. Removed 30 resolved observations on Linux/macOS and 28 on Windows; reviewed retained pre-existing main differences. Normalized only process addresses in exception repr diagnostics, not raw signatures or default literals. The gate now runs immediately after the existing build to avoid expensive tests after a detected signature regression.

Final-head validation passed: CI run 42496222992 passed on head 9671f56cee21682a26cc4c52f44924cb72114c5b, including the signature gate and native tests on Linux, Windows and macOS. This PR remains draft for maintainer review. Do not interpret old green CI or self-comparison as validation of current main. This gate is an observed-surface ratchet, not a claim of zero signature mismatches, complete discovery, argument behavior parity or CPython compatibility. Unknowns remain unverified. No runtime/Lib changes are included.

Validation

  • 59 focused whatsleft and gate tests pass under CPython 3.15.1.
  • All three current hosted platform artifacts pass the refreshed checker and matching-context CLI; deliberate reintroduction of a resolved queue mismatch fails on every platform.
  • Independent review reproduced the important negative paths: missing provenance is rejected; resolved exceptions require deletion; unchanged observations pass.
  • Repository hooks and git diff --check pass after rebase.
  • Previous head hosted CI passed Rust/C API tests, Clippy, WASI, lint, and native snippets/CPython tests on all three OSes; only the expected baseline gate failed. Final-head changes are checker/tests/baselines and step ordering only. Final-head hosted integration subsequently passed (run 42496222992), including all three signature gates, native tests, Rust tests, Clippy, WASI and lint; no additional local Rust build was performed.

AI assistance

Implementation and independent review assisted by dot; exact model version unavailable in this runtime.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

  • X
  • Mastodon
  • Reddit
  • LinkedIn
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowdot youknowdot mentioned this pull request Nov 3, 2026
Record reference module discovery #9645
Merged
@youknowdot youknowdot changed the title Record native raw text signatures separately Record native raw signatures and scan coverage Nov 3, 2026
@codspeed

codspeed Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowdot:fix/whatsleft-raw-text-observations-20261103 (5654448) with main (e9d1d9f)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowone youknowone changed the title Record native raw signatures and scan coverage Consolidate whats_left reporting and comparison artifacts Nov 5, 2026
This was referenced Nov 5, 2026
Preserve whats_left comparisons in CI #9565
Closed
Report uncomparable reference signatures #9577
Closed
Preserve whats_left scan coverage and reference errors #9726
Merged
@youknowdot
youknowdot force-pushed the fix/whatsleft-raw-text-observations-20261103 branch from c03518e to 4057bc8 Compare November 5, 2026 15:41
@youknowone youknowone changed the title Consolidate whats_left reporting and comparison artifacts Record raw text signatures after the reporting prerequisite Nov 5, 2026
@youknowdot
youknowdot force-pushed the fix/whatsleft-raw-text-observations-20261103 branch from 4057bc8 to f13307d Compare November 5, 2026 15:50
@youknowone youknowone changed the title Record raw text signatures after the reporting prerequisite Add raw text signatures and persist comparison results Nov 5, 2026
@youknowdot
youknowdot force-pushed the fix/whatsleft-raw-text-observations-20261103 branch from f13307d to fa1c3b2 Compare November 5, 2026 15:55
youknowone
youknowone requested changes Nov 6, 2026
View reviewed changes

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tell me what's the benefit by doing this

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@youknowdot
youknowdot force-pushed the fix/whatsleft-raw-text-observations-20261103 branch 2 times, most recently from 143c481 to 4a6ddde Compare November 8, 2026 14:49
@youknowdot youknowdot changed the title Add raw text signatures and persist comparison results Guard raw and inspected signature observations in CI Nov 8, 2026
This was referenced Nov 8, 2026
Add checked legacy Python 3.15 text signature overrides #9756
Draft
Honor preserve_exc truth conversion before interpreter calls #9759
Merged
codex and others added 6 commits November 9, 2026 20:41
@codex @youknowdot
Guard raw and inspected signature observations in CI
970c858
Keep the raw/persistence follow-up together, with exact per-platform observations and selected-scope coverage. Reject stale resolved exceptions to prevent reintroduction.

Assisted-by: dot: model version unavailable
@codex @youknowdot
Refresh signature baselines from three-platform CI
6cc9410
Remove resolved observations, normalize only exception repr addresses, and fail before costly CPython tests when the signature gate detects a regression.

Assisted-by: dot: model version unavailable
@codex @youknowdot
Allow additional RustPython signatures without a baseline exception
b53d3eb
Keep raw observations and coverage while accepting extra text when CPython has no signature. Preserve unknown/context and reverse missing-signature checks. Reclassify the existing three-platform artifacts without widening their scopes.

Assisted-by: OpenAI:ChatGPT
@codex @youknowdot
Classify additional signatures as allowed in whatsleft reports
cac88e5
Keep reference diagnostics in JSON while excluding allowed extra signatures from human warning output and warning counts.

Assisted-by: OpenAI:ChatGPT
@youknowdot
Narrow whatsleft CI baselines to signature regressions
2a4113b
Keep signature mismatches and four checked-surface fingerprints instead of freezing unrelated unknowns, imports, missing APIs and class-selection decisions. Preserve missing or invalid signatures on existing comparable APIs and reject ambiguity or loss of checked coverage. Reclassify the recorded three-platform artifacts without changing interpreter behavior.

Assisted-by: OpenAI:ChatGPT
@youknowdot
Remove buffer signature exceptions resolved on current main
229579a
Rebase on a8bd43e and remove the four platform-independent raw IO default mismatches fixed by RustPython#9744. The stored provenance remains the original three-platform artifacts, with this source-checked adjustment recorded explicitly.

Assisted-by: OpenAI:ChatGPT
@youknowdot
youknowdot force-pushed the fix/whatsleft-raw-text-observations-20261103 branch from 537513b to 229579a Compare November 9, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone requested changes

Requested changes must be addressed to merge this pull request.

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@youknowdot @youknowone @codex
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.