GeezerVison is showing github.com — Stop reuse self-tests posting fake CI notices by mikolalysenko · Pull Request #1634 · SocketDev/socket-patch
Conversation
Problem
Every lint-ecosystems run (PR, merge queue and main push) posts 10 fake notice annotations, for example on main's latest push run (check run 117423824414 at 7d3086f):
Could not verify merge-queue CI (TimeoutExpired); running all checks.
Could not compare with the previous PR head (CalledProcessError); running all checks.
… 10 in total
Nothing failed. The reuse scripts never run in that job. These lines come from the job's python3 -B -m unittest discover -s scripts/tests step: the fallback self-tests let ci-reuse-merge-group.py and ci-reuse-pr-head.py print their real ::notice:: workflow commands to stdout, and GitHub turns those lines into annotations. A local run of the suite prints 15 of them, and GitHub keeps 10 (its per-step cap).
The harm:
- A reader can take the annotations to mean the merge-queue reuse check or the PR-head reuse check failed, which sends anyone debugging a CI-reuse decision down the wrong path.
- They fill the step's notice cap, so a real notice from that step would be dropped.
CI is otherwise healthy this run: the merge queue is empty, nothing is queued or running, main is green, and the only failed run in 3 days (38806080761) was fixed by that PR's next push. So this falls under priority 4 (make CI more robust).
Root cause
test_api_failure_timeout_and_bad_payload_* in scripts/tests/test_ci_reuse_merge_group.py and scripts/tests/test_ci_reuse_pr_head.py call reusable_run on its error path without capturing stdout.
Fix
A small assert_falls_back helper in each test file. It runs reusable_run under contextlib.redirect_stdout, asserts that the result is None (unchanged), and also asserts that the ::notice:: fallback line was printed. The fallback is now checked, not just silenced. No production script or workflow changes.
Proof
python3 -B -m unittest discover -s scripts/tests: 347 tests, OK (6 skipped). Lines starting with :: in stdout went from 15 to 0.- Negative check: removing
::notice:: from ci-reuse-merge-group.py makes the new assertion fail 7 subtests.
Removed or moved tests
None. Every test still runs in the same place (lint-ecosystems).
https://claude.ai/code/session_01SeHhBxEvePf8ERTaTTenMK
Generated by Claude Code
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
The fallback tests for ci-reuse-merge-group.py and ci-reuse-pr-head.py
let the scripts print their real "::notice::Could not verify
merge-queue CI (...)" lines to stdout. lint-ecosystems runs those tests
in a GitHub job, so every run on PRs, the merge queue and main gained
ten fake annotations claiming the reuse check had failed. They fill the
step's ten-notice cap and send anyone debugging a reuse decision down
the wrong path.
Capture stdout in those cases and assert that the notice is printed.
The fallback path is now checked, not just silenced.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01SeHhBxEvePf8ERTaTTenMK
cursor[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 564c31a. Configure here.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Labeled Ready for review (burn-down agent).
- Head:
564c31a
- CI: 54/54 green (35 success, 19 skipped). No main-wide failures.
- Bugbot: reviewed
564c31a, no findings. 0 open threads.
- Mergeable with no conflicts.
CHANGELOG.md is not touched.
- Reviewer focus: only the two reuse self-tests under
scripts/tests/ change; check they no longer emit real ::notice annotations.
Slack announcement not sent this run (the Slack send tool was unavailable). The next run will retry.
Generated by Claude Code
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
The merge queue removed this PR at 17:49 UTC because e2e (ubuntu-latest, e2e_vex_build, pipenv:: --ignored, 2026.8.0) failed (run 39467483928). That failure doesn't come from this PR:
Cause: uv 0.13.2 (released 2026-10-19) rejects the environments = [] that Pipenv 2026.8.0 writes, and the Pipenv/pip legs install the latest uv. I bisected it locally: 0.13.1 accepts the file and 0.13.2–0.13.4 reject it.
#1639 pins setup-uv to 0.13.1 and fixes this. I'm not porting the pin here, so the fix doesn't land twice. Once #1639 merges, this PR can go back in the queue unchanged, and its approval still stands.
Generated by Claude Code
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
[final reviewer] Re-added to the merge queue (squash, auto-merge) at 564c31a13, which is the commit Wenxin Jiang (@Wenxin-Jiang) approved. The earlier eviction was the main-wide uv 0.13.2 Pipenv pylock break. main now pins setup-uv to 0.13.1 (#1605), and the same e2e … pipenv:: --ignored, 2026.8.0 leg is green on main a54753a. This is the one re-enqueue after that eviction.
Generated by Claude Code
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Labels
ci-janitor
Opened by the CI janitor routine (flakes, redundant tests, CI perf)
Ready for review
Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|