GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Set python args to be like CPython in CI by ShaharNaveh · Pull Request #8181 · RustPython/RustPython

Conversation

@ShaharNaveh

@ShaharNaveh ShaharNaveh commented Jun 26, 2026 •
edited by coderabbitai Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor
  • Closes #xxxx
  • This PR follows our AI policy

Summary

I've recently made some patches to CPython, and noticed that their CI output is less noisy. made it easier to find the failing tests

lmk what you think

Summary by CodeRabbit

Summary by CodeRabbit

  • Tests
    • Updated CI CPython and flaky retry runs to use standardized test execution flags (-m test --slow-ci) for more consistent coverage.
    • Centralized timeout and related runtime settings in the job matrix to improve reliability and reduce command-level variance.
    • Adjusted the environment “polluters” verification loop to follow the same standardized test behavior.
  • Chores
    • Enabled consistent colored output in CI logs via a workflow-level setting.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

doma17 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@ShaharNaveh
ShaharNaveh marked this pull request as ready for review June 26, 2026 17:30
@coderabbitai

coderabbitai Bot commented Jun 26, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: ca747555-366e-47cd-895b-dc1657211b16

📥 Commits

Reviewing files that changed from the base of the PR and between 7b47348 and 674bfa3.

📒 Files selected for processing (1)
  • .github/workflows/ci.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/ci.yaml

📝 Walkthrough

Walkthrough

The CI workflow now sets FORCE_COLOR=1, expands matrix test arguments, and updates three CPython test invocations to use --slow-ci with revised timeout and option handling.

Changes

CI workflow updates

Layer / File(s) Summary
Workflow env and matrix args
.github/workflows/ci.yaml
The workflow sets FORCE_COLOR globally and expands the snippets_cpython matrix extra_test_args with timeout and python-opts flags.
CPython test commands
.github/workflows/ci.yaml
The main CPython run, flaky MP retry loop, and env-polluter verification command switch to --slow-ci and updated argument handling.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: youknowone

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: CI test commands are adjusted to use CPython-like Python args and options.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
  • X
  • Mastodon
  • Reddit
  • LinkedIn

Comment @coderabbitai help to get the list of available commands.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-advanced-security[bot]
github-advanced-security Bot found potential problems Jun 26, 2026
View reviewed changes
Comment thread .github/workflows/ci.yaml Fixed

Uh oh!

There was an error while loading. Please reload this page.

coderabbitai[bot]
coderabbitai Bot reviewed Jun 26, 2026
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/ci.yaml (1)

361-374: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Move workflow expressions out of run: to satisfy zizmor.

The new flags are fine, but Lines 363 and 374 still interpolate ${{ ... }} directly into the shell body. The matrix values shown here are hardcoded, so this is not an exploitable injection path today, but zizmor will keep flagging the step until those expressions are moved into env: (or otherwise removed from run:).

Suggested shape
       - name: Run CPython tests
+        env:
+          RUSTPYTHON_SKIP_ENV_POLLUTERS: true
+          EXTRA_TEST_ARGS: ${{ join(matrix.extra_test_args, ' ') }}
+          SKIPS: ${{ join(matrix.skips, ' ') }}
         run: |
-          target/release/rustpython -u -W error -bb -E -m test -j ${{ steps.cores.outputs.cores }} ${{ join(matrix.extra_test_args, ' ') }} --slowest --fail-env-changed --timeout 600 -x ${{ env.FLAKY_MP_TESTS }} ${{ join(matrix.skips, ' ') }}
+          read -r -a extra_test_args <<< "$EXTRA_TEST_ARGS"
+          read -r -a skips <<< "$SKIPS"
+          target/release/rustpython -u -W error -bb -E -m test -j ${{ steps.cores.outputs.cores }} "${extra_test_args[@]}" --slowest --fail-env-changed --timeout 600 -x ${{ env.FLAKY_MP_TESTS }} "${skips[@]}"
         timeout-minutes: ${{ matrix.timeout }}
-        env:
-          RUSTPYTHON_SKIP_ENV_POLLUTERS: true

As per coding guidelines, "If you modify any file under .github/workflows/, the change must pass a zizmor scan in CI."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci.yaml around lines 361 - 374, Move the workflow
expressions out of the shell bodies in the CPython test steps so zizmor stops
flagging them. In the “Run CPython tests” and “Run flaky MP CPython tests”
steps, take the `${{ ... }}` values currently embedded in `run:` (for example
the matrix args and flaky test flags) and pass them through `env:` instead, then
reference the environment variables inside the commands. Keep the existing step
names and test commands in `.github/workflows/ci.yaml`, but ensure `run:`
contains no direct GitHub Actions expressions.

Sources: Coding guidelines, Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/ci.yaml:
- Around line 361-374: Move the workflow expressions out of the shell bodies in
the CPython test steps so zizmor stops flagging them. In the “Run CPython tests”
and “Run flaky MP CPython tests” steps, take the `${{ ... }}` values currently
embedded in `run:` (for example the matrix args and flaky test flags) and pass
them through `env:` instead, then reference the environment variables inside the
commands. Keep the existing step names and test commands in
`.github/workflows/ci.yaml`, but ensure `run:` contains no direct GitHub Actions
expressions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 0d89bebf-1572-4dd8-99a2-ba243a324d40

📥 Commits

Reviewing files that changed from the base of the PR and between 0009dd6 and 16899d2.

📒 Files selected for processing (1)
  • .github/workflows/ci.yaml

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
github-advanced-security[bot]
github-advanced-security Bot found potential problems Jun 26, 2026
View reviewed changes
Comment thread .github/workflows/ci.yaml Fixed

Uh oh!

There was an error while loading. Please reload this page.

coderabbitai[bot]
coderabbitai Bot reviewed Jun 26, 2026
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/ci.yaml (1)

374-399: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Keep -W error out of these special-purpose retry loops.

Line 399’s loop only recognizes exit code 3 as “still polluting”. With -W error, a warning now fails with a different status, so the job can incorrectly report “no longer polluting” even though it never saw a clean run. The same flag on Line 374 also turns deterministic warning failures into five pointless retries.

Suggested fix
-            target/release/rustpython -u -W error -bb -E -m test -j 1 ${{ join(matrix.extra_test_args, ' ') }} --slowest --fail-env-changed --timeout 600 ${{ env.FLAKY_MP_TESTS }}
+            target/release/rustpython -u -bb -E -m test -j 1 ${{ join(matrix.extra_test_args, ' ') }} --slowest --fail-env-changed --timeout 600 ${{ env.FLAKY_MP_TESTS }}
-              target/release/rustpython -u -W error -bb -E -m test -j 1 --slowest --fail-env-changed --timeout 600 "${thing}"
+              target/release/rustpython -u -bb -E -m test -j 1 --slowest --fail-env-changed --timeout 600 "${thing}"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci.yaml around lines 374 - 399, Remove the `-W error` flag
from the retry-based test invocations in the CI workflow so these loops don’t
turn warnings into different failure modes; update the `rustpython` command used
in the retry loop and the cpython env-polluter check to rely on the existing
exit-code logic instead. Keep the special handling in the retry loops (`status`,
`--fail-env-changed`, and the `thing`/`TARGETS` iterations) unchanged so exit
code `3` remains the only signal for continued pollution.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/ci.yaml:
- Around line 374-399: Remove the `-W error` flag from the retry-based test
invocations in the CI workflow so these loops don’t turn warnings into different
failure modes; update the `rustpython` command used in the retry loop and the
cpython env-polluter check to rely on the existing exit-code logic instead. Keep
the special handling in the retry loops (`status`, `--fail-env-changed`, and the
`thing`/`TARGETS` iterations) unchanged so exit code `3` remains the only signal
for continued pollution.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 8eda90b1-ad3c-4957-81cc-a08d767a915f

📥 Commits

Reviewing files that changed from the base of the PR and between 16899d2 and ef40c7e.

📒 Files selected for processing (1)
  • .github/workflows/ci.yaml

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@ShaharNaveh
Set python args to be like CPython in CI (ish)
bd809ce
@ShaharNaveh
ShaharNaveh force-pushed the ci-cpython-like-args branch from ef40c7e to bd809ce Compare June 26, 2026 18:18
@ShaharNaveh
ShaharNaveh marked this pull request as draft June 26, 2026 18:18
github-advanced-security[bot]
github-advanced-security Bot found potential problems Jun 26, 2026
View reviewed changes
Comment thread .github/workflows/ci.yaml Fixed

Uh oh!

There was an error while loading. Please reload this page.

@ShaharNaveh
Use --slow-ci
128cee9
github-advanced-security[bot]
github-advanced-security Bot found potential problems Jun 27, 2026
View reviewed changes
Comment thread .github/workflows/ci.yaml
- name: Run CPython tests
run: |
target/release/rustpython -m test -j ${{ steps.cores.outputs.cores }} ${{ join(matrix.extra_test_args, ' ') }} --slowest --fail-env-changed --timeout 600 -v -x ${{ env.FLAKY_MP_TESTS }} ${{ join(matrix.skips, ' ') }}
target/release/rustpython -u -m test --slow-ci -j ${{ steps.cores.outputs.cores }} ${{ join(matrix.extra_test_args, ' ') }} -x ${{ env.FLAKY_MP_TESTS }} ${{ join(matrix.skips, ' ') }}
ShaharNaveh added 2 commits June 27, 2026 12:16
@ShaharNaveh
--dont-add-python-opts
74aca0c
@ShaharNaveh
unmark passing test
7b47348
@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/future.py

dependencies:

  • future

dependent tests: (35 tests)

  • future: test_flufl test_future_stmt test_generator_stop test_pydoc test_zoneinfo
    • codeop: test_codeop
      • pdb: test_pdb
      • traceback: test_asyncio test_builtin test_code_module test_contextlib test_contextlib_async test_coroutines test_dictcomps test_exceptions test_http_cookiejar test_importlib test_iter test_listcomps test_pyexpat test_setcomps test_socket test_ssl test_subprocess test_sys test_threadedtempfile test_threading test_traceback test_unittest test_with test_zipimport
    • importlib.metadata: test_importlib
    • pydoc: test_enum
      • xmlrpc.server: test_docxmlrpc test_xmlrpc

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ShaharNaveh
ShaharNaveh commented Jun 27, 2026
View reviewed changes
Comment thread Lib/test/test_future_stmt/test_future.py
exec("from __future__ import unicode_literals; x = ''", {}, scope)
self.assertIsInstance(scope["x"], str)

@unittest.expectedFailure # TODO: RUSTPYTHON; barry_as_FLUFL (<> operator) not supported

@ShaharNaveh ShaharNaveh Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that adding --dont-add-python-opts makes this pass 😟

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@doma17 doma17 Jul 1, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that does seem a bit suspicious. Any idea why?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ShaharNaveh ShaharNaveh Jul 1, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I've now debugged it, it seems that it's actually because of the FORCE_COLOR=1 environment variable 🤦‍♂️

IMO the fix should be done over CPython side to add @support.force_not_colorized on the test method

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ShaharNaveh ShaharNaveh Jul 1, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python/cpython#152749 contains a bit more more information about why

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

doma17 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@ShaharNaveh
ShaharNaveh marked this pull request as ready for review June 27, 2026 11:00
@ShaharNaveh
Merge remote-tracking branch 'upstream/main' into ci-cpython-like-args
674bfa3
@ShaharNaveh ShaharNaveh mentioned this pull request Jul 1, 2026
Use @force_not_colorized in test_syntactical_future_repl python/cpython#152749
Merged
@ShaharNaveh
ShaharNaveh requested a review from youknowone July 2, 2026 11:13
youknowone
youknowone approved these changes Jul 4, 2026
View reviewed changes
Comment thread .github/workflows/ci.yaml
- os: macos-latest
extra_test_args:
- '-u all'
- '--timeout 600'

@youknowone youknowone Jul 4, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if everything requires same args, dose it still need to be extra_test_args?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ShaharNaveh ShaharNaveh Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's repeated multiple times, I think this is the easiest way to manage it for now.

we repeat this arg in 2/3 times that we run the test suite. I can "hardcode" it in all other places instead, not a big deal

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@youknowone youknowone Jul 6, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra_test_args changes the test name. then merge queue settings also must be updated.
that's not a big deal when it occasionally changes. but it happens more than common these days. So i wish that's still an argument, but not being a part of job title.

also it makes the CI job name unnecessary longer, not a big deal itself too though.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ShaharNaveh ShaharNaveh Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra_test_args changes the test name. then merge queue settings also must be updated. that's not a big deal when it occasionally changes. but it happens more than common these days. So i wish that's still an argument, but not being a part of job title.

also it makes the CI job name unnecessary longer, not a big deal itself too though.

CPython uses an external file to manage their CI commands, we could do the same using just for example.

bevy wrote their own tool for that, we could do something similar.

I'm in favor of starting with the just approach (or similar), if you think it's a good idea.

There's another option which I'm less of a fan of, but still worth mentioning. cargo alias can also do that, but that's adding another layer without the full features that justgives us, i.e. reuse of variables. (we can't define a variable called common_args and use it everywhere).

lmk if that makes sense to you and/or worth implementing

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@youknowone youknowone Jul 6, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either way i am good as long as contributors still can copy the same CI command from web CI view to their terminal

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ShaharNaveh reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@youknowone
youknowone merged commit de387fc into RustPython:main Jul 6, 2026
27 checks passed

Uh oh!

There was an error while loading. Please reload this page.

youknowone pushed a commit that referenced this pull request Sep 16, 2026
@ShaharNaveh
Set python args to be like CPython in CI (#8181)
0664e38
* Set python args to be like CPython in CI (ish)

* Use `--slow-ci`

* `--dont-add-python-opts`

* unmark passing test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@youknowone youknowone youknowone approved these changes
@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments
+2 more reviewers
@github-advanced-security github-advanced-security[bot] github-advanced-security[bot] left review comments
@doma17 doma17 doma17 left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@ShaharNaveh @youknowone @github-advanced-security @doma17
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.