GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||
GeezerVison is showing github.com — Run the pyperformance suite on a CodSpeed Macro runner by moreal · Pull Request #9623 · RustPython/RustPython Conversation
|
| Commit | Review? | Content |
|---|---|---|
benches: add a pyperf shim … |
yes | shim + its unittest file |
scripts: add vendor-pyperformance.py |
yes | vendoring script |
benches: vendor pyperformance 1.14.0 benchmarks |
no (generated, ~13k lines) | verbatim upstream files; verified by vendor-pyperformance.py --check in CI |
Run the pyperformance suite under CodSpeed … |
yes | Criterion target, tests, workflow, docs, deletions |
cargo bench --bench pyperformance -- --test (65/65), cargo test --test pyperformance, the shim tests under CPython and RustPython, vendor-pyperformance.py --check, actionlint, shellcheck.
Written with Claude Code (claude-fable-5-1); every commit carries an Assisted-by trailer. The design decisions (Macro runner, walltime mode, running upstream pyperformance unmodified, build on a hosted runner and run only the executables on codspeed-macro, which benchmarks to include) were made by the author in dialogue; the Rust harness, the pyperf shim, the vendoring script, the workflow, the documentation and this description were drafted by the AI under that direction. The benchmark selection runs and the local verification listed above were executed on the author's macOS machine and reviewed by the author.
The codspeed-macro runner must be enabled for this repository, or the run job waits for a runner. Benchmark names change to pyperformance/<name>, so the CodSpeed history starts fresh.
🤖 Generated with Claude Code
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews: auto_review: drafts: true
Tip 🎯 Customize your risk assessments
✅ Automatic approvals, with your rules
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Merging this PR will not alter performance
Comparing Footnotes
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…cess The upstream pyperformance run_benchmark.py files will be vendored unmodified under benches/pyperformance/ and executed by a Criterion harness inside a RustPython interpreter. The one thing they need that is not stdlib is `pyperf`; this module stands in for it. Each bench_* method runs the benchmark body exactly once and does no timing: repetition and measurement belong to Criterion, which runs the whole script per iteration. parse_args goes through real argparse so benchmark variants (extra_opts placed in sys.argv) take the upstream code path. Only the API surface the vendored benchmarks use exists; anything else raises AttributeError instead of silently measuring nothing. Tests run under both CPython and RustPython with unittest. Assisted-by: Claude Code:claude-fable-5-1
Copies a fixed list of bm_* directories from a pinned pyperformance release tag into benches/pyperformance/, verbatim. The release archive is checked against a SHA-256 hash before extraction, unlisted bm_* directories already in the tree are removed, and a few files that are listed but unusable here (the asyncio_tcp SSL variant and its certificates) are dropped. `--check` compares the tree with the archive without writing and exits non-zero on any difference; CI runs it so the vendored copy cannot drift from upstream. Assisted-by: Claude Code:claude-fable-5-1
Generated by `python3 scripts/vendor-pyperformance.py`; nothing in this commit is hand-written. 43 bm_* directories (65 benchmarks counting variants) are copied unmodified from the pyperformance 1.14.0 release archive, including their data files. Assisted-by: Claude Code:claude-fable-5-1
f6e4f2e to
ecb22c3
Compare
CodSpeed now measures wall time on a CodSpeed Macro runner instead of simulated CPU on a hosted runner, and what it measures is the vendored upstream pyperformance programs, unmodified. benches/pyperformance.rs is a new Criterion target that discovers the benchmarks from pyperformance's own [tool.pyperformance] metadata, runs each run_benchmark.py as __main__ in its own interpreter with the variant's extra_opts in sys.argv, and does not link pyo3, so the built executable can be moved between machines. tests/pyperformance.rs covers discovery and runs two benchmarks end to end; it replaces tests/benchmark_scripts.rs, which exercised the hand-ported files. The 13 hand-ported pyperformance workloads in benches/benchmarks/ go away in favour of the originals. Every one of the 73 upstream directories is accounted for in benches/pyperformance/README.md: third-party dependencies, start-up benchmarks, two that fail under RustPython (decimal_factorial, sqlite_synth), one that needs child interpreters (concurrent_imap) and three that take 13 to 46 seconds per iteration (bpe_tokeniser, decimal_pi, asyncio_tcp_ssl) are excluded. The workflow is split in two: `build` compiles the benchmark executables on ubuntu-latest with `cargo codspeed build --measurement-mode walltime` and uploads target/codspeed/walltime/; `run` on the codspeed-macro runner downloads them and runs `cargo codspeed run`, which in walltime mode also aggregates the results the compat layer writes. cargo-codspeed is installed from its released musl executable (v5.0.3, matching codspeed-criterion-compat 5.x) by .github/scripts/install-cargo-codspeed.sh. scripts/codspeed-environment.py and its guard steps are removed: the Macro runner is fixed hardware. The execution and microbenchmarks targets stay available to `cargo bench` but leave CodSpeed, since they link libpython through pyo3 for the CPython comparison. The CI lint job gains a `vendor-pyperformance.py --check` step. Assisted-by: Claude Code:claude-fable-5-1
e3b51f2 to
46683eb
Compare
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.