GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — ci: add pyperformance benchmarking scripts and workflow (#8691) · RustPython/RustPython@44dcd1d

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
  • .github
    • workflows
      • pyperformance-comment.yaml
      • pyperformance.yaml
    • zizmor.yml
  • .gitignore
  • scripts/pyperformance
    • README.md
    • compare.py
    • pr_diff.py
    • run_all.py
    • stub_psutil
      • pyproject.toml
      • src/psutil
        • __init__.py
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎.github/workflows/pyperformance-comment.yaml‎

Copy file name to clipboard
+81Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
on:
2+
workflow_run:
3+
workflows: [pyperformance]
4+
types: [completed]
5+
6+
name: pyperformance comment
7+
8+
# Same split as `codspeed-comment.yaml`: a `pull_request` run from a fork gets
9+
# a read-only token however it is configured, so the comparison is computed
10+
# there and handed over as an artifact, and this workflow -- which
11+
# `workflow_run` always runs in the base repo's context -- is the one trusted
12+
# with a token that may write the comment.
13+
permissions:
14+
contents: read
15+
actions: read # needed by download-artifact to fetch another run's artifact
16+
pull-requests: write
17+
18+
jobs:
19+
comment:
20+
if: ${{ github.event.workflow_run.conclusion == 'success' }}
21+
runs-on: ubuntu-latest
22+
steps:
23+
- name: Download the comparison produced by the pyperformance workflow
24+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
25+
with:
26+
name: pyperformance-diff
27+
run-id: ${{ github.event.workflow_run.id }}
28+
github-token: ${{ github.token }}
29+
continue-on-error: true # absent when the job was skipped or dispatched
30+
31+
- name: Post or update the PR comment
32+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
33+
with:
34+
script: |
35+
const fs = require('fs');
36+
if (!fs.existsSync('diff.md')) {
37+
core.info('No diff.md in this run; nothing to comment.');
38+
return;
39+
}
40+
41+
const marker = '<!-- pyperformance-local-diff -->';
42+
const run = context.payload.workflow_run;
43+
let prNumber = (run.pull_requests || []).map((pr) => pr.number)[0];
44+
if (!prNumber) {
45+
const { data } = await github.rest.repos.listPullRequestsAssociatedWithCommit({
46+
owner: context.repo.owner,
47+
repo: context.repo.repo,
48+
commit_sha: run.head_sha,
49+
});
50+
prNumber = data.find((pr) => pr.state === 'open')?.number;
51+
}
52+
if (!prNumber) {
53+
core.info(`No open pull request for ${run.head_sha}; nothing to comment.`);
54+
return;
55+
}
56+
57+
const raw = fs.readFileSync('diff.md', 'utf8').slice(0, 60000);
58+
const body = `${marker}\n\n${raw.replace(/<!--/g, '<!-​-')}`;
59+
60+
const comments = await github.paginate(github.rest.issues.listComments, {
61+
owner: context.repo.owner,
62+
repo: context.repo.repo,
63+
issue_number: prNumber,
64+
});
65+
const existing = comments.find((c) => c.body && c.body.includes(marker));
66+
67+
if (existing) {
68+
await github.rest.issues.updateComment({
69+
owner: context.repo.owner,
70+
repo: context.repo.repo,
71+
comment_id: existing.id,
72+
body,
73+
});
74+
} else {
75+
await github.rest.issues.createComment({
76+
owner: context.repo.owner,
77+
repo: context.repo.repo,
78+
issue_number: prNumber,
79+
body,
80+
});
81+
}
Collapse file

‎.github/workflows/pyperformance.yaml‎

Copy file name to clipboard
+220Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
on:
2+
pull_request:
3+
types: [labeled, opened, synchronize, reopened]
4+
workflow_dispatch:
5+
6+
name: pyperformance
7+
8+
permissions:
9+
contents: read
10+
11+
concurrency:
12+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
13+
cancel-in-progress: true
14+
15+
env:
16+
CARGO_INCREMENTAL: 0
17+
CARGO_TERM_COLOR: always
18+
CI: true
19+
# Pure-Python benchmarks only: anything whose workload needs a real C
20+
# extension cannot install under RustPython at all, and the async/SSL ones
21+
# are far too slow here to be worth a runner's time. Keep this list short --
22+
# every entry is paid for three times (CPython, base, head) in one job.
23+
BENCHMARKS: >-
24+
chaos,comprehensions,coroutines,deepcopy,deltablue,fannkuch,float,
25+
generators,go,hexiom,json_dumps,json_loads,logging,mdp,nbody,nqueens,
26+
pickle_pure_python,raytrace,regex_compile,regex_v8,richards,scimark,
27+
spectral_norm,telco,typing_runtime_protocols,unpack_sequence,
28+
unpickle_pure_python,xml_etree
29+
# A benchmark that needs longer than this on the base commit is one this
30+
# comparison cannot afford to wait for; it shows up as a failure in the
31+
# table, which is itself the interesting signal.
32+
BENCH_TIMEOUT: 300
33+
34+
jobs:
35+
compare:
36+
# `skip:ci` mirrors the other workflows; `run:pyperformance` opts a PR in,
37+
# because the job costs well over an hour and most PRs do not need it.
38+
if: >-
39+
github.event_name == 'workflow_dispatch' ||
40+
(!contains(github.event.pull_request.labels.*.name, 'skip:ci') &&
41+
contains(github.event.pull_request.labels.*.name, 'run:pyperformance'))
42+
name: base vs. head vs. CPython
43+
runs-on: ubuntu-latest
44+
timeout-minutes: 300
45+
permissions:
46+
contents: read
47+
# Honoured for a pull request opened from this repo, which is then
48+
# commented on directly by the last step; a fork's run is handed
49+
# read-only whatever is asked for here and goes the artifact route.
50+
pull-requests: write
51+
steps:
52+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
53+
with:
54+
ref: ${{ github.event.pull_request.head.sha }}
55+
fetch-depth: 0
56+
persist-credentials: false
57+
58+
- uses: dtolnay/rust-toolchain@stable
59+
60+
# Both the third interpreter under test and the host pyperformance
61+
# itself runs on (it only runs under a real CPython; RustPython is just
62+
# the `--python` target it drives).
63+
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
64+
id: cpython
65+
with:
66+
python-version: "3.14"
67+
68+
- name: Restore cache
69+
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
70+
with:
71+
path: |
72+
~/.cargo/registry/index/
73+
~/.cargo/registry/cache/
74+
~/.cargo/git/db/
75+
target/
76+
key: pyperformance-${{ hashFiles('**/Cargo.lock') }}
77+
restore-keys: pyperformance-
78+
continue-on-error: true
79+
80+
# The base commit may predate these scripts, and `git checkout <base>`
81+
# below would then delete them; run them from a copy outside the tree.
82+
- name: Copy the benchmark scripts out of the checkout
83+
run: cp -r scripts/pyperformance "${RUNNER_TEMP}/pyperformance-scripts"
84+
85+
# Each build is copied aside with the stdlib of its own commit, so the
86+
# second does not overwrite the first and a PR that touches `Lib/` is
87+
# still compared against the base commit's `Lib/`. A binary moved away
88+
# from its checkout finds the stdlib only through RUSTPYTHONPATH, which
89+
# `run_all.py` forwards through pyperf into the venv it builds.
90+
- name: Build head
91+
run: |
92+
cargo build --release --locked
93+
mkdir -p "${RUNNER_TEMP}/head"
94+
cp target/release/rustpython "${RUNNER_TEMP}/head/rustpython"
95+
cp -r Lib "${RUNNER_TEMP}/head/Lib"
96+
97+
- name: Build base
98+
continue-on-error: true # a base that cannot build leaves head vs. CPython
99+
env:
100+
BASE_SHA: ${{ github.event.pull_request.base.sha }}
101+
run: |
102+
git checkout --quiet "${BASE_SHA}" --
103+
cargo build --release --locked
104+
mkdir -p "${RUNNER_TEMP}/base"
105+
cp target/release/rustpython "${RUNNER_TEMP}/base/rustpython"
106+
cp -r Lib "${RUNNER_TEMP}/base/Lib"
107+
108+
- name: Restore head checkout
109+
env:
110+
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
111+
run: git checkout --quiet "${HEAD_SHA}" --
112+
113+
- name: Save cache
114+
if: always()
115+
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
116+
with:
117+
path: |
118+
~/.cargo/registry/index/
119+
~/.cargo/registry/cache/
120+
~/.cargo/git/db/
121+
target/
122+
key: pyperformance-${{ hashFiles('**/Cargo.lock') }}
123+
continue-on-error: true
124+
125+
# The three runs are deliberately adjacent, with no build between them,
126+
# so nothing but the interpreter differs across the comparison.
127+
- name: Measure CPython
128+
env:
129+
CPYTHON: ${{ steps.cpython.outputs.python-path }}
130+
run: |
131+
python3 "${RUNNER_TEMP}/pyperformance-scripts/run_all.py" \
132+
--python "${CPYTHON}" \
133+
--label cpython --no-psutil-stub \
134+
--benchmarks "${BENCHMARKS//[$'\n' ]/}" \
135+
--timeout "${BENCH_TIMEOUT}" --out "${RUNNER_TEMP}/pyperf-results"
136+
137+
- name: Measure base
138+
continue-on-error: true # as above: the table falls back to head alone
139+
env:
140+
RUSTPYTHONPATH: ${{ runner.temp }}/base/Lib
141+
run: |
142+
python3 "${RUNNER_TEMP}/pyperformance-scripts/run_all.py" \
143+
--python "${RUNNER_TEMP}/base/rustpython" \
144+
--label base \
145+
--benchmarks "${BENCHMARKS//[$'\n' ]/}" \
146+
--timeout "${BENCH_TIMEOUT}" --out "${RUNNER_TEMP}/pyperf-results"
147+
148+
- name: Measure head
149+
env:
150+
RUSTPYTHONPATH: ${{ runner.temp }}/head/Lib
151+
run: |
152+
python3 "${RUNNER_TEMP}/pyperformance-scripts/run_all.py" \
153+
--python "${RUNNER_TEMP}/head/rustpython" \
154+
--label head \
155+
--benchmarks "${BENCHMARKS//[$'\n' ]/}" \
156+
--timeout "${BENCH_TIMEOUT}" --out "${RUNNER_TEMP}/pyperf-results"
157+
158+
- name: Render the comparison
159+
env:
160+
CPYTHON_VERSION: ${{ steps.cpython.outputs.python-version }}
161+
PR_NUMBER: ${{ github.event.pull_request.number }}
162+
run: |
163+
python3 "${RUNNER_TEMP}/pyperformance-scripts/pr_diff.py" \
164+
--results-dir "${RUNNER_TEMP}/pyperf-results" \
165+
--cpython-version "CPython ${CPYTHON_VERSION}" \
166+
--md diff.md --json diff.json
167+
echo "${PR_NUMBER}" > pr_number.txt
168+
169+
# A `pull_request` run triggered from a fork carries a read-only token no
170+
# matter what this workflow grants, so the comment is posted from
171+
# `pyperformance-comment.yaml`, which `workflow_run` runs in the base
172+
# repo's context. This artifact is the hand-off.
173+
- name: Upload the comparison
174+
if: always()
175+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
176+
with:
177+
name: pyperformance-diff
178+
path: |
179+
diff.md
180+
diff.json
181+
pr_number.txt
182+
retention-days: 14
183+
if-no-files-found: ignore
184+
185+
# A pull request opened from this repo can be commented on from here,
186+
# which puts the table on the PR in the same run instead of waiting for
187+
# the companion workflow. Forks fall through to that workflow instead.
188+
- name: Post or update the PR comment
189+
if: >-
190+
github.event_name == 'pull_request' &&
191+
github.event.pull_request.head.repo.full_name == github.repository
192+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
193+
with:
194+
script: |
195+
const fs = require('fs');
196+
const body = fs.readFileSync('diff.md', 'utf8');
197+
const marker = '<!-- pyperformance-local-diff -->';
198+
199+
const comments = await github.paginate(github.rest.issues.listComments, {
200+
owner: context.repo.owner,
201+
repo: context.repo.repo,
202+
issue_number: context.payload.pull_request.number,
203+
});
204+
const existing = comments.find((c) => c.body && c.body.includes(marker));
205+
206+
if (existing) {
207+
await github.rest.issues.updateComment({
208+
owner: context.repo.owner,
209+
repo: context.repo.repo,
210+
comment_id: existing.id,
211+
body,
212+
});
213+
} else {
214+
await github.rest.issues.createComment({
215+
owner: context.repo.owner,
216+
repo: context.repo.repo,
217+
issue_number: context.payload.pull_request.number,
218+
body,
219+
});
220+
}
Collapse file

‎.github/zizmor.yml‎

Copy file name to clipboardExpand all lines: .github/zizmor.yml
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ rules:
44
# pull_request_target is needed to label and assign PRs from forks with issues: write.
55
# The workflow does not check out or execute pull request code.
66
- oscca-pr.yml:3
7+
# workflow_run is needed to post a PR comment from a fork's run, which is
8+
# handed a read-only token however the triggering workflow is configured.
9+
# The workflow does not check out or execute pull request code: it reads
10+
# one artifact and writes a comment.
11+
- pyperformance-comment.yaml:1
712
excessive-permissions:
813
ignore:
914
# pull_request_target is needed to post PR comments with pull-requests: write.
Collapse file

‎.gitignore‎

Copy file name to clipboardExpand all lines: .gitignore
+5-1Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,4 +31,8 @@ Lib/test/data/*
3131
!Lib/test/data/README
3232
cpython/
3333
.claude/
34-
docs/superpowers/
34+
docs/superpowers/
35+
36+
scripts/pyperformance/results/
37+
scripts/pyperformance/stub_psutil/build/
38+
scripts/pyperformance/stub_psutil/src/psutil.egg-info/

0 commit comments

Comments
0 (0)
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.