GeezerVison is showing github.com — Integrate remaining Python 3.15 final runtime and libraries by youknowdot · Pull Request #8954 · RustPython/RustPython
Integrate remaining Python 3.15 final runtime and libraries
#8954
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement . We’ll occasionally send you account related emails.
Already on GitHub?
Sign in
to your account
Show file tree
Hide file tree
Changes from all commits
Commits
File filter
Filter by extension
.def
(1)
.json
(2)
.lock
(1)
.md
(3)
.orig
(1)
.patch
(10)
.py
(639)
.rs
(56)
.snap
(808)
.toml
(5)
.txt
(3)
.yaml
(4)
.yml
(1)
No extension
(2)
dotfile
(1)
All 15 file types selected
Only manifest files
Viewed files
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
The table of contents is too big for display.
Diff view
Hide whitespace
Diff view
Hide whitespace
Uh oh!
There was an error while loading. Please reload this page.
There are no files selected for viewing
Original file line number
Diff line number
Diff line change
Expand Up
@@ -211,6 +211,7 @@ prec
preds
preinitialized
pybuilddir
pycfunction
pycore
PYGC
pyinner
Expand Down
Expand Up
@@ -311,3 +312,4 @@ xibufferview
xstat
XXPRIME
ENOTCAPABLE
unmarshalled
Original file line number
Diff line number
Diff line change
Expand Up
@@ -306,3 +306,4 @@ xmlcharrefreplace
xoptions
xopts
yieldfrom
issubnormal
Original file line number
Diff line number
Diff line change
Expand Up
@@ -41,3 +41,5 @@ rustpythonw
struc
unscanned
zelf
embedders
nocapture
Original file line number
Diff line number
Diff line change
Expand Up
@@ -584,30 +584,57 @@
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
- name: Verify the CPython reference environment
run: python -c "import bz2, ctypes, lzma, pathlib, platform, sqlite3, ssl; expected = pathlib.Path('.python-version').read_text().strip(); actual = platform.python_version(); assert actual == expected, (actual, expected)"
- name: Install macOS dependencies
uses: ./.github/actions/install-macos-deps
with:
openssl: true
# Keep features in sync with update-caches.yml CARGO_ARGS.
- name: build rustpython
run: cargo build --locked --release --verbose --features=threading,jit ${{ env.CARGO_ARGS }}
- name: run snippets
run: python -m pip install -r requirements.txt && pytest -v
working-directory: ./extra_tests
- name: Detect available cores
id: cores
shell: bash
run: |
cores=$(python -c 'print(__import__("os").process_cpu_count())')
echo "cores=${cores}" >> "$GITHUB_OUTPUT"
- name: Run CPython tests
- name: Build production RustPython
id: build_default
run: cargo build --locked --release --verbose --features=threading,jit ${{ env.CARGO_ARGS }}
- name: Record Windows timestamp compatibility
if: runner.os == 'Windows'
shell: bash
run: |
python - <<'PYTIME'
import json, subprocess, sys
code = """
import datetime, json, time
rows = []
for timestamp in (-1, -2147483649, -11644473600, -11644473601, 2**40):
for name in ('localtime', 'gmtime'):
try:
value = getattr(time, name)(timestamp)
result = [list(value), value.tm_zone, value.tm_gmtoff]
except Exception as error:
result = [type(error).__name__, str(error), getattr(error, 'errno', None)]
rows.append([timestamp, name, result])
print(json.dumps(rows))
"""
for executable in (sys.executable, 'target/release/rustpython.exe'):
result = subprocess.check_output([executable, '-c', code], text=True)
print(json.dumps({'executable': executable, 'timestamps': json.loads(result)}))
PYTIME
- name: run snippets
run: python -m pip install -r requirements.txt && python -m pytest -v
working-directory: ./extra_tests
- name: Run CPython tests (production)
run: |
# shellcheck disable=SC2086
target/release/rustpython -u -m test --slow-ci -j "$CORES" $EXTRA_TEST_ARGS $SKIPS
target/release/rustpython -u -m test --slow-ci -j "$CORES" $EXTRA_TEST_ARGS $SKIPS -x test_frame test_inspect test_monitoring
timeout-minutes: ${{ matrix.timeout }}
shell: bash
env:
Expand Down
Expand Up
@@ -680,6 +707,28 @@
shell: bash
run: python -I scripts/whats_left.py --signature --rustpython target/release/rustpython
# Run all production checks before rebuilding the same binary with C API.
# Keep these features in sync with update-caches.yml CARGO_ARGS.
- name: Build RustPython with C API
id: build_capi
if: ${{ !cancelled() && steps.build_default.outcome == 'success' }}
run: cargo build --locked --release --verbose --features=threading,jit,capi ${{ env.CARGO_ARGS }}
env:
PYO3_CONFIG_FILE: ${{ github.workspace }}/crates/capi/pyo3-rustpython.config
PYO3_NO_PYTHON: "1"
# A failed production check must not suppress this cohort or become green.
- name: Run CPython tests (C API)
if: ${{ !cancelled() && steps.build_capi.outcome == 'success' }}
run: |
target/release/rustpython -u -m test --slow-ci -j "$CORES" $EXTRA_TEST_ARGS test_frame test_inspect test_monitoring
Check warning
on line 724
in .github/workflows/ci.yaml
View workflow job for this annotation
GitHub Actions / Lint
[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:1:60: Double quote to prevent globbing and word splitting [shellcheck]
Raw Output:
i:.github/workflows/ci.yaml:724:70: shellcheck reported issue in this script: SC2086:info:1:60: Double quote to prevent globbing and word splitting [shellcheck]
timeout-minutes: ${{ matrix.timeout }}
shell: bash
env:
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
CORES: ${{ steps.cores.outputs.cores }}
EXTRA_TEST_ARGS: ${{ join(matrix.extra_test_args, ' ') }}
clippy:
# Named after the matrix entry rather than left to be named for it: a
# generated name lists every value in the entry, so adding or removing one
Expand Down
Expand Up
@@ -734,7 +783,12 @@
continue-on-error: true
- name: Clippy
run: cargo clippy --locked --keep-going ${{ env.CARGO_ARGS_MOST_FEATS }} --workspace --all-targets --bins --examples --tests ${{ env.WORKSPACE_EXCLUDES }} -- -Dwarnings
run: cargo clippy --locked --keep-going ${{ env.CARGO_ARGS_MOST_FEATS }} --workspace --exclude rustpython-capi --all-targets --bins --examples --tests ${{ env.WORKSPACE_EXCLUDES }} -- -Dwarnings
# Load the C-API crate's RustPython-specific PyO3 configuration.
- name: Clippy C API
working-directory: crates/capi
run: cargo clippy --locked --all-targets -- -Dwarnings
# macOS/Windows clippy uses the same OS-surface filter as cargo check.
# Force with the run:cross-check label. Check names stay clippy (<os>).
Expand Down
Expand Up
@@ -797,6 +851,9 @@
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
- name: Verify the CPython reference environment
run: python -c "import bz2, ctypes, lzma, pathlib, platform, sqlite3, ssl; expected = pathlib.Path('.python-version').read_text().strip(); actual = platform.python_version(); assert actual == expected, (actual, expected)"
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
Expand Down
Expand Up
@@ -1058,10 +1115,13 @@
builtin_posixshmem
builtin_thread
cli_int_max_str_digits
import_lazy_native
protocol_index_argument
recursion
stdlib_ast
stdlib_asyncio
stdlib_asyncio_reinitialization_315
stdlib_bytearray_take_bytes_threads_315
stdlib_contextvars
stdlib_ctypes
stdlib_ctypes_byvalue
Expand All
@@ -1070,17 +1130,20 @@
stdlib_genericalias
stdlib_io
stdlib_io_blocking_buffer
stdlib_lock_precedence_315
stdlib_mmap
stdlib_multiprocessing_condition
stdlib_multiprocessing_fork_sharer
stdlib_multiprocessing_sharedctypes
stdlib_os
stdlib_pwd
stdlib_queue_signal
stdlib_rlock_constructor_315
stdlib_select
stdlib_socket
stdlib_sqlite
stdlib_ssl
stdlib_ssl_introspection
stdlib_ssl_short_recv
stdlib_subinterpreters
stdlib_subprocess
Expand All
@@ -1096,6 +1159,8 @@
stdlib_types
stdlib_typing
stdlib_urllib_https_misaligned_recv
stdlib_venv_cycles
stdlib_weakref_gc_threads_315
stdlib_zoneinfo_cache
syntax_async
syntax_async_comprehension
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -17,6 +17,32 @@ env:
CARGO_ARGS: --no-default-features --features stdlib,importlib,stdio,encodings,ssl-rustls-aws-lc,jit,host_env,doc,ucd_3_2_0
jobs:
manager-coverage-regression:
name: Verify manager XML-RPC under coverage
runs-on: ubuntu-latest
permissions:
contents: read
# Leave room for the instrumented build and the normal 600-second test limit.
timeout-minutes: 30
if: ${{ github.repository == 'RustPython/RustPython' || github.event_name != 'schedule' }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: taiki-e/install-action@8cca0918f8471c33515202b73f56ab27f2bed283 # v2.87.29
with:
tool: cargo-llvm-cov
# Keep this strict check independent of the full coverage collection below.
- name: Run the canonical manager XML-RPC regression
run: >-
cargo llvm-cov --no-report run --locked --
-m test -v -u all --timeout 600 --match '*test_remote'
test_multiprocessing_forkserver.test_manager
# codecov collects code coverage data from the rust tests, python snippets and python test suite.
# This is done using cargo-llvm-cov, which is a wrapper around llvm-cov.
codecov:
Expand All
@@ -41,15 +67,32 @@ jobs:
- run: sudo apt-get update && sudo apt-get -y install lcov
- name: Clean previous coverage artifacts
run: cargo llvm-cov clean --workspace
- name: Run cargo-llvm-cov with Rust tests.
run: cargo llvm-cov --no-report --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher --exclude rustpython-capi --verbose --no-default-features --features stdlib,importlib,stdio,encodings,ssl-rustls-aws-lc,jit,host_env,doc,ucd_3_2_0
- name: Run cargo-llvm-cov with Python snippets.
run: python scripts/cargo-llvm-cov.py
continue-on-error: true
# Keep the C API cohort and its build features in sync with ci.yaml.
# Retain both feature sets' instrumented artifacts for the final report.
- name: Run cargo-llvm-cov with C API Python tests
run: >-
cargo llvm-cov --no-report run --locked --features=threading,jit,capi
--no-default-features --features stdlib,importlib,stdio,encodings,sqlite,ssl-rustls-aws-lc,host_env,doc,ucd_3_2_0
-- -u -m test --slow-ci -v -u all --timeout 600 --dont-add-python-opts
test_frame test_inspect test_monitoring
env:
PYO3_CONFIG_FILE: ${{ github.workspace }}/crates/capi/pyo3-rustpython.config
PYO3_NO_PYTHON: "1"
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
RUST_BACKTRACE: full
- name: Run cargo-llvm-cov with Python test suite.
run: cargo llvm-cov --no-report run -- -m test -u all --slowest --fail-env-changed
run: cargo llvm-cov --no-report run -- -m test -u all --slowest --fail-env-changed -x test_frame test_inspect test_monitoring
continue-on-error: true
- name: Prepare code coverage data
Expand Down
Expand Up
@@ -110,6 +153,9 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
- name: Verify the CPython reference environment
run: python -c "import bz2, ctypes, lzma, pathlib, platform, sqlite3, ssl; expected = pathlib.Path('.python-version').read_text().strip(); actual = platform.python_version(); assert actual == expected, (actual, expected)"
- name: build rustpython
run: cargo build --release --verbose --features sqlite
Expand Down
Expand Up
@@ -170,6 +216,12 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
- name: Verify the CPython reference environment
run: python -c "import bz2, ctypes, lzma, pathlib, platform, sqlite3, ssl; expected = pathlib.Path('.python-version').read_text().strip(); actual = platform.python_version(); assert actual == expected, (actual, expected)"
- name: Make the reference libpython available to benchmark executables
run: echo "LD_LIBRARY_PATH=$(python -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))')${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" >> "$GITHUB_ENV"
- run: cargo install cargo-criterion
- name: build benchmarks
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -111,43 +111,80 @@ jobs:
- name: Run deps check
if: steps.changed-files.outputs.modules != ''
id: deps-check
run: |
# Run deps for all modules at once
echo "deps_output<<EOF" >> "$GITHUB_OUTPUT"
output=$(python scripts/update_lib deps "${MODULES}" --depth 2 2>&1 || true)
output=$(python -c '
import sys
output = sys.stdin.read()
limit = 60000
encoded = output.encode()
if len(encoded) > limit:
output = encoded[:limit].decode("utf-8", errors="ignore")
output += "\n\n[Dependency report truncated to fit in a GitHub comment.]"
print(output, end="")
' <<< "$output")
echo "$output" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
python scripts/update_lib deps "${MODULES}" --depth 2 > "$RUNNER_TEMP/lib-deps-report.md" 2>&1 || true
env:
MODULES: ${{ steps.changed-files.outputs.modules }}
- name: Prepare dependency report
if: steps.changed-files.outputs.modules != ''
id: deps-comment
shell: python
run: |
from os import environ
from pathlib import Path
report_path = Path(environ["RUNNER_TEMP"]) / "lib-deps-report.md"
report = report_path.read_text(encoding="utf-8")
if report:
heading = "## 📦 Library Dependencies\n\n"
body = (
heading
+ "The following Lib/ modules were modified. Here are their dependencies:\n\n"
+ report
+ "\n\n**Legend:**\n"
+ "- `[+]` path exists in CPython\n"
+ "- `[x]` up-to-date, `[ ]` outdated\n"
)
report_path.write_text(body, encoding="utf-8")
body_size = len(body.encode("utf-8"))
run_url = (
f"{environ['GITHUB_SERVER_URL']}/{environ['GITHUB_REPOSITORY']}"
f"/actions/runs/{environ['GITHUB_RUN_ID']}"
)
# Job summaries have a 1 MiB limit; keep larger reports as artifacts.
needs_artifact = body_size > 1024 * 1024
summary = body
if needs_artifact:
summary = (
heading
+ "The dependency report exceeds the job summary limit. "
+ "Download the full report from this run's `lib-deps-report` artifact.\n"
)
with open(environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as fd:
fd.write(summary)
# Leave room for the action's marker, counting UTF-8 bytes, not characters.
comment = body
if body_size >= 60000:
comment = (
heading
+ "The dependency report is too large for a pull request comment. "
+ f"[View the full report in the workflow run]({run_url}#summary).\n"
)
comment_path = Path(environ["RUNNER_TEMP"]) / "lib-deps-comment.md"
comment_path.write_text(comment, encoding="utf-8")
with open(environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as fd:
fd.write("has_report=true\n")
fd.write(f"needs_artifact={str(needs_artifact).lower()}\n")
- name: Upload oversized dependency report
if: steps.deps-comment.outputs.needs_artifact == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: lib-deps-report
path: ${{ runner.temp }}/lib-deps-report.md
if-no-files-found: error
- name: Post comment
if: steps.deps-check .outputs.deps_output != ' '
if: steps.deps-comment .outputs.has_report == 'true '
uses: marocchino/sticky-pull-request-comment@5770ad5eb8f42dd2c4f34da00c94c5381e49af88 # v3.0.5
with:
header: lib-deps-check
number: ${{ github.event.pull_request.number }}
message: |
## 📦 Library Dependencies
The following Lib/ modules were modified. Here are their dependencies:
${{ steps.deps-check.outputs.deps_output }}
**Legend:**
- `[+]` path exists in CPython
- `[x]` up-to-date, `[ ]` outdated
path: ${{ runner.temp }}/lib-deps-comment.md
- name: Remove comment if no Lib changes
if: steps.changed-files.outputs.modules == ''
Expand Down
Loading
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Loading
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.