GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Introduce autopatch_cargo · soucod/graalpython@c3cb073

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
  • .agents/skills/third-party-package-patches
    • SKILL.md
  • graalpython
    • com.oracle.graal.python.test/src/tests
      • test_autopatch_cargo.py
    • lib-graalpython
      • modules
        • autopatch_cargo.py
        • graalpy_pip_extensions.py
      • patches
        • README.md
        • crates
          • metadata.toml
          • pyo3-ffi-0.28.patch
          • pyo3-ffi-0.29.0.patch
  • mx.graalpython
    • mx_graalpython.py
    • verify_patches.py
  • scripts
    • get_pypi_source.py
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎.agents/skills/third-party-package-patches/SKILL.md‎

Copy file name to clipboardExpand all lines: .agents/skills/third-party-package-patches/SKILL.md
+18-2Lines changed: 18 additions & 2 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: third-party-package-patches
3-
description: Create or update GraalPy third-party package compatibility patches under graalpython/lib-graalpython/patches, including PyPI source preparation, rebasing existing patches, metadata.toml updates, license checks, version-range validation, and verify_patches.py validation.
3+
description: Create or update GraalPy third-party package and Rust crate compatibility patches under graalpython/lib-graalpython/patches, including PyPI source preparation, Cargo crate autopatching, rebasing existing patches, metadata.toml updates, license checks, version-range validation, and verify_patches.py validation.
44
---
55

66
# Third-Party Package Patches
@@ -11,6 +11,8 @@ Use this skill when creating or updating compatibility patches for packages inst
1111
- Source preparation: `scripts/get_pypi_source.py`
1212
- Patch metadata: `graalpython/lib-graalpython/patches/metadata.toml`
1313
- Patch directory: `graalpython/lib-graalpython/patches/`
14+
- Cargo autopatcher: `graalpython/lib-graalpython/modules/autopatch_cargo.py`
15+
- Crate patch metadata: `graalpython/lib-graalpython/patches/crates/metadata.toml`
1416
- Metadata verifier: `mx.graalpython/verify_patches.py`
1517

1618
## Workflow
@@ -20,7 +22,7 @@ Use this skill when creating or updating compatibility patches for packages inst
2022
```bash
2123
python scripts/get_pypi_source.py package==version
2224
```
23-
The script prints `Prepared source at: ...`. Use that directory as the working tree. It is already a temporary git repository with an initial commit, and it has already been processed by `graalpython/lib-graalpython/modules/autopatch_capi.py`.
25+
The script prints `Prepared source at: ...`. Use that directory as the working tree. It is already a temporary git repository with an initial commit, and it has already been processed by `autopatch_capi.py` and `autopatch_cargo.py`.
2426

2527
3. Inspect `graalpython/lib-graalpython/patches/metadata.toml` for existing `[[package.rules]]` entries.
2628
- If a matching patch exists for the requested version, apply it first.
@@ -74,6 +76,19 @@ python mx.graalpython/verify_patches.py graalpython/lib-graalpython/patches
7476

7577
12. If you were asked to build or test the patched package, you need to rebuild GraalPy with `mx python-jvm` to pick up the changes. Create a venv with `mx python -m venv venv_name` and use it for building and testing.
7678

79+
## Rust Crate Patches
80+
81+
Use `autopatch_cargo.py` when the incompatibility is in a transitive crates.io dependency rather than the Python package source. It finds matching crates in `Cargo.lock`, copies only those crates into the source tree, applies registered patches, and adds Cargo path overrides.
82+
83+
1. Add rules to `graalpython/lib-graalpython/patches/crates/metadata.toml`. Use the crate package name as the table key and specify `version`, `patch`, and `license` for every rule.
84+
2. Generate crate patch files from a clean checkout of the published crate. Patch paths must be relative to the crate root. Do not hand-edit generated patch files.
85+
3. Run the tool directly when validating a prepared source tree:
86+
```bash
87+
python graalpython/lib-graalpython/modules/autopatch_cargo.py /path/to/source
88+
```
89+
4. Test every crate release covered by the version range, then build representative Python packages that resolve those releases.
90+
5. Use the Python package rule's `autopatch = false` only when both `autopatch_capi` and `autopatch_cargo` must be disabled for that package version.
91+
7792
## Metadata Reference
7893
Rule keys accepted by the verifier are:
7994
- `version`
@@ -83,6 +98,7 @@ Rule keys accepted by the verifier are:
8398
- `dist-type`
8499
- `install-priority`
85100
- `note`
101+
- `autopatch`
86102

87103
Allowed `dist-type` values are `wheel` and `sdist`.
88104

Collapse file

‎graalpython/com.oracle.graal.python.test/src/tests/test_autopatch_cargo.py‎

Copy file name to clipboard
+242Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
# Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
2+
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
3+
#
4+
# The Universal Permissive License (UPL), Version 1.0
5+
#
6+
# Subject to the condition set forth below, permission is hereby granted to any
7+
# person obtaining a copy of this software, associated documentation and/or
8+
# data (collectively the "Software"), free of charge and under any and all
9+
# copyright rights in the Software, and any and all patent rights owned or
10+
# freely licensable by each licensor hereunder covering either (i) the
11+
# unmodified Software as contributed to or provided by such licensor, or (ii)
12+
# the Larger Works (as defined below), to deal in both
13+
#
14+
# (a) the Software, and
15+
#
16+
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
17+
# one is included with the Software each a "Larger Work" to which the Software
18+
# is contributed by such licensors),
19+
#
20+
# without restriction, including without limitation the rights to copy, create
21+
# derivative works of, display, perform, and distribute the Software and make,
22+
# use, sell, offer for sale, import, export, have made, and have sold the
23+
# Software and the Larger Work(s), and to sublicense the foregoing rights on
24+
# either these or other terms.
25+
#
26+
# This license is subject to the following condition:
27+
#
28+
# The above copyright notice and either this complete permission notice or at a
29+
# minimum a reference to the UPL must be included in all copies or substantial
30+
# portions of the Software.
31+
#
32+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
33+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
34+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
35+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
36+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
37+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
38+
# SOFTWARE.
39+
40+
import hashlib
41+
import json
42+
import os
43+
import shutil
44+
import subprocess
45+
import sys
46+
import tarfile
47+
import tempfile
48+
import textwrap
49+
import tomllib
50+
import unittest
51+
from pathlib import Path
52+
from unittest.mock import patch
53+
54+
55+
if sys.implementation.name == "graalpy":
56+
import autopatch_cargo
57+
58+
59+
class AutoPatchCargoTest(unittest.TestCase):
60+
def setUp(self):
61+
self.tempdir = Path(tempfile.mkdtemp())
62+
self.cargo_home = self.tempdir / "cargo-home"
63+
self.workspace = self.tempdir / "project"
64+
self.patch_dir = self.tempdir / "patches"
65+
self.workspace.mkdir()
66+
self.patch_dir.mkdir()
67+
self.env_patch = patch.dict(os.environ, {"CARGO_HOME": str(self.cargo_home)})
68+
self.env_patch.start()
69+
70+
def tearDown(self):
71+
self.env_patch.stop()
72+
shutil.rmtree(self.tempdir)
73+
74+
@staticmethod
75+
def write(path, contents):
76+
path.parent.mkdir(parents=True, exist_ok=True)
77+
path.write_text(textwrap.dedent(contents).lstrip())
78+
79+
def prepare_cached_crate(self, name="made-up-crate", version="1.2.3"):
80+
crate = self.cargo_home / "registry" / "src" / "made-up-index" / f"{name}-{version}"
81+
self.write(
82+
crate / "Cargo.toml",
83+
f"""
84+
[package]
85+
name = "{name}"
86+
version = "{version}"
87+
edition = "2021"
88+
""",
89+
)
90+
self.write(crate / "src" / "lib.rs", 'pub fn message() -> &\'static str { "unpatched" }\n')
91+
return crate
92+
93+
def prepare_repository(self, version=">=1,<2"):
94+
self.write(
95+
self.patch_dir / "metadata.toml",
96+
f"""
97+
[[made-up-crate.rules]]
98+
version = "{version}"
99+
patch = "made-up-crate.patch"
100+
license = "MIT"
101+
""",
102+
)
103+
self.write(
104+
self.patch_dir / "made-up-crate.patch",
105+
"""
106+
--- a/src/lib.rs
107+
+++ b/src/lib.rs
108+
@@ -1 +1 @@
109+
-pub fn message() -> &'static str { "unpatched" }
110+
+pub fn message() -> &'static str { "patched" }
111+
""",
112+
)
113+
return autopatch_cargo.DirectoryPatchRepository(self.patch_dir)
114+
115+
def prepare_workspace(self, checksum=None):
116+
if checksum is None:
117+
checksum = "0" * 64
118+
existing_patch = self.workspace / "existing-patch"
119+
self.write(
120+
existing_patch / "Cargo.toml",
121+
"""
122+
[package]
123+
name = "existing-patch"
124+
version = "1.0.0"
125+
edition = "2021"
126+
""",
127+
)
128+
self.write(existing_patch / "src" / "lib.rs", "")
129+
self.write(
130+
self.workspace / "Cargo.toml",
131+
"""
132+
[package]
133+
name = "made-up-project"
134+
version = "0.1.0"
135+
edition = "2021"
136+
137+
[dependencies]
138+
made-up-crate = "=1.2.3"
139+
existing-patch = "=1.0.0"
140+
141+
[patch.crates-io]
142+
existing-patch = { path = "existing-patch" }
143+
""",
144+
)
145+
self.write(self.workspace / "src" / "lib.rs", "pub use made_up_crate::message;\n")
146+
self.write(
147+
self.workspace / "Cargo.lock",
148+
"""
149+
version = 4
150+
151+
[[package]]
152+
name = "existing-patch"
153+
version = "1.0.0"
154+
155+
[[package]]
156+
name = "made-up-crate"
157+
version = "1.2.3"
158+
source = "registry+https://github.com/rust-lang/crates.io-index"
159+
checksum = "{checksum}"
160+
161+
[[package]]
162+
name = "made-up-project"
163+
version = "0.1.0"
164+
dependencies = [
165+
"existing-patch",
166+
"made-up-crate",
167+
]
168+
""".format(checksum=checksum),
169+
)
170+
171+
def test_patches_locked_crate_and_adds_cargo_override(self):
172+
cached_crate = self.prepare_cached_crate()
173+
repository = self.prepare_repository()
174+
self.prepare_workspace()
175+
176+
assert autopatch_cargo.auto_patch_tree(self.workspace, repository) == 1
177+
patched_crate = self.workspace / ".graalpy" / "crates" / "made-up-crate-1.2.3"
178+
assert '"patched"' in (patched_crate / "src" / "lib.rs").read_text()
179+
assert '"unpatched"' in (cached_crate / "src" / "lib.rs").read_text()
180+
181+
manifest = tomllib.loads((self.workspace / "Cargo.toml").read_text())
182+
cargo_patches = manifest["patch"]["crates-io"]
183+
assert cargo_patches["existing-patch"]["path"] == "existing-patch"
184+
override = cargo_patches["graalpy-made-up-crate-1_2_3"]
185+
assert override == {
186+
"package": "made-up-crate",
187+
"path": ".graalpy/crates/made-up-crate-1.2.3",
188+
}
189+
lock_packages = tomllib.loads((self.workspace / "Cargo.lock").read_text())["package"]
190+
locked_crate = next(package for package in lock_packages if package["name"] == "made-up-crate")
191+
assert "source" not in locked_crate
192+
assert "checksum" not in locked_crate
193+
194+
if cargo := shutil.which("cargo"):
195+
metadata = subprocess.run(
196+
[cargo, "metadata", "--format-version", "1", "--locked", "--offline"],
197+
cwd=self.workspace,
198+
check=True,
199+
stdout=subprocess.PIPE,
200+
text=True,
201+
)
202+
packages = json.loads(metadata.stdout)["packages"]
203+
crate = next(package for package in packages if package["name"] == "made-up-crate")
204+
assert "/.graalpy/crates/made-up-crate-1.2.3/Cargo.toml" in crate["manifest_path"]
205+
206+
assert autopatch_cargo.auto_patch_tree(self.workspace, repository) == 0
207+
208+
def test_downloads_and_verifies_uncached_crate(self):
209+
crate = self.tempdir / "archive-source" / "made-up-crate-1.2.3"
210+
self.write(
211+
crate / "Cargo.toml",
212+
"""
213+
[package]
214+
name = "made-up-crate"
215+
version = "1.2.3"
216+
edition = "2021"
217+
""",
218+
)
219+
self.write(crate / "src" / "lib.rs", 'pub fn message() -> &\'static str { "unpatched" }\n')
220+
archive = self.tempdir / "made-up-crate-1.2.3.crate"
221+
with tarfile.open(archive, "w:gz") as tar:
222+
tar.add(crate, arcname=crate.name)
223+
checksum = hashlib.sha256(archive.read_bytes()).hexdigest()
224+
repository = self.prepare_repository()
225+
self.prepare_workspace(checksum)
226+
227+
with open(archive, "rb") as response:
228+
with patch.object(autopatch_cargo.urllib.request, "urlopen", return_value=response) as urlopen:
229+
assert autopatch_cargo.auto_patch_tree(self.workspace, repository) == 1
230+
231+
urlopen.assert_called_once()
232+
patched_crate = self.workspace / ".graalpy" / "crates" / "made-up-crate-1.2.3"
233+
assert '"patched"' in (patched_crate / "src" / "lib.rs").read_text()
234+
235+
def test_ignores_crate_outside_registered_version_range(self):
236+
self.prepare_cached_crate()
237+
repository = self.prepare_repository(version=">=2")
238+
self.prepare_workspace()
239+
240+
assert autopatch_cargo.auto_patch_tree(self.workspace, repository) == 0
241+
assert not (self.workspace / ".graalpy").exists()
242+
assert "graalpy-made-up-crate" not in (self.workspace / "Cargo.toml").read_text()

0 commit comments

Comments
0 (0)
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.