GeezerVison is showing github.com — Support hosted and vendored PDM patches by mikolalysenko · Pull Request #243 · SocketDev/socket-patch
Conversation
LLM Description written by Codex:gpt-6-astra
Hosted PDM scans did not produce a confirmed redirect, while vendored scans rejected historical lock formats and CRLF files. Both modes now share a validated source-and-integrity rewriter with legacy metadata.files support, base/extras entries, byte-preserving edits, idempotency and exact rollback. Conflicting sources, ambiguous package entries, malformed strategies and unsupported schemas are refused before files change. Failed PDM rewrites cannot be reported as successful through a generic manifest rewrite.
The native compatibility matrix covers every stable PDM major family (0.x, 1.x, 2.x), with thirteen releases spanning the lock-format transitions. PDM 0.12.3 supports both modes with pdm sync; the CLI warns about its upstream freshness bug, which can cause ordinary install to regenerate some locks. PDM 1.15.5 and 2.0–2.7 lose URL/path candidate identity in native dependency lookup, so their schemas refuse safely without changing inputs. PDM 2.8.2 through 2.29.2 pass both patch modes. Unknown future formats also refuse.
Validation:
- Core library: 3,512 tests passed; CLI library: 546 tests passed; all 30 PDM-focused tests passed. Workspace clippy with all features and warnings denied passed.
- Real Windows/Linux native matrix: 584 cases across 26 jobs, built from this PR's
75476589b04f1d66d0d92a747ccbb7dc79673067 commit.
- Checks actual installed patch file hashes, repeated scans, lock and manifest preservation, rollback, corrupt-hash rejection, direct/transitive dependencies, extras, optional/development groups, included/excluded Python and platform markers, CRLF, static URLs and space/Unicode paths.
- Unsupported releases have direct native source-identity probes and unchanged-input assertions; their original registry locks still install.
- Fifteen real native lock snapshots and hardened pure/filesystem tests are checked in. Full installer evidence and reproduction live in depscan fixture PR #26378, stacked on core logic PR #26371. The shared annotation integration suite passes all 15,253 assertions.
Note Medium Risk
Changes how Python lockfiles are rewritten and how hosted scans attest PyPI patches when pdm.lock is present; incorrect confirmation or lock edits could affect supply-chain enforcement, though the design is fail-closed with extensive tests.
Overview
Adds PDM (pdm.lock) to hosted registry redirects and aligns vendored PDM wiring with the same lock rewriter.
A shared pdm_lock module validates supported lock versions/strategies, rewrites wheel url/path and file hashes (including legacy [metadata.files] and extras variants), preserves line endings, and produces fragment edits for replay. Hosted scan collects pdm.lock and only confirms PyPI patches when confirmed_pdm_uuids says the PDM rewrite fully landed; refused PDM uuids are excluded like pnpm/cargo.
The redirect pipeline runs PDM first and drops overrides that PDM refused before other ecosystems run. Vendor pypi_pdm drops bespoke text splicing in favor of rewrite_pdm_lock / pdm_lock_edits, broadens supported lock formats (with fail-closed refusal for untested versions), treats legacy [tool.pdm.*-dependencies] as direct deps, and handles CRLF/non-canonical spacing via span-based edits. Replay groups redirect_pdm_lock_package with other PyPI edits. Byte-exact pdm-native lock fixtures and .gitattributes -text rules back the matrix.
Reviewed by Cursor Bugbot for commit 7547658. Configure here.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
cursor
Bot
left a comment
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Leftover PDM lock blocks live PyPI redirects
- PDM rewriter now only runs when pdm.lock is the active lock file (uv.lock not present), preventing stale pdm.lock from blocking uv.lock/requirements.txt patches.
You can send follow-ups to the cloud agent here.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 7547658. Configure here.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Review + hardening: rebased onto main, 14 correctness fixes, full real-PDM matrix
I ran an adversarial multi-agent review of this PR and a full real-PDM compatibility
matrix, then hardened the code path. The work is on a churn-free rebase of this PR
onto current main (which now carries the merged Poetry #241).
Rebase + cleanup
- Rebased onto
main. Poetry Support hosted and legacy Poetry patches #241 merged after this PR forked; the branch needed
it anyway. The vendored PDM backend already shipped on main, so this PR's real new
surface is hosted PDM + a shared pdm.lock rewriter the vendored backend is
refactored onto.
- Dropped ~100 files of unrelated
cargo fmt churn that commit 607e20f folded in
without mentioning it (its actual 13-line fix was buried in +1834/-886 of reflow).
The rebased diff is PDM-only.
Defects found by testing + fixed (verified against real PDM 0.12–2.29)
- Hosted lock-only checkouts were never redirected — the lock inventory didn't read
pdm.lock, so a fresh clone / CI scan --mode hosted discovered nothing. Fixed
(discovery-only, mirrors poetry).
- Hosted rollback broke after an external
pdm lock — PDM un-patches and reflows
CRLF→LF on relock, leaving the ledger's recorded fragment stale (a matrix FAIL on
2.27/2.29). Fixed via the shared rebase machinery plus a PDM-specific original
adoption; Poetry's relock (which keeps the patch) is unchanged.
- Coexistence gating —
pdm.lock now drives (and can veto siblings) only when it's
the actual PyPI install driver (!uv.lock && !poetry.lock), so a leftover pdm.lock
neither blocks nor is falsely attested.
Correctness fixes from the review (adversarially verified)
- Strategy validation: refuse a lock whose
strategy lacks inherit_metadata
(--strategy no_inherit_metadata) — PDM 2.12–2.23 re-resolves it and can't match the
rewritten source (CandidateNotFound); read the 4.3 boolean strategy spelling.
- Veto breadth: a
pdm.lock refusal withholds the sibling rewriters only for a
package the lock contains but refuses, never one it simply lacks (a requirements.txt
redirect that used to land was being dropped).
- Stale confirmation: an inactive
pdm.lock no longer feeds the substring
confirmation probe (an untouched uv/poetry project could report a bogus redirected: 1).
- Warm-install advisory:
redirect_pdm_stale_install_risk for PDM < 2.11 locks, which
don't reinstall over an already-installed same-version package.
- Multi-version marker forks refused with an accurate message (hosted + vendored).
- Vendored integrity: refuse a symlinked
pdm.lock before wire/revert; require a real
sha256 in the pre-flight guard; refuse a pdm add/partial-relock unit that kept the
patched hash but lost the vendored path; supersede a prior Socket URL in place.
- Fragments/keys: trailing fragment boundary (replay convergence); legacy
[metadata.files] keyed by PDM's safe_name().lower() (dotted names).
- Boundary correction: PDM 2.8.0 writes an indistinguishable
4.3 lock but shares the
identity-loss bug, so the supported floor is 2.8.1, not 2.8.0 (docs).
Evidence
- Compatibility matrix (
scripts/backtest-pdm.py, new): real PDM 0.12–2.29 × 17
shapes × hosted/vendored/agent, installed-byte verification, tamper rejection, relock
convergence, byte-exact rollback. Final run: 523 pass, 268 refused-as-expected, 46
unsupported (__pypackages__ agent/vendored, documented), 0 failures.
- Core lib 3530 tests green,
clippy --all-features -D warnings clean, ~9 new PDM
unit tests plus 3 CLI-level hosted regression tests.
- New CI:
.github/workflows/pdm-compatibility.yml (Linux/Windows/macOS), docs at
docs/testing/pdm-compatibility.md, and the ecosystem/hosted-e2e/CLI_CONTRACT/CHANGELOG
docs corrected (they still denied hosted PDM).
Bugbot's HIGH ("leftover pdm.lock blocks live redirects") is resolved: the !uv.lock gate
now also excludes poetry.lock, and the leftover/inactive lock is excluded from
confirmation.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Adds hosted PDM redirects and refactors the vendored PDM backend onto one
validated `pdm.lock` rewriter (`utils/pdm_lock.rs`). Hosted `scan --mode hosted`
rewrites the target `[[package]]` to the hosted wheel URL + patched SHA-256
(`redirect_pdm_lock_package`); vendored wires the same unit to a committed
wheel. Both support the legacy `[metadata.files]` table and `extras` entries,
preserve line endings and non-canonical spacing, are idempotent, and leave
`pyproject.toml` / `content_hash` untouched. Supported lock formats are `2`
(PDM 0.12-1.4) and `4.3`-`4.5.1` (PDM 2.8+); `3.1` / `4.0`-`4.2` (PDM 1.8-2.7)
and unknown formats are refused before any write.
Hardening beyond the original change:
- Hosted lock-only checkouts are discovered and redirected: the lock inventory
now reads `pdm.lock` (discovery-only, mirroring poetry). Vendored keeps
refusing a lock-only checkout since PDM installs into `__pypackages__`, which
the crawler does not probe.
- A re-scan after an external `pdm lock` rebases the ledger's edits onto the
relocked text. PDM un-patches AND may reflow line endings (CRLF -> LF) on
relock, so for the pdm edit kind the rebase adopts the fresh `original` (the
relocked registry fragment); poetry's relock keeps the patch and keeps its
older original, so its path is unchanged.
- When several Python lockfiles coexist, `uv.lock` and `poetry.lock` drive
hosted PyPI redirects ahead of `pdm.lock`; a leftover `pdm.lock` neither
blocks a live redirect nor is falsely attested. The same predicate gates the
redirect dispatch and the hosted confirmation filter.
Adds byte-exact `pdm-native` fixtures and CLI-level hosted regression tests
(lock-only redirect + attest + rollback, legacy `[metadata.files]`, and
relock-reflow convergence).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Documents hosted PDM support (ecosystems.md, hosted-production-e2e.md,
CLI_CONTRACT.md new codes, CHANGELOG) and adds the per-release matrix:
docs/testing/pdm-compatibility.md with a generated table, scripts/backtest-pdm.py
(real PDM releases via uv, hosted/vendored/agent, installed-byte verification,
tamper rejection, relock convergence, rollback), and
.github/workflows/pdm-compatibility.yml (Linux/Windows/macOS).
Measured 0.12-2.29 across 17 shapes x 3 modes: 523 pass, 268 refused as
expected (identity-losing 3.1/4.0-4.2, forked two-version locks, non-default
lock filenames), 46 unsupported (`__pypackages__` agent/vendored, a documented
layout limitation), 0 failures.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A 12-finder review (42 confirmed findings) plus a real-PDM verification pass
surfaced correctness gaps beyond the initial three. Fixes, each verified
against real PDM binaries:
Confirmation / attestation (false-success class):
- Refuse a lock whose `strategy` lacks `inherit_metadata` (lock_version
4.4.1/4.5.x) — PDM 2.12-2.23 re-resolves such a lock and cannot match a
rewritten `url`/`path` source (CandidateNotFound); read the 4.3 boolean
strategy spelling (`cross_platform`/`static_urls`) that PDM 2.8-2.9 write.
- A `pdm.lock` refusal now vetoes the sibling pypi rewriters only for a package
the lock CONTAINS but refuses, never one it simply lacks (which a
`requirements.txt`/pylock legitimately installs) — the old veto silently
dropped those redirects.
- An inactive `pdm.lock` (a `uv.lock`/`poetry.lock` drives) no longer feeds the
substring confirmation probe, so an untouched uv/poetry project cannot report
a bogus `redirected: 1`.
- Emit `redirect_pdm_stale_install_risk` for PDM < 2.11 locks (2/4.3/4.4), which
do not reinstall over an already-installed same-version package.
- A marker/multi-target lock holding one package at several versions is refused
with an accurate, ordering-independent message (hosted and vendored).
Vendored integrity:
- Refuse a symlinked `pdm.lock` before wire/revert (the rename-over would
replace the link and strand its target) — mirrors the uv/hosted guards.
- Require a real sha256 in the pre-flight hash guard so a non-sha256 lock is
refused before the wheel is built.
- Refuse a `pdm add`/partial-relock unit that kept the patched hash but lost the
vendored `path`, so a stale patched hash is never recorded as the registry
original.
- Supersede a prior Socket-hosted `url` (rotated token / new uuid, same origin
and wheel leaf) in place instead of refusing it as foreign.
- Fix the version-mismatch refusal to print the plain version, not `Some("…")`.
Fragments / keys:
- Give the recorded package fragment a trailing boundary so a lock_version-2
pristine fragment is not a prefix of the rewritten one (replay convergence).
- Key legacy `[metadata.files]` by PDM's `safe_name().lower()` (dots preserved),
so dotted-name packages in format-2 locks are found.
Docs: PDM 2.8.0 writes an indistinguishable `4.3` lock but shares the
identity-loss bug, so the supported floor is 2.8.1; the docker vendored e2e
awk counter now matches the inline `files` shape.
Adds unit tests for the strategy gate, multi-version fork, URL supersede,
dotted-name key, absent-package veto skip, stale advisory, symlink guard and
the non-sha256 pre-flight refusal.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Uh oh!
There was an error while loading. Please reload this page.
Rebased onto main (07a6b88) — conflicts with Hatch #244 and Pipenv #242 resolved
The branch is again the same three PDM commits, now on top of main. Nine files overlapped with the two PRs that landed after this one forked; how each was merged:
Hosted redirect dispatch (core/src/patch/redirect/mod.rs) — order is now pdm (only when pdm.lock drives: no uv.lock / poetry.lock) → pipenv → the rest. This follows the vendored flavor precedence on main (uv > poetry > pdm > pipenv): a pdm refusal for a package the lock contains is withheld from the Pipenv rewriter and every later pypi rewriter; a live-Pipfile conflict still vetoes the siblings as #242 specified. Hatch already yields to any lock, pdm.lock included.
Hosted confirmation filter (cli/src/commands/scan/hosted.rs) — refused-pipenv and refused-pdm both short-circuit to unconfirmed; when pdm.lock drives, a pypi dep is confirmed only by confirmed_pdm_uuids. That pdm check sits ahead of the hatch gate from #244: is_hatch is true for any pyproject.toml with build-backend = "hatchling.build", and in that case the hatch rewriter registers every pypi uuid as hatch-owned and then yields to the lock without confirming any — so with the gate first, a PDM project using hatchling as its build backend would rewrite pdm.lock and attest nothing. Regression test added (hatchling_build_backend_does_not_veto_the_pdm_lock_redirect); verified it fails with the gate first and passes with this ordering. The third commit's pdm_inactive exclusion from the substring probe carried over unchanged.
Lock inventory (core/src/vendor/lock_inventory.rs) — pdm.lock slots between poetry.lock and the Pipfile.lock + requirements.txt pair (which #242 reads together), same precedence as above. Both new inventory readers kept.
replay.rs / utils/mod.rs — additive (pdm kind joins the pypi ReplaceFragment group beside pipenv's PipenvEntry and hatch's HatchDocument).
Docs — CLI_CONTRACT.md rewriter paragraph and the ecosystems.md PyPI row now describe uv / poetry / pdm / pipenv all as hosted; hosted-production-e2e.md no longer claims any PyPI lock flavor is vendored-only; both CHANGELOG entries kept.
One test adjusted from #244: hatch_confirmation_uses_successful_lock_writers used a package-less pdm.lock stub; the pdm rewriter (deliberately, per the hardening commit) refuses a lock with no [[package]] array, which withheld the uuid from hatch and broke the stub's assertion. It now uses the real pdm-native/2.29.2.lock fixture, mirroring the test's poetry fixtures, and additionally asserts pdm confirmed through its own set.
Verification on the rebased branch: core lib 3,585 tests green; CLI suite 2,961 tests across 204 binaries green; cargo clippy --workspace --all-features -- -D warnings clean. No non-PDM file changes beyond the merge sites (35 files vs main, same set as before).
Main-side finding, not fixed here (out of scope): the same hatch-gate ordering affects Pipenv on current main — a Pipfile.lock project whose pyproject.toml names hatchling.build fails scan --mode hosted with exit 1 (nothing confirmed). Reproduced by adding such a pyproject.toml to lock_only_pipenv_project_redirects_attests_rescans_and_rolls_back. The fix is the same one-line ordering change for confirmed_pipenv_uuids; happy to open a follow-up.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|