GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Fix npm rewiring git/URL/file lock entries (#326) by mikolalysenko · Pull Request #345 · SocketDev/socket-patch Conversation
|
| #326 case | Regression test | Red on main |
Green |
|---|---|---|---|
hosted: git, URL-tarball, file: tarball (direct) |
patch::redirect::tests::npm_non_registry_entries_are_skipped_with_loud_warning |
✅ fails | ✅ |
| hosted: nested git copy skipped, hoisted registry copy still rewired | npm_nested_git_copy_is_skipped_and_registry_copy_rewired |
✅ fails | ✅ |
| hosted, v2: legacy mirror of the git copy untouched, registry copy's mirror rewired | npm_v2_legacy_mirror_of_a_non_registry_entry_is_not_rewired |
✅ fails (guard disabled) | ✅ |
vendored: git, URL, file: (refuses, writes nothing) |
vendor::npm_lock::tests::non_registry_only_instances_refuse_and_write_nothing |
✅ fails | ✅ |
| vendored: nested git copy skipped | nested_git_instance_is_skipped_with_warning |
✅ fails | ✅ |
| vendored, v2: legacy mirror of the git copy untouched | v2_legacy_mirror_of_a_git_instance_is_not_rewritten |
✅ fails (guard disabled) | ✅ |
VEX: hosted- and vendored-wired entry with a git/URL/file: spec, nested git copy contests the wired copy, plus a registry control |
vex::discover::npm::tests::entries_npm_installs_from_a_non_registry_spec_are_not_attested |
✅ fails | ✅ |
real npm 10 / 12: URL-tarball dependency, vendor then vex |
e2e_vendor_npm_build::npm_vendor_refuses_a_remote_tarball_dependency |
✅ fails (applied: 1) |
✅ |
| classifier | vendor::npm_origin::tests::* (9 tests: spec classification, node lookup, workspaces, aliases) |
new | ✅ |
Red was shown by stashing the three core source files and re-running the new tests against main's implementation. For the two v2-mirror tests, it was shown by disabling the new guard.
cargo test -p socket-patch-core --all-features --lib -- npm vex::discover redirect: 1494 passed and 1 failed. The failure is vlt_heal::tests::an_unremovable_hidden_lock_keeps_every_store_entry, which fails on the unchanged branch too (the sandbox runs as root).SOCKET_PATCH_NPM_E2E_REQUIRED=1 cargo test -p socket-patch-cli --all-features --test e2e_vendor_npm_build --test e2e_redirect_npm_build -- --include-ignored (real npm 10.9.7, Node 22): 14 + 15 passed. The new e2e also passes against npm 12.1.0.
EALLOWREMOTE). That made the new e2e fail its fixture install in CI's install-proof (12.x) legs, so it now passes --allow-remote=all on npm ≥ 12.cargo clippy --workspace --all-features -- -D warnings: clean.cargo fmt --all -- --check: the new code is rustfmt-clean. The remaining diffs are already on main, in files and hunks this PR doesn't touch.cargo test --workspace --all-features --no-fail-fast (on d53ef13): 10,407 passed and 22 failed. The 22 are the same sandbox-only set reported on Fix #258: state the real Maven Trusted Checksums floor #322, Fix Poetry venv discovery to match Poetry (#327, #329) #330 and Fix npm VEX attesting packages with an unpatched bundled copy (#325) #337: write-failure and permission tests that fail because the sandbox runs as root, plus the peak-RSS stage_local_artifact_caps_oversized_artifact_before_buffering. None of them are in npm or VEX code.install-proof (npm registry) and macOS Poetry 1.8.5 (PyPI).scan --vex goes through the same discovery gate.vex/discover/npm.rs next to Fix npm VEX attesting packages with an unpatched bundled copy (#325) #337, which adds a same-shaped gate for bundled copies. The two edit different functions (extract_package_lock here, push_uncontested there), so whichever lands second should need at most a trivial merge.overrides entry forces a URL spec onto a registry-range edge, the lock alone doesn't show it. The resolved check still catches forced git and file: sources.🤖 Generated with Claude Code
https://claude.ai/code/session_01LycgFGuki2BqZ25VNhwxJ5
Note
Medium Risk
Changes npm lock rewrite, vendoring, and VEX attestation gates for a common dependency pattern; incorrect classification could skip legitimate registry patches or still over-attest, but behavior is covered by extensive tests and fail-closed warnings.
Overview
Fixes false “patched” reporting for npm deps installed from git, URLs, or file: specs (#326). npm honors the dependent’s spec, not the lock entry’s resolved, so rewiring those entries did not change what npm ci installed while hosted/vendored flows and VEX still claimed success.
A shared npm_non_registry_entries classifier (new vendor/npm_origin.rs) drives consistent behavior: hosted scan skips with redirect_npm_non_registry_entry_skipped, vendor skips or refuses with vendor_non_registry_entry_skipped / vendor_lock_entry_not_rewritable, and VEX discovery drops attestations for affected name@version pairs (patched_ref_unattributable). Lockfile v2 legacy dependencies mirrors of skipped entries are left untouched. Docs and e2e/unit tests cover git, URL tarball, file:, and mixed registry/non-registry trees.
Reviewed by Cursor Bugbot for commit f1e0108. Configure here.
Generated by Claude Code
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Assisted-by: Claude Code:claude-opus-5-5
npm installs a git, remote-tarball or file: dependency from the dependent's spec and ignores the lock entry's resolved. Hosted and vendored mode rewired those entries anyway, so the scan reported the package patched and vex attested it while npm ci installed the original bytes. Both rewriters now skip such entries with a loud stays-UNPATCHED warning (vendored refuses when no registry copy is left), and vex no longer attests a name@version while a non-registry copy of it is in the lock. Fixes #326 Assisted-by: Claude Code:claude-opus-5-5
4a797e2 to
ad605fe
Compare
Assisted-by: Claude Code:claude-opus-5-5
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
npm 12 refuses remote-tarball specs unless allow-remote permits them, so the new vendored e2e now passes --allow-remote=all there. The VEX test messages and the new diagnostic no longer print patch URLs or uuids. Assisted-by: Claude Code:claude-opus-5-5
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent]
This PR only changes npm lock handling and npm VEX discovery; no PyPI or Poetry code is touched. I've re-run the failed jobs of run 36755620097 once. If it fails again, I'll treat it as a real failure and dig in. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent]
This PR doesn't change any vlt code. I can't re-run the job yet because workflow run 36755620093 still has jobs in progress (the API returns 403). I'll re-run it once when the run finishes, and treat a second failure as real. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
The hosted rewriter and the vendored backend skipped a git, URL or file: packages entry, but still rewired its lockfileVersion 2 legacy dependencies mirror when that mirror stored the plain version. Each legacy node now maps to the packages key it mirrors, and a node whose twin is non-registry is left alone. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01LycgFGuki2BqZ25VNhwxJ5
|
[agent]
This PR only changes npm lock handling and npm VEX; no Composer code is touched. I can't re-run it yet because run 36758922826 still has jobs in progress (403). I'll re-run it once when the run finishes, and treat a second failure as real. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Bring #326's fix onto the v5 workflow from #277 so the PR merges cleanly. Main dropped the pre-index npm/yarn oracles from the lock index equivalence tests, so the PR's oracle tweak goes with them; the vex npm discover imports follow main's move to formats::pnpm and keep the npm_origin import; the changelog entry moves into main's condensed Unreleased "Fixed" section. Co-Authored-By: Claude <[email protected]> Claude-Session: https://claude.ai/code/session_01GQoii5oP1pwcJh5mzzo1HU
|
bugbot run Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[burn-down agent] Ready for review on Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Union the CHANGELOG Fixed entries from both sides. Co-Authored-By: Claude <[email protected]>
|
bugbot run Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Co-Authored-By: Claude <[email protected]>
|
bugbot run Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f1e0108. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] Ready for review on
Slack announcement not sent this run: the Slack send tool isn't available in the agent session. The next run will retry. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Keeps both CHANGELOG entries (gem manifest fix and the npm non-registry entry fix from #345). Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01L6i4cQ51yarBFnFs2b8HRx
Bun installs a dependency declared by remote tarball URL or `file:` tarball from that spec, never from the registry. When the lock also holds a registry copy of the same name@version (for example nested under a dependent), hosted and vendored mode rewired only the registry copy and reported plain success, while the root copy that the app loads stayed unpatched after `bun install --frozen-lockfile`. Lockfile VEX then attested not_affected for that name@version. That covers vendored (default and --no-verify), hosted --no-verify, and hosted in a lockfile-only checkout. This is the Bun twin of npm's #326 / #345. The tarball tuple / record carries no version of its own, so its version is read from the artifact leaf, `<name>-<semver>.tgz` (`user_tarball_version`, shared by every Bun path). Then: - the hosted bun.lock and bun.lockb rewriters leave such a copy alone with a `redirect_bun_non_registry_entry_skipped` stays-UNPATCHED warning, and keep the uuid out of the in-run VEX assumptions; - the vendored text and binary backends warn with `vendor_non_registry_entry_skipped`, and refuse with `vendor_lock_entry_not_rewritable` when no registry copy is left; - lockfile VEX discovery counts the copy as an unpatched install of that version, so every ref for it in the lock is withdrawn with a patched_ref_unattributable diagnostic and other locks are contested. Regression tests use fixture text locks and real Bun 1.1.45 bun.lockb fixtures (URL and file: shapes) generated for this issue. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The npm/hosted bench drifted +15% wall / +12% CPU over a week of main (2463257..9c43dfc). The new npm lock passes from #345, #491, #646 and #799 each added a walk of the 3000-entry package-lock.json, on top of two costs that scale badly with every extra walk: - Every hosted scan ran a full second lockfile discovery after the rewrite (`hosted_state_from_lockfiles`) only to classify a hosted vs vendored takeover, even with no vendored ledger to overlap, and then a THIRD one inside `classify_overlap_takeover_with` when there was one. The classifier now discovers once, and not at all when the vendored ledger has no entries (the hosted common case). - `Discovery::resolved_elsewhere` deduped with `Vec::contains` on every call, so recording a lock's registry entries was quadratic in its size. `finalize` already sorts and dedups the list and every earlier reader only asks whether some entry matches, so the per-call check is gone. Also trimmed per-entry allocations on the new passes: the npm extractor builds each entry's purl once (it was built for `mention` and again for `unwired`), `resolved_is_non_registry` no longer formats a prefix per entry, and `npm_spec_is_registry` no longer lowercases every edge spec. Instructions retired for `scan --json --dry-run` on the bench's npm fixture (median of 9, macOS): base 2463257 2.647G, branch head 2.755G (+4.1%), this commit 2.353G (-11.1% vs base, -14.6% vs head). Scan output is byte-identical to before. Regression test: `takeover_classifier_discovers_at_most_once` counts discoveries (test-only counter in `discover_wiring`); new unit tests pin the rewritten registry/tarball predicates. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
npm hosted and vendored modes rewire git-sourced lock entries, so npm ci silently installs the unpatched git bytes while scan and VEX report success
There was an error while loading. Please reload this page.