GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||
GeezerVison is showing github.com — Fix npm crawler missing Bun, Deno and Yarn 4 stores (#366, #373, #405, #495) by mikolalysenko · Pull Request #496 · SocketDev/socket-patch Conversation
|
| Issue | Regression test |
|---|---|
| #366 | in_process_alternate_installers::bun_isolated_linker_transitive_only_dep_apply_patches_store: real bun install with linker = "isolated", apply patches .bun/[email protected], and is-odd loads the patched copy. Also the core unit test test_bun_isolated_store_transitive_packages_are_found (scan, resolver, peer +hash twin, hosted-URL entry name, scoped). |
| #373 | in_process_alternate_installers::deno_node_modules_dir_transitive_only_dep_apply_patches_store (a Deno 2.x-shaped .deno tree), plus test_deno_node_modules_store_transitive_packages_are_found (scan, resolver, _peer twin, scoped, .deno.lock, hoist dir). |
| #405 | e2e_vex_redirect::bun_hosted_ref_is_judged_by_the_bun_store_copy: hosted bun.lock pin with the copy only in .bun, under both the stale ([email protected]) and fresh (mangled-URL) entry names. A pristine copy now gives not_applied, where main falsely attested it (exit 0). |
| #495 | e2e_yarn4_pnpm_linker_build::yarn4_pnpm_linker_agent_apply_patches_transitive_store_copy: real yarn 4.12.0 with nodeLinker: pnpm, apply patches .store/is-number-npm-6.0.0-*/package, yarn node loads it from is-odd, and rollback restores it. Also test_yarn4_pnpm_linker_store_transitive_packages_are_found. |
| #496 review | test_yarn4_pnpm_linker_bundled_copy_inside_store_package_is_resolved and real yarn 4.12.0 yarn4_pnpm_linker_agent_apply_patches_bundled_copy_inside_store_package: a package bundling [email protected] keeps that copy at .store/parent-…/package/node_modules/is-number, which Node loads. Apply patches it alongside the regular store copy, and rollback restores both. |
On main each of these fails: package_not_installed (apply exit 1), or the VEX false attestation.
cargo clippy --workspace --all-features -- -D warnings: clean. The workspace-wide cargo fmt --check reports the same pre-existing diffs as main (7f41839 reverted the unrelated reformat; CI doesn't run fmt).
cargo test -p socket-patch-core --lib npm_crawler: 67 passed, including the randomized oracle-equivalence tests.
cargo test --workspace --all-features --no-fail-fast: every suite passes except 14 tests in 5 targets that can't pass in this sandbox, all unrelated to the crawler:
covgap_commands_vendor, in_process_redirect, repair, and 4 core lib tests). The container runs as root, so the writes never fail.mode_migration_npm berry takeover tests, which fail TLS against the sandbox proxy's CA.CI runs all of these as non-root, with normal network.
Real-toolchain e2e: bun 1.3.14 and yarn 4.12.0 legs pass, and each fails when run against main's crawler. The yarn 4 bundled-copy test (2d9eb4d) passes with SOCKET_PATCH_YARN_E2E_REQUIRED=1 and fails with the fix disabled.
CI: green on 3f8e0c8 (382/382 non-skipped, Windows included). 3f8e0c8 only fixes the bundled-copy e2e test's Windows path comparison. Bugbot clean on 2d9eb4d.
No wrapper changes are needed: npm/, pypi/ and gem/ only dispatch to the Rust binary.
🤖 Generated with Claude Code
Sorry, something went wrong.
There was an error while loading. Please reload this page.
With Bun's isolated linker, Deno's isolated nodeModulesDir or Yarn 4's pnpm linker, a transitive dependency lives only in the package manager's store, which the crawler never looked in. Agent-mode apply and scan reported those packages as not installed and left them unpatched with exit 0, and VEX treated a hosted or vendored Bun pin as having no installed copy, so it attested without checking the bytes. The crawler now walks node_modules/.bun and node_modules/.deno like pnpm's store, in scan, apply's resolver and the peer-copy fan-out. In Yarn 4's node_modules/.store it finds each package at the entry's package/ dir, which the entry's own node_modules link points to. Assisted-by: Claude Code:claude-opus-5-5
Installs is-odd with the real yarn 4 pnpm linker, so is-number lives only in node_modules/.store, then checks agent-mode apply patches it, is-odd loads the patched copy, and rollback restores it. Assisted-by: Claude Code:claude-opus-5-5
With bun.lock pinning the hosted tarball and the package installed only in node_modules/.bun, a pristine store copy must be reported not_applied, and a patched one attested. Before the crawler fix VEX saw no installed copy and attested the pristine one. Assisted-by: Claude Code:claude-opus-5-5
A real bun install with linker = "isolated" puts is-number only in node_modules/.bun; apply must patch it and is-odd must load the patched copy. A Deno-shaped node_modules/.deno tree gets the same check without needing deno installed. Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
b5985fc ran cargo fmt over the whole workspace, so 124 files outside the crawler fix changed formatting only. That buried the real change for reviewers and invites conflicts with every other open PR. Restore those files to their main versions; npm_crawler.rs keeps its fix. Co-Authored-By: Claude <[email protected]>
|
bugbot run Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Burn-down agent: ready for review at
Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Reviewed The new Yarn 4 store support can report a successful apply while the package actually loaded at runtime remains unpatched.
Validation: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
A Yarn 4 pnpm-linker store entry keeps its package at <entry>/package and links <entry>/node_modules/<name> to it. The scan descends into that package's own node_modules through the link, but the resolver never follows links, so it missed bundled dependencies there. Apply then patched the regular store copy and reported success while Node kept loading the unpatched bundled copy. When a store entry links to its own package dir, the resolver now also enqueues that dir's node_modules. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01T2DjA5mBwq29D5rrFa5Fvd
There was an error while loading. Please reload this page.
|
Fixed the P1 in 2d9eb4d. Thanks for the repro. The resolver now does what the scan does. When a store entry's Tests:
Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] Independent check of 2d9eb4d: I had drafted the same fix in parallel. I ran my two extra regressions against 2d9eb4d and both pass, so I dropped my branch and pushed nothing:
Both fail on 7f41839 and pass on 2d9eb4d. CI on 2d9eb4d is still running, and I have asked Bugbot to re-review. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2d9eb4d. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Follow-up reviewed The resolver now visits bundled dependencies inside a Yarn 4 store entry’s physical Validation on this head: original reviewer resolver regression 1 passed; |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
The bundled-copy e2e test matched Node's resolved path against a
forward-slash substring, which fails on Windows where Node prints
backslashes. Compare the canonical path of what `require('parent')`
resolves to with the bundled copy's instead.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T2DjA5mBwq29D5rrFa5Fvd
Keep the Bun, Deno, and Yarn 4 store regressions alongside main's Yarn modules-folder and Rush install-root regressions when resolving the appended-test conflicts. Production changes from both parents are preserved. Signed-off-by: Mikola Lysenko <[email protected]>
There was an error while loading. Please reload this page.
|
Fixed the merge conflicts and re-reviewed All three conflicts were overlapping appended tests. Both regression sets are preserved: Bun/Deno/Yarn 4 isolated stores, plus Yarn’s configured modules folder and Rush’s shared install root. No production implementation was discarded, and no further actionable defect was found. Validation: 78 crawler tests, 3 hosted VEX tests, 3 apply fixtures, and 2 real Yarn 4 pnpm-linker tests pass. The Yarn tests verify runtime-loaded bytes and rollback, including bundled copies; they ran in required mode without skips. Tests ran on macOS with Node 24.21.0; real Bun installation and the full platform matrix were not rerun locally. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
A rescan's preparatory scan returned early on a validation failure, skipping the tree restore. The project stayed rewritten, so in `compare` the other binary's next run started from a rescanned tree and was reported INVALID for the first binary's fault: on the 2026-10-04 weekly A/B a pre-#496 base failed bun-isolated/rescan's prep and the head (valid on its own) was flagged invalid too. Restore before returning the error, as the measured run already does. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_016Gubp3gsnxFDvRbguaeLqT
* Bench: cover Bun's isolated .bun store Bun 1.3.2+ installs with the isolated linker by default, keeping every package only in node_modules/.bun/<name>@<version>/node_modules/<name>. #496 taught the npm crawler (scan, apply's resolver, VEX) to walk that store, but the suite only had a hoisted Bun layout, so the new walk was never timed and a regression back to "2700 lockfile-only packages" would have gone unnoticed. Add bun-isolated/{hosted,rescan}: the same text bun.lock as bun/*, with a pnpm-shaped .bun store, per-entry dependency links, Bun's .bun/node_modules hoist links and root links for direct deps only. A pre-#496 binary fails it (lockfileOnlyPackages: got 2700, want 0). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Restore fixture after a failed rescan prep A rescan's preparatory scan returned early on a validation failure, skipping the tree restore. The project stayed rewritten, so in `compare` the other binary's next run started from a rescanned tree and was reported INVALID for the first binary's fault: on the 2026-10-04 weekly A/B a pre-#496 base failed bun-isolated/rescan's prep and the head (valid on its own) was flagged invalid too. Restore before returning the error, as the measured run already does. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_016Gubp3gsnxFDvRbguaeLqT --------- Co-authored-by: Claude <[email protected]>
Bun's isolated linker never prunes node_modules/.bun. After the usual in-place `bun install` of a hosted-rewired lock (or any version churn), the old `<name>@<version>` entry stays on disk while every importer, the `.bun/node_modules` hoist links and the dependents' entries are re-linked to the new entry. The npm crawler's pnpm-shaped store walk (since #496) enumerated every `.bun` entry dir as an installed copy with no reachability check, so vex judged the install by an orphan nothing can load and refused every hosted patch as not_applied (exit 1) until `rm -rf node_modules`; vendored mode raised false vendored_tree_out_of_sync warnings, and agent-mode apply fanned out to the orphans. The `.bun` store listing, shared by the scan, the purl resolver (apply) and the peer-variant finder, now keeps only entries reachable through links from the importer `node_modules`, the hoist dir, workspace members' `node_modules` (the crawler's own workspace walk, only when needed) and live entries. A stale link Bun left behind still counts, since the runtime resolves through it, and a store with no links at all keeps every entry. A quick first walk guesses link targets by package name (unique names, and scopes without duplicated packages) and reuses the entry listings the scan reads anyway, so a clean store costs no extra readdir per entry; only when it leaves an entry unreached is the store re-walked reading every link before anything is dropped (an alias link can defeat the name guess). bench bun-isolated/{hosted,rescan} stay within noise. Verified against real Bun 1.3.14 and 1.4.2 trees (orphaned [email protected] after rewiring is-odd/is-number to tarball URLs): vex exits 1 on main, 0 with this change. Fixtures that left store entries unlinked are updated to link them the way Bun does. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
Yarn 4 pnpm linker: transitive packages that live only in node_modules/.store are "not installed" in agent mode and stay unpatched With Bun's isolated linker,vex attests a hosted patch as not_affected (verified) while the installed copy under node_modules/.bun is still unpatched (v5 regression)
Deno nodeModulesDir: transitive npm packages under node_modules/.deno are "not installed", and apply/scan exit 0 leaving them unpatched
Bun isolated linker: transitive packages under node_modules/.bun are "not installed" in agent mode, and scan --mode agent exits 0 with them unpatched
There was an error while loading. Please reload this page.