GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Fix npm crawler missing configured install roots (#493, #518) by mikolalysenko · Pull Request #520 · SocketDev/socket-patch Conversation
|
| Issue | Test | Without fix |
|---|---|---|
| #493 | npm_crawler::tests::test_yarnrc_modules_folder_is_a_crawl_root |
roots [deps/outer/node_modules], not [deps] |
| #493 | npm_crawler::tests::test_parse_yarnrc_modules_folder |
(new parser) |
| #493 | e2e_vex_redirect::yarn_modules_folder_install_is_hash_verified_not_lockfile_attested |
exit 0, verified / not_affected over the unpatched deps/ copy |
| #493 | e2e_vex_vendor::vendored_modules_folder_tree_out_of_sync_warns |
no vendored_tree_out_of_sync warning |
| #493 | in_process_alternate_installers::yarn_modules_folder_install_is_patched |
apply exit 1 |
| #493 | in_process_alternate_installers::yarn_classic_modules_folder_install_then_apply_patches_file (real yarn install 1.22.22) |
apply exit 1 |
| #518 | npm_crawler::tests::test_rush_common_temp_is_a_crawl_root |
roots lack common/temp/node_modules |
| #518 | e2e_vex_redirect::rush_common_temp_install_is_hash_verified_not_lockfile_attested |
exit 0, verified / not_affected over the unpatched store copy |
| #518 | in_process_alternate_installers::rush_transitive_dep_in_common_temp_store_is_patched |
apply exit 1 |
| #518 | e2e_redirect_rush_sim tier 1 (real pnpm@9 install --frozen-lockfile): new stale-store leg |
rush, unpatched store copy: exit Some(0) |
| both | npm_crawler::tests::test_merge_configured_install_roots |
(new helper) |
| review | test_resolve_modules_folder, test_escaping_modules_folder_is_not_a_crawl_root |
escaping ../outside / absolute dir crawled |
| review | test_fifo_yarnrc_does_not_block_the_crawl |
(would block on open(2)) |
| review | test_inherited_yarnrc_modules_folder_resolves_against_its_dir |
[] (looked in project/project/deps) |
| review | test_install_scoped_modules_folder_takes_precedence |
[general], not [specific] |
Commands run locally (Linux):
cargo clippy --workspace --all-features -- -D warnings (the CI command): clean.cargo test -p socket-patch-core --lib: all 71 crawler tests pass. Four tests elsewhere fail only because this container runs as root (read-only-directory tests can't fail a write as uid 0): copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_…, pypi_requirements::wire_failure_rolls_back_…. This PR doesn't touch them, and they pass in CI.cargo test -p socket-patch-cli --all-features --test e2e_vex_redirect (27/27), --test e2e_vex_vendor (23/23), --test in_process_alternate_installers (21/21), --test e2e_redirect_rush_sim -- --ignored (3/3, real pnpm).yarn install --offline 1.22.22 confirms the inherited and install-scoped layouts: installs land in project/deps, project/specific and project/specific.cargo test --workspace --all-features build exceeds this container's disk allowance, so the full matrix runs in CI.cargo fmt --all -- --check isn't a CI gate and isn't clean on main. Only this PR's hunks were formatted; no unrelated reformatting is included..yarnrc FIFO could block the crawl. It's now read through the regular-file guard.--modules-folder path could escape the project. It's now normalized and gated like composer vendor-dir..yarnrc modules folder must resolve against its own directory.--install.modules-folder takes precedence over --modules-folder.native (ubuntu-latest, 0.0.0-16) (vlt) and native (ubuntu-latest, 1.3.10) (Bun) failed once during a production patch-API outage (Connection reset by peer, 02:28–02:35 UTC). Each passed on its single re-run.🤖 Generated with Claude Code
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Assisted-by: Claude Code:claude-opus-5-5
The npm crawler found a project's installed packages only in dirs named node_modules, so it never looked where yarn classic installs with `--modules-folder` or where Rush installs (common/temp). Agent mode reported those packages as not installed and left them unpatched, and hosted vex treated the unpatched copy as absent and attested the patch from the lockfile pin alone. The crawler now also reads the nearest .yarnrc --modules-folder and adds common/temp/node_modules in a Rush repo (rush.json at the root). Fixes #493, #518 Assisted-by: Claude Code:claude-opus-5-5
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
.yarnrc
Or push these changes by commenting:
@cursor push 515f68cd5e
diff --git a/crates/socket-patch-core/src/crawlers/npm_crawler.rs b/crates/socket-patch-core/src/crawlers/npm_crawler.rs --- a/crates/socket-patch-core/src/crawlers/npm_crawler.rs +++ b/crates/socket-patch-core/src/crawlers/npm_crawler.rs @@ -8,7 +8,7 @@ use super::types::{CrawledPackage, CrawlerOptions}; use super::walk_pool::{par_map, run_walk}; use crate::patch::path_safety; -use crate::utils::fs::{is_dir, is_dir_sync, read_dir_entries_sync}; +use crate::utils::fs::{is_dir, is_dir_sync, read_dir_entries_sync, read_regular_to_string_sync}; use crate::utils::purl::{percent_decode_purl_component, strip_purl_qualifiers}; use crate::vendor::vlt_lock_text::decode_vlt_dep_id; @@ -46,7 +46,11 @@ pub(super) fn configured_install_roots(start_path: &Path) -> Vec<PathBuf> { let mut roots = Vec::new(); if let Some(folder) = yarnrc_modules_folder(start_path) { - roots.push(start_path.join(folder)); + if let Some(normalized) = normalize_modules_folder(&folder) { + if path_safety::is_safe_multi_segment(&normalized) { + roots.push(start_path.join(normalized)); + } + } } if start_path.join("rush.json").is_file() { roots.push(start_path.join("common").join("temp").join("node_modules")); @@ -78,10 +82,12 @@ } /// The `--modules-folder` value from the nearest `.yarnrc` at or above -/// `start_path`, if any `.yarnrc` sets it. +/// `start_path`, if any `.yarnrc` sets it. Read `.yarnrc` via +/// `read_regular_to_string_sync` for the same reason `package.json` is: +/// a FIFO planted at that workspace path would wedge a plain read forever. fn yarnrc_modules_folder(start_path: &Path) -> Option<String> { start_path.ancestors().find_map(|dir| { - let rc = std::fs::read_to_string(dir.join(".yarnrc")).ok()?; + let rc = read_regular_to_string_sync(&dir.join(".yarnrc")).ok()?; parse_yarnrc_modules_folder(&rc) }) } @@ -137,6 +143,29 @@ (end > 0).then(|| (s[..end].to_string(), &s[end..])) } +/// Reduce a `--modules-folder` value to plain `a/b` segments before the +/// safety gate. Yarn accepts `./`-prefixed and `.`-interleaved values +/// (`./deps`, `lib/./deps`) and either separator, so those shapes must +/// resolve rather than be refused. `..` is resolved lexically the same way +/// Composer's config.vendor-dir normalization does; a value that climbs +/// above the project root (or reduces to it) fails closed as `None`. +fn normalize_modules_folder(raw: &str) -> Option<String> { + if raw.starts_with(['/', '\\']) { + return None; + } + let mut segments: Vec<&str> = Vec::new(); + for segment in raw.split(['/', '\\']) { + match segment { + "" | "." => {} + ".." => { + segments.pop()?; + } + other => segments.push(other), + } + } + (!segments.is_empty()).then(|| segments.join("/")) +} + // --------------------------------------------------------------------------- // Helper: read and parse package.json // --------------------------------------------------------------------------- @@ -1237,7 +1266,11 @@ /// Inside a store entry (`store_entry`) a link is a dependency edge into /// a sibling entry, whose own visit records that copy, so only a real /// directory there matches. - fn visit_resolver_dir(nm_path: PathBuf, store_entry: bool, pending: &[Target]) -> ResolverVisit { + fn visit_resolver_dir( + nm_path: PathBuf, + store_entry: bool, + pending: &[Target], + ) -> ResolverVisit { let listing = list_dir_sync(&nm_path); let probe_filter = ProbeFilter::new(&listing); let matched = pending @@ -4329,6 +4362,30 @@ } } + #[test] + fn test_normalize_modules_folder() { + let n = normalize_modules_folder; + // Yarn-legal `./` prefixes and `.` segments reduce to the + // plain path; either separator is accepted. + assert_eq!(n("./deps").as_deref(), Some("deps")); + assert_eq!(n("./lib/deps").as_deref(), Some("lib/deps")); + assert_eq!(n("lib/./deps").as_deref(), Some("lib/deps")); + assert_eq!(n("lib\\deps").as_deref(), Some("lib/deps")); + assert_eq!(n("lib/../deps").as_deref(), Some("deps")); + assert_eq!(n("deps").as_deref(), Some("deps")); + // Escaping the project, reducing to it, or absolute — fail closed. + assert_eq!(n(".."), None); + assert_eq!(n("../elsewhere"), None); + assert_eq!(n("lib/../.."), None); + assert_eq!(n("."), None); + assert_eq!(n("a/.."), None); + assert_eq!(n("/etc/deps"), None); + assert_eq!(n("\\\\share\\deps"), None); + // A drive-letter segment survives normalization; the + // `is_safe_multi_segment` gate downstream rejects the colon. + assert_eq!(n("C:\\deps").as_deref(), Some("C:/deps")); + } + fn local_options(cwd: &Path) -> CrawlerOptions { CrawlerOptions { cwd: cwd.to_path_buf(),
You can send follow-ups to the cloud agent here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
The .yarnrc --modules-folder value comes from the project being scanned and names a tree apply writes patches into. An absolute or escaping value (/etc, ../elsewhere) is now ignored, like composer's vendor-dir, so the crawl and apply stay inside the project. The .yarnrc is also read with the regular-file guard, so a FIFO planted at that path can no longer hang scan, apply or vex. Assisted-by: Claude Code:claude-opus-5-5
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Stale Bugbot comment from a previous run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent]
No fix to port exists, since nothing in the diff reaches this path. I'll re-run the failed job once when the workflow run finishes. If it fails again I'll treat it as real and root-cause it. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] Ready for review at
Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Reviewed The default modules-folder and Rush paths are covered, but two valid Yarn Classic configurations still resolve to the wrong install root.
Validation: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Two valid yarn classic setups still pointed the crawler at the wrong install root, leaving the real tree unpatched and letting vex fall back to lockfile-only evidence: - A modules folder set in an ancestor .yarnrc is relative to that file's directory, not the project. /repo/.yarnrc with "--modules-folder project/deps" installs /repo/project into /repo/project/deps; the crawler looked in project/project/deps. - --install.modules-folder wins over --modules-folder regardless of line order or which .yarnrc defines each, because yarn merges each key through the rc files separately and applies install-scoped args last. The crawler took whichever line came last. The resolved folder must still lie strictly inside the project. Verified against yarn 1.22.22 installs for all three layouts. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_012pqLNDF3U1KUabGfPsxoZ8
|
[agent] Thanks, both findings are confirmed and fixed in ec95949. I checked yarn 1.22.22's own
Tests:
Both new crawl tests fail on 113ae51 ( Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
BugBot review Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit ec95949. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Follow-up reviewed Inherited Validation on this head: both original reviewer regression tests pass (2 passed), and |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
npm crawler never looks in Rush's common/temp store: hostedvex attests a transitive dep not_affected while its installed copy is unpatched, and agent apply reports it package_not_installed
Agent mode ignores yarn classic's --modules-folder: packages installed there are reported "not installed" and scan --mode agent exits 0 leaving them unpatched
There was an error while loading. Please reload this page.