GeezerVison is showing github.com — Perf regression: npm/hosted wall +15% (2463257a..9c43dfc9) · Issue #993 · SocketDev/socket-patch
Perf regression: npm/hosted wall +15% (2463257a..9c43dfc9) #993Description[agent] Bench: npm/hosted wall time is +15% to +20% over the last 7 days of main (CPU +12% to +14%, peak RSS +3% to +5%). npm/rescan, npm/dry-run and npm/public-proxy moved the same way. Request counts are unchanged (127 / 67 / 151).
There is no single culprit commit. The cost built up across the range, about half before and half after 045d7ec7. This is gradual drift, not one bad PR.
Same-machine interleaved compare (base 2463257a #277 → head 9c43dfc9, run 2026-10-07):
| scenario |
base wall |
head wall |
Δ wall [95% CI] |
Δ CPU |
Δ RSS |
requests |
npm/hosted (full run) |
287.9 ms |
317.5 ms |
+15.0% [+10.1, +17.8] |
+12.0% |
+3.1% |
127 |
npm/hosted (confirm, 15 pairs) |
271.9 ms |
321.8 ms |
+19.6% [+10.8, +25.6] |
+13.7% |
+4.8% |
127 |
npm/rescan (full run) |
270.5 ms |
314.2 ms |
+15.8% [+9.3, +23.5] |
+13.0% |
+6.7% |
127 |
npm/rescan (confirm) |
256.2 ms |
290.4 ms |
+12.6% [+7.5, +19.4] |
+11.8% |
+8.2% |
127 |
npm/dry-run |
272.6 ms |
297.0 ms |
+15.1% [+2.8, +18.4] |
+13.2% |
+3.1% |
67 |
npm/public-proxy |
272.4 ms |
302.7 ms |
+10.1% [+4.4, +16.5] |
+10.1% |
+3.0% |
151 |
An A/A run of the same binary at two paths on this runner was clean (npm/hosted -2.3%), so the result isn't runner noise. On 2026-10-06 the weekly ratio was already 1.100 (CI [-0.3, +15.9]); before that it held at 1.02-1.05.
Narrowing (one extra build, 045d7ec7, the main SHA measured 10-03 to 10-05):
| window |
npm/hosted |
npm/rescan |
2463257a..045d7ec7 (55 commits) |
+5.8% [-8.4, +11.6] |
+3.0% [-0.3, +16.4] |
045d7ec7..9c43dfc9 (74 commits) |
+10.6% [-3.7, +21.2] |
+4.8% [+0.3, +13.6] |
Neither half is significant alone, so bisecting further won't name one commit.
Profile (callgrind, npm/dry-run fixture, base vs head): total instructions went up 9.4%, from 948M to 1,036M Ir. The added work is in new npm lock passes during VEX discovery and the rewrite:
vex::discover::discover_with_ctx: +57M Ir. It includes vendor::npm_origin::npm_non_registry_entries (27M, new, from Fix npm rewiring git/URL/file lock entries (#326) #345 and Fix npm overrides of git deps being refused (#490) #491), vex::discover::npm::drop_non_registry_installs (18M, new, Fix npm rewiring git/URL/file lock entries (#326) #345) and vex::discover::npm::NpmLockRefs::mention (20M, new, Fix npm VEX attesting a patch the twin lock lacks (#798) #799).cli::commands::context::ProjectContext::locks (102M inclusive, new, Full Gradle support in agent, hosted and vendored modes #646): the shared lock cache now parses the lock through vendor::common::parse_json_manifest and JsonLayout.patch::redirect::rewrite_registry_redirect_withholding_vlt: +23M.
Candidates, in order: #799 (48085cef), #646 (0685ba8c), #345 (6e7ef748) and #491 (73b17db5). Each one re-walks the 3000-entry package-lock.json, so sharing one parsed lock walk across the VEX npm passes would likely win back most of this.
Repro
export CARGO_PROFILE_PERF_INHERITS=release CARGO_PROFILE_PERF_LTO=thin CARGO_PROFILE_PERF_STRIP=none
git worktree add /tmp/base 2463257a && (cd /tmp/base && CARGO_TARGET_DIR=/tmp/tb cargo build --locked --profile perf -p socket-patch-cli)
cargo build --locked --profile perf -p socket-patch-cli -p socket-patch-bench
target/perf/socket-patch-bench compare --base /tmp/tb/perf/socket-patch --head target/perf/socket-patch -f '^npm/hosted$' -f '^npm/rescan$' Runner: 4 vCPU, Intel(R) Xeon(R) Processor @ 2.80GHz (cloud sandbox), nproc = 4.
Why this was filed: the regression is confirmed (it held in a second round), and npm/hosted has measured ≥10% over the week window on two consecutive runs (10-06 1.100, 10-07 1.150/1.196). Progress is logged in #575.
Generated by Claude Code
Backlog review — 2026-10-08
Priority: P1 → P3. A roughly 15% microbenchmark regression at subsecond absolute runtime needs performance tracking, not P1 incident priority. Keep active #1008. Activitymikolalysenko commented on Oct 7, 2026 CollaboratorAuthor More actions [agent] Triaged: priority:p1 (npm-family). No duplicate or open PR covers it. Progress is tracked in #575. The profile points at repeated package-lock.json walks in VEX discovery (#799, #345, #491) and in the lock cache (#646). The suggested fix is to share one parsed lock walk across the npm VEX passes.
Generated by Claude Code mikolalysenko commented on Oct 7, 2026 CollaboratorAuthor More actions Being fixed in draft PR #1008 (batch fix for open pm:npm issues). mikolalysenko commented on Oct 8, 2026 CollaboratorAuthor More actions [agent] Bench: run 2026-10-08. main is now ea097142 (#1044). The npm drift is smaller today and below the gate on npm/hosted, but npm/dry-run still confirms.
Weekly same-machine interleaved compare, base 2463257a (#277) → head ea097142, on a 4 vCPU Xeon @ 2.10GHz. A/A was clean.
| scenario |
base wall |
head wall |
Δ wall [95% CI] |
Δ CPU |
Δ RSS |
verdict |
npm/hosted |
205.8 ms |
215.3 ms |
+4.5% [-3.2, +7.2] |
+6.0% |
+5.7% |
≈ |
npm/rescan (round 1) |
225.1 ms |
238.1 ms |
+6.8% [+2.7, +19.5] |
+11.3% |
+8.8% |
flagged |
npm/rescan (round 2, 30+ pairs) |
190.7 ms |
210.2 ms |
+8.9% [+5.6, +12.9] |
+8.8% |
+9.2% |
≈ (not confirmed) |
npm/dry-run (round 1) |
183.0 ms |
209.4 ms |
+14.7% [+10.4, +17.2] |
+10.8% |
+5.8% |
flagged |
npm/dry-run (round 2, 30+ pairs) |
211.3 ms |
238.9 ms |
+10.0% [+6.1, +14.5] |
+9.4% |
+5.6% |
confirmed |
npm/public-proxy |
201.2 ms |
219.5 ms |
+4.4% [+1.0, +13.4] |
+5.9% |
+5.8% |
≈ |
The daily A/B (9c43dfc9 → ea097142) shows no change on any npm scenario: hosted -3.2%, rescan +1.4%, dry-run +2.6%. Request counts are unchanged. The overhead is still there, about +5-10% wall and CPU and about +6-9% RSS on every npm scan, but it is no longer above 10% on npm/hosted. #1008 is the open fix.
Repro: socket-patch-bench compare --base <2463257a build> --head <ea097142 build> -f '^npm/(hosted|rescan|dry-run)$' --runs 30
Generated by Claude Code mikolalysenko commented on Oct 12, 2026 CollaboratorAuthor More actions [agent] Bench: run 2026-10-12. The npm wall-time drift has measured back within 5% of the pre-drift baseline for 2 consecutive runs, so I'm closing this issue.
Same-machine interleaved compare, base 61cfb9b2 → head 17793173 (today's main), 15 pairs, on a 4 vCPU Xeon @ 2.10GHz. A/A was clean. 61cfb9b2 comes from the same day as the original base 2463257a, 17 core/CLI commits later, and both predate the drift window.
| scenario |
base wall |
head wall |
Δ wall [95% CI] |
Δ CPU |
Δ RSS |
requests |
npm/hosted |
212.6 ms |
209.4 ms |
-1.4% [-4.5, +2.2] |
-0.9% |
+26.1% |
127 |
npm/rescan |
193.2 ms |
155.7 ms |
-19.8% [-29.2, -14.6] |
-16.1% |
+11.1% |
127 |
npm/dry-run |
200.6 ms |
197.6 ms |
-1.5% [-7.9, +4.9] |
-1.2% |
+24.2% |
67 |
Run 1 was 2026-10-09: npm/hosted -0.1% vs 61cfb9b2, and npm/dry-run wall did not confirm. The peak-RSS growth (+24–26%) is tracked separately in the bench ledger. It is below the 30% RSS issue gate (#1058 and ea097142..ef48495c).
Generated by Claude Code
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|