GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — test(cli): give the Rust test suite its own package-manager caches by jdalton · Pull Request #143 · SocketDev/socket-patch Conversation
|
| Variable | Without it, this leaks |
|---|---|
npm_config_cache |
npm's package cache and its debug logs |
PNPM_HOME |
the pnpm store and global bin |
YARN_CACHE_FOLDER, YARN_GLOBAL_FOLDER |
yarn classic's cache, yarn berry's global cache |
COREPACK_HOME |
every package manager corepack downloads |
BUN_INSTALL, BUN_INSTALL_CACHE_DIR |
bun's install cache |
GOPATH, GOMODCACHE |
Go's downloaded modules |
GOCACHE |
Go's build cache — a different directory that follows neither of the two above |
CARGO_HOME |
the cargo registry |
PIP_CACHE_DIR, UV_CACHE_DIR |
pip's and uv's caches |
GEM_SPEC_CACHE, BUNDLE_USER_HOME |
the RubyGems spec cache and bundler's per-user state |
COMPOSER_HOME, COMPOSER_CACHE_DIR |
composer's home and cache |
NUGET_PACKAGES, NUGET_HTTP_CACHE_PATH |
the NuGet package and HTTP caches |
XDG_CACHE_HOME, XDG_DATA_HOME, XDG_STATE_HOME |
anything XDG-aware on Linux |
XDG_CONFIG_HOME is deliberately left alone. It is not a cache, and when someone has set it explicitly it usually points at real configuration — a registry mirror, a corporate CA bundle — that the installs still need.
This is the part that would have bitten people. rbenv, pyenv, nvm, fnm, volta, asdf, mise, sdkman and rustup all live under the home directory by default. Move HOME without carrying them and the shim cannot find its own root, so ruby or node or cargo fails to launch at all.
That failure would have been quiet rather than loud: most of these suites respond to a failed fixture install by printing SKIP and returning, so the coverage would have vanished without anything going red.
isolate() therefore seeds each root variable from the real home first — but only when the variable is not already exported and the default directory actually exists, which makes it a no-op on machines and CI runners that do not use that version manager. asdf and mise also read a global ~/.tool-versions and neither accepts an absolute path to it from the environment, so that one file is copied into the sandbox home.
Five self-tests in the module pin this: that every leak-prone variable is still in the list, that no override resolves inside the real home, that isolate() really applies them to a Command, and that the root-preservation pass never invents a path. They are pure path arithmetic, so they cost nothing in the binaries that pick the module up. They follow the same "ungated #[test] in a #[path] module" pattern the existing oracle_selftests in common/mod.rs uses, and for the same reason: integration-test crates never get cfg(test), so gating them would silently exclude them.
Command's environment operations are keyed by variable name and the last call for a name wins, so ordering decides who gets the final say:
SOCKET_* / npm_config_* / YARN_* prefix scrubs run first — they iterate the parent environment and would otherwise strip the sandbox values right back out;isolate();Step 3 is why the cold-cache tests keep working. Several suites deliberately pass an empty directory to prove a fresh checkout installs the patched bytes — e2e_vendor_golang_build.rs even asserts its empty GOMODCACHE stays empty afterwards. Those values are applied last, so they still win.
| File | What changed |
|---|---|
common/cache_env.rs |
New. The sandbox, isolate(), and the self-tests. |
common/mod.rs |
Re-exports the module; npm_run / pnpm_run / cargo_run and the has_command probe use it. |
e2e_golang_build.rs, e2e_vendor_golang_build.rs |
The reported gap: both set GOMODCACHE and stop, so go build still filled the real home. |
e2e_vendor_pypi_build.rs |
pip now gets what uv in the same file already had. |
e2e_vendor_{npm,bun,pnpm,yarn_berry,yarn_classic,yarn_classic_dev_flow,gem,composer}_build.rs |
Fixture installs and the corepack probes. |
e2e_redirect_{npm,bun,rush_sim,yarn_berry,yarn_classic}_build.rs |
Same, for the redirect capstones. |
in_process_alternate_installers.rs |
The reported gap: yarn / pnpm / npm / bundler installs, none #[ignore]d. |
in_process_gem_apply.rs |
The reported gap: gem install ran with an empty environment. |
global_packages_e2e.rs |
Two variables only — see the last fold. |
e2e_{npm,scan,gem,pypi}.rs |
#[ignore]d, so not part of a plain cargo test, but the dedicated e2e CI job runs them with --ignored. |
e2e_vendor_yarn_classic_dev_flow.rs ran its scrub in the wrong order. The scrub ends with env_remove("YARN_CACHE_FOLDER"), and the helper called it after seeding the caller's private cache — so the override was wiped and every fixture install quietly used the developer's global yarn cache. Measured: 165 files. e2e_vendor_yarn_classic_build.rs carries a comment describing having fixed exactly this; the dev-flow file, added later, kept the bug. The scrub now runs first.
The availability probes leaked more than the installs did. has_command("pnpm") looks harmless, but where pnpm or yarn is a corepack shim, pnpm --version makes corepack download the package manager — around 900 files from a single probe. has_command and has_corepack_pm are now isolated too. That has a second benefit: the probe now answers for the same environment the fixture install will run in, instead of a different one.
Method: build the test binaries normally, then run each one with HOME pointed at an empty stand-in directory and count the files in it afterwards. That proves the leak and the fix without writing to anyone's actual home. The same binaries were built from the base commit for the "before" column.
Ran — every suite this PR touches, before and after:
| Suite | Files written to the home directory, before | After |
|---|---|---|
e2e_golang_build |
438 (Go build cache) | 0 |
e2e_vendor_golang_build |
69 | 0 |
e2e_vendor_pnpm_build |
1,984 (corepack) | 0 |
in_process_alternate_installers |
926 (corepack, bundler, yarn, pnpm, npm) | 0 |
e2e_vendor_yarn_classic_dev_flow |
165 (global yarn cache) | 0 |
e2e_vendor_yarn_classic_build |
12 | 0 |
in_process_gem_apply |
3 (RubyGems spec cache) | 0 |
e2e_vendor_yarn_berry_build |
2 | 0 |
global_packages_e2e |
2 warm / 907 cold | 0 |
| everything else in the set | 0 | 0 |
| total for the set | 3,601 | 0 |
And the sandbox really is where it all went: the Go build cache and corepack directories under it are populated with exactly the content that used to land in the home directory.
Ran — the whole CLI integration suite on both sides, 136 test binaries each. Identical pass/fail, no status changes in either direction. One binary fails on my machine in both arms: in_process_alternate_installers's bundler leg looks for the gem under vendor/bundle/ruby/<version>/, which is the Bundler 2 layout; Bundler 1.17 puts it at vendor/bundle/. I confirmed it fails identically on the base commit with an untouched home, so it is pre-existing and unrelated.
Ran — cargo clippy --workspace --all-features --all-targets on the pinned toolchain. No new warnings from any file here. (Three pre-existing clippy findings in socket-patch-core reproduce identically on the base commit.) cargo fmt --check reports no diff in any file this PR touches.
Did not run — the Docker e2e suites and the setup matrix. Neither is affected: both already run inside containers whose home is thrown away, and the setup matrix is #142's subject.
Trade-off — the sandbox is a stable directory under the OS temp dir, not a fresh one per run. A throwaway directory would re-download the same handful of fixture packages every single time and buy no extra safety, since the sandbox is outside the home directory either way. The cost is that it persists between runs and grows to a couple of hundred megabytes, mostly the Go build cache and corepack. Note if you ever want to delete it by hand: Go writes its module cache read-only, so a plain rm -rf fails partway through with permission errors — run go clean -modcache against it first, or chmod -R u+w the tree. The module docs say so too.
Trade-off — the sandbox home means a fixture install no longer reads your ~/.npmrc. That is the intended direction (it is what makes the run reproducible), but if you install through a private registry configured only in your home npmrc, the fixture installs will now go to the public one.
Two suites still write to the home directory after this change, and the cause is not the tests: in_process_cargo_apply and in_process_remote_ecosystems_apply (~900 files each, cold). They call apply in process, and the crawler's global-prefix resolution shells out to npm / yarn / pnpm itself. Those children inherit the test process's environment, so the only fix is to set the variables on the test process rather than on a Command.
That is a real change of mechanism — this repo has already had to fix races caused by tests mutating the process environment (#135, #136) — so it does not belong in the same PR as a per-Command helper. global_packages_e2e.rs is the one place I did handle it here, and only partly: it spawns the binary as a child, so it can take COREPACK_HOME and npm_config_cache directly. It deliberately does not get the full isolate(), because resolving the real global prefixes is the entire point of that file and those come out of $HOME and PNPM_HOME. The two it does take cannot change the answer, and they take it from 907 files to 0.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit cb1c007. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| ("ASDF_DIR", ".asdf"), | ||
| ("ASDF_DATA_DIR", ".asdf"), | ||
| ("SDKMAN_DIR", ".sdkman"), | ||
| ("MISE_DATA_DIR", ".local/share/mise"), |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Medium Severity
isolate() always pins XDG_DATA_HOME to the sandbox, but mise root preservation only checks $HOME/.local/share/mise. Mise’s real default is ${XDG_DATA_HOME:-$HOME/.local/share}/mise, so when MISE_DATA_DIR is unset and mise lives under a custom parent XDG_DATA_HOME, shims lose their installs and fixture setup fails into silent SKIPs — the failure mode TOOLCHAIN_ROOTS is meant to prevent.
Reviewed by Cursor Bugbot for commit cb1c007. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
b38fe97 to
34a64e2
Compare
|
Heads up on the red check: Bundler falls back from the compact index to the dependency API against the gem patch-registry and gets a body it cannot unmarshal ( I A/B'd it on this branch:
The job last passed on |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Several suites under crates/socket-patch-cli/tests/ do real installs as
part of their fixture setup — npm, corepack yarn/pnpm, bun, go build,
pip, gem, bundler. None of them are #[ignore]d, so a plain `cargo test`
runs them all, and with no environment of their own each one writes into
the home directory of whoever ran the suite. Measured on macOS, one full
run of the CLI integration tests left 3,601 files there: corepack's
downloaded package managers, Go's build cache, yarn classic's global
cache, bundler's compact index, the RubyGems spec cache.
It also makes results depend on what happened to be lying around: a
fixture install can succeed against something an unrelated run already
cached, then fail on a clean CI runner.
tests/common/cache_env.rs adds `isolate()`, which points one child
process at a sandbox under the OS temp dir. It pins HOME plus every
variable that outranks HOME — GOCACHE (separate from GOMODCACHE and
GOPATH), COREPACK_HOME, PNPM_HOME, npm_config_cache, YARN_*, BUN_*,
CARGO_HOME, PIP_CACHE_DIR, UV_CACHE_DIR, GEM_SPEC_CACHE,
BUNDLE_USER_HOME, COMPOSER_*, NUGET_* — and carries over the version
manager roots (RUSTUP_HOME, RBENV_ROOT, MISE_DATA_DIR, ~/.tool-versions,
…) so a redirected home cannot make the toolchain itself unresolvable.
The sandbox is a stable directory, not a fresh one per run: these
fixtures reinstall the same handful of packages every time. Tests that
need a genuinely cold cache keep passing their own empty directory,
which still wins because `isolate()` runs before the test's own env.
Two things this turned up:
* e2e_vendor_yarn_classic_dev_flow.rs ran its ambient-env scrub AFTER
seeding the caller's YARN_CACHE_FOLDER. The scrub ends with
env_remove("YARN_CACHE_FOLDER"), so the private cache was wiped and
the fixture install silently used the developer's global one (165
files). e2e_vendor_yarn_classic_build.rs documents having fixed the
same bug; this file kept it.
* The `has_command` / `has_corepack_pm` availability probes leaked
more than the installs did. Where pnpm or yarn is a corepack shim,
`pnpm --version` downloads the package manager (~900 files). The
probes are now isolated too, which also makes them answer for the
environment the install will actually run in.
Same measurement after the change: 0 files. No test changed status —
136 test binaries, identical pass/fail before and after.
34a64e2 to
bb2bd5d
Compare
There was an error while loading. Please reload this page.
|
Follow-up to the note above: done, this PR no longer touches One correction to what I implied earlier: that does not make this PR green. What the split buys is a clean diff, not a green check. The failure write-up and the full A/B live in #144 now. Unblocking the check itself needs either the gem patch-registry fix or a repo admin setting |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
#142 + #143 + #144, plus review fixes) (#146) Squashes jdalton's cache-isolation stack — #142 (setup-matrix per-case HOME), #143 (cache_env::isolate() across the Rust integration suites), #144 (hosted production suite) — and folds in the fixes from a 13-agent adversarial review of the stack: - e2e_hosted_production.rs: widen the gem leg's known-defect matcher to the post-depscan#23630 registry signature (compact index 404s as not_built; the /api/v1/dependencies fallback serves an empty 200 body, which dies as "marshal data too short" under bundler 2.x classic Marshal and as "undefined method 'bytes' for nil" under SafeMarshal rubies). This unblocks the hosted-e2e required check, red on every branch since the 2026-08-02 deploy. Verified live: the leg passes against the current production registry. Also isolate the has_command probe (the corepack-download leak the stack closes everywhere else). - e2e_npm.rs: partial COREPACK_HOME/npm_config_cache pin on the binary runner — the macOS global auto-discovery test spawns the CLI, whose yarn/pnpm prefix probes downloaded ~900 corepack files into the real home (full isolate() would defeat the test; same trade-off as global_packages_e2e.rs). - e2e_vendor_gem_build.rs, e2e_vendor_pypi_build.rs: isolate the bundler/ruby/python/uv availability probes so probe and fixture install answer for the same environment. - run-case.sh: export MAVEN_OPTS=-Duser.home so maven actually follows the case home (the JVM derives user.home from passwd, not $HOME — mvn kept writing the real ~/.m2 while the verify helper scanned the empty fake one); preserve rbenv/pyenv/nvm/volta/asdf/sdkman roots and copy ~/.tool-versions so --host runs still launch their toolchains. - cache_env.rs: document why GEM_HOME/GEM_PATH are deliberately not pinned and the --install-dir/BUNDLE_PATH invariant that keeps that safe. Co-authored-by: jdalton <[email protected]> Co-authored-by: Claude Fable 5 <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.