GeezerVison

LLM Description written by Claude Code:claude-opus-5-5

Fixes #409

Summary

A venv made with --system-site-packages imports packages from its base interpreter (Debian dist-packages, the base site-packages, the user site). pip treats a release found there as already installed and leaves it in place. socket-patch only looked inside the venv, so:

• vex thought nothing was installed and attested not_affected from the lock pin, even though the code that runs is not patched.
• The hosted and vendored stale-install guards stayed silent.
• Agent mode said "not installed; run your package manager's install first", which can't help because the install has nothing to do.

This covers plain python -m venv --system-site-packages / virtualenv, Poetry's virtualenvs.options.system-site-packages, and Pipenv's --site-packages / PIPENV_SITE_PACKAGES. All of them write include-system-site-packages = true into pyvenv.cfg.

Root cause

Local Python discovery (find_local_venv_site_packages, non_hatch_local_venv_site_packages, and through them PythonCrawler::get_site_packages_paths) returned only the venv's own site-packages. It never read pyvenv.cfg, so the base interpreter's site dirs behind a system-site venv were never crawled.

Fix

• Discovery (add_system_site_packages): for each discovered venv whose pyvenv.cfg enables system site-packages, the base interpreter's site dirs are appended after the venv's own, in sys.path order:

  • the user site, unless PYTHONNOUSERSITE is set;
  • Debian's /usr/local/lib/pythonX.Y/dist-packages when the prefix is /usr;
  • <prefix>/lib{64,}/pythonX.Y/site-packages;
  • <prefix>/lib/python3/dist-packages.

  The prefix comes from home, following a symlinked interpreter to its stdlib landmark (Homebrew, Debian /usr/local/bin/python3 -> /usr/bin/python3.11). Only the venv's own X.Y (including a t free-threaded suffix) is probed. Everything is derived from pyvenv.cfg and the filesystem: the project's interpreter is never executed, because it is untrusted project content. pyvenv.cfg is parsed like CPython's site.venv(): keys are case-insensitive, the last one wins, and a missing key counts as true.

• Shadowing (PythonCrawler::set_search_order): Python imports the first copy of a name on sys.path, and pip treats that copy as the installed one. So a base copy is hidden when the venv (or an earlier base dir) installs the same name. That case is common after pip install --force-reinstall in the venv, which leaves the base copy in place. Without hiding it, the stale guard would keep warning, vex would refuse forever, and agent apply would try to write into /usr/lib/python3/dist-packages. A base dir that several venvs share hides only what every one of them hides. get_site_packages_paths declares its own order. The two call sites that use the free discovery functions, the hosted stale probe and the vendored Pipenv stale probe, declare it explicitly. Global mode clears it, so every copy stays visible there.

Tests (red → green)

I also checked the derived dirs by hand against a real Debian python3 -m venv --system-site-packages (Python 3.11): sys.path = venv site-packages, /usr/local/lib/python3.11/dist-packages, /usr/lib/python3/dist-packages, which matches the discovery order.

Local checks: cargo fmt --all -- --check ✅, cargo clippy --workspace --all-features -- -D warnings ✅, cargo test --workspace --all-features ✅ except tests that fail only because this sandbox runs as root, so permission-denied setups (chmod 0555, unremovable files) don't take effect. These are the *write_failure* / *unremovable* / *cleanup_failure* tests in covgap_commands_vendor, in_process_redirect, repair and the core lib (copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry/pypi_requirements wire-failure tests). There is also e2e_bun_lockb::binary_shared_bundled_record_hosted_pin_is_managed, which fails the same way on origin/main here. None of them involve Python discovery, and CI runs them green on this head. The Python e2e suite (e2e_pypi) needs the live API, so CI covers it.

CI: green on f1ba00d (36 pass / 50 path-skipped). Bugbot: no issues on f1ba00d.

Decisions and follow-ups (for reviewers)

• Agent mode now patches the base copy that a system-site venv actually imports. On a distro interpreter that write is root-owned and fails loudly (EACCES), where before it was silently skipped. On a writable interpreter (pyenv, Homebrew, user site), it patches a copy other projects on that interpreter also import. This follows the existing rule that a Python project with no venv falls back to the global interpreters. If maintainers prefer agent mode to refuse instead of writing outside the venv, that's a small follow-up.
• /code-review high findings not fixed here:
  • Shadow state lives on the PythonCrawler instance, not in the discovery return type. That would be a cleaner API, but it would change every Python consumer's signature. get_site_packages_paths resets the state, and each of the two free-function callers sets it.
  • set_search_order lists the venv and base dirs once more, only when a system-site venv exists.
  • The vendored Hatch stale probe (hatch_stale_install_warning) still judges only Hatch's own env dirs. A Hatch env with system-packages = true gets the base copy judged by the hosted guard (generic remedy), but not by the vendored Hatch-specific warning.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MPfWuTkaymyECuH3w5YJ49

Generated by Claude Code