GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||
GeezerVison is showing github.com — feat(vendor,scan): lockfile-driven discovery + verified auto-fetch of missing packages by mikolalysenko · Pull Request #113 · SocketDev/socket-patch Conversation
Collaborator
SummaryTwo capabilities that make 1. Vendor auto-fetch (no flag — automatic)A manifest patch whose package has no installed copy no longer fails with
Anything the lock can't verify is never fetched ( Hardening: http(s)-only, download/decompression/entry-count/entry-size caps, regular-files-only extraction with traversal guards (fail-closed, nothing half-extracts), exec bits preserved (the deterministic re-pack reads modes from disk). 2. Scan lockfile supplement (default-on)
Test plan
Out of scope (documented)
🤖 Generated with Claude Code
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactionsRead-only inventories of the dependency set a lockfile resolves, independent of what is installed: name/version/purl plus the lock's artifact URL and content verifier (typed LockIntegrity: SRI, yarn sha1 fragment, berry cache-zip checksum, sha256 hex, go.sum h1 — the latter two for the ecosystems that follow). Powers scan's lockfile supplement and vendor's missing-package fetch. Covers all five npm flavors via detect_npm_lock_flavor (package-lock/ shrinkwrap, pnpm v9, yarn classic, yarn berry, bun). Fail-soft per entry, fail-closed per value (names/versions path-guarded; git/file/ link/workspace specs and our own vendored entries excluded; duplicate instances dedup preferring a verifier). lookup() bridges percent- encoded manifest purls. Reuses the wiring backends' parsers via pub(super) visibility bumps. Co-Authored-By: Claude Fable 5 <[email protected]> Downloads the artifact a lockfile entry resolves (lock-recorded URL, else the conventional npm registry URL; SOCKET_NPM_REGISTRY override), verifies it against the lock-recorded integrity FAIL-CLOSED before any disk write (strongest hash of a multi-hash SRI; yarn sha1 fragment; sha256 hex), and extracts to a private tempdir the vendor pipeline can stage from. Entries with no verifier are refused without any network I/O (Unverifiable). Hardening: http(s)-only, download/decompression/entry-count/entry-size caps, regular-files-only extraction with first-component strip + is_safe_relative_subpath (fail-closed on traversal-bearing tarballs, nothing half-extracts), exec bits preserved so the deterministic re-pack keeps bin scripts executable. Co-Authored-By: Claude Fable 5 <[email protected]> …overy
vendor: a manifest patch whose package has no installed copy is now
satisfied automatically (no flag) instead of failing with
package_not_installed:
- already-vendored purls stage from their own committed artifact,
sha256-verified against the vendor ledger (fresh-clone re-vendor and
in-sync runs work offline, no registry traffic);
- otherwise the lockfile-resolved pristine artifact is fetched
(lock-recorded URL else conventional registry URL), verified against
the lock's integrity FAIL-CLOSED, and staged from a private tempdir —
the project tree is never touched.
Reason codes: vendor_fetched_missing (skip-warning beside the Applied
event), vendor_fetch_failed (distinct Failed, suppresses the duplicate
not-installed skip), vendor_fetch_unverifiable (no lock integrity →
calm skip). --offline keeps the calm skip and names the lockfile as
the would-be source.
scan: discovery now supplements the installed-tree crawl with
(a) lockfile-only dependencies — warned '[NOT INSTALLED]' in the table
+ a stderr note, JSON lockfileOnlyPackages + packages[].notInstalled,
counted as scanned so a wiped node_modules no longer prunes
lockfile-listed entries, partitioned out of --apply BEFORE download
(calm skipped/package_not_installed records, exit 0, no manifest
writes) while --vendor passes them to the auto-fetch; and
(b) vendored-ledger entries — the committed artifact IS the dependency,
so updates[] detection and scan --vendor keep working on a fresh
clone before any install.
scan --json --vendor now vendors a completely fresh clone end-to-end
(e2e-proven, second run already_vendored).
Co-Authored-By: Claude Fable 5 <[email protected]>
…aging tests Berry locks never hash the tarball — the checksum is sha512 of the deterministic cache zip. The fetch rebuilds that zip from the fetched bytes via the same spike-pinned berry_zip recipe the wiring uses and compares the 10c0/<hex> value fail-closed (foreign cacheKeys are Unverifiable). Plus unit coverage for stage_local_artifact's ledger-sha256 gate. Co-Authored-By: Claude Fable 5 <[email protected]> - Cargo.lock [[package]] inventory: crates.io-sourced entries carry
their sha256 .crate checksum (Sha256Hex); workspace members skipped,
git/custom-registry sources discovery-only. Fetch from
static.crates.io (SOCKET_CRATES_REGISTRY override), verify, extract
({name}-{version}/ top dir) — feeds vendor_cargo_crate's pristine_src.
- go.sum inventory: module-zip h1: lines (the /go.mod manifests-only
lines skipped). Fetch from the module proxy (SOCKET_GOPROXY, else the
standard GOPROXY's first non-direct element, else proxy.golang.org)
with Go's !uppercase path escaping; verify the dirhash Hash1/HashZip
in memory BEFORE extraction (algorithm validated against a live
sum.golang.org lookup for golang.org/x/[email protected]); extract with
the explicit module@version/ prefix (module paths contain slashes, so
a first-component strip would be wrong) — feeds vendor_go_module.
- lookup() generalized across ecosystems; inventory_project() returns
the union the scan supplement and vendor auto-fetch consume.
Co-Authored-By: Claude Fable 5 <[email protected]>
…etch - composer.lock packages[]/packages-dev[]: zip dists with their sha1 shasum (frequently empty → discovery-only); names lowercased to the packagist form, pretty leading v dropped; path dists (ours) excluded. Fetch verifies sha1 and strips the variable zipball top dir. - Gemfile.lock GEM/specs + bundler 2.6 CHECKSUMS sha256 (older locks discovery-only); the GEM remote drives the /downloads/<gem> URL. Platform-suffixed specs skipped (unsupported for vendoring). The fetched .gem (plain tar) is sha256-verified whole, then data.tar.gz extracts at the root (no prefix strip). - pypi: uv.lock registry packages with a pure py3-none-any wheel carry a fetchable URL + sha256; poetry.lock and ==-pinned requirements.txt contribute discovery-only entries (PEP 503-normalized names). The unzipped wheel is a site-packages-shaped stage for the pypi backend. Co-Authored-By: Claude Fable 5 <[email protected]> scan's lockfile supplement now consumes inventory_project (npm-family, Cargo.lock, go.sum, composer.lock, Gemfile.lock, uv/poetry/requirements) with per-ecosystem counts; the vendor auto-fetch pass likewise serves every inventoried ecosystem. CLI_CONTRACT.md gains the lockfile- supplement and vendor-auto-fetch sections + the three reason codes; README notes the fresh-clone flow; the exact-shape empty-scan contract test pins the additive lockfileOnlyPackages field; the cargo build e2e scrubs ambient CARGO_TARGET_DIR from child builds. Co-Authored-By: Claude Fable 5 <[email protected]>
This was referenced
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ReviewersNo reviewsUh oh!
There was an error while loading. Please reload this page. AssigneesNo one assignedLabels
None yet
ProjectsMilestoneNo milestoneDevelopmentSuccessfully merging this pull request may close these issues. Uh oh!
There was an error while loading. Please reload this page. Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later. | ||||||||||||||||||||||||
| You are visitor 01402520 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |