GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||
GeezerVison is showing github.com — fix(gem): bundler vendored+hosted audit — 13 confirmed bugs fixed, tests hardened by mikolalysenko · Pull Request #175 · SocketDev/socket-patch Conversation
|
|
Both Bugbot findings addressed in d063cb1:
Note: took the guard-relocation fix rather than the prepared one-line autofix so the interactive and JSON-apply arms are covered too. 🤖 Generated with Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…sts hardened ULTRACODE review + full test matrix over the gem/bundler ecosystem (vendored and hosted modes, every configuration). 24 findings survived adversarial verification; the 13 well-scoped ones are fixed here, each pinned by a test that fails without its fix. Hosted redirect (patch/redirect/mod.rs, gem section): - splice the Gemfile edit by regex byte range: a commented-out duplicate of the gem line no longer gets rewritten instead of the live line - grant-agnostic idempotency guard: re-running scan --mode hosted with a rotated grant token refreshes the source URL in place instead of nesting a second source block (new edit kind redirect_gemfile_source_url) - fail closed on gem-level git:/github:/path:/source: options (they override the enclosing source block, making the redirect an attested no-op); warn redirect_gem_source_option and skip the dep - fail closed on platform-suffixed CHECKSUMS siblings (redirect_gem_platform_unsupported) instead of inserting a duplicate bare-coordinate pin - recognize paren/tab/multi-space gem declarations; gate the append-branch on the gem being genuinely undeclared (no more duplicate declarations bundler rejects) - never pin the lock CHECKSUMS when the Gemfile source redirect did not land (mixed state guaranteed a checksum failure) - warn that a redirected pair breaks frozen/deployment installs - record the upstream sha256 line as original on the CHECKSUMS edit so a future revert can restore it (golden fixture updated) Vendored backend (vendor/gem.rs): - insert new PATH sections at bundler's sorted position (identifier order, verified against real bundler 4.0.15 bundle lock) — two or more vendored gems no longer churn the committed lock - fail closed on platform-suffixed GEM-specs siblings on no-CHECKSUMS locks (mirrors the existing CHECKSUMS guard) - re-vendor on a patch UPDATE (new uuid, same purl): recognize our own path: wiring and rewire in place instead of refusing with gemfile_declaration_not_editable — the documented automatic re-vendor contract now actually works for gem Auto-fetch (vendor/registry_fetch.rs): - stage fetched gems into the canonical <name>-<version> leaf instead of a dir literally named "gem", which vendor_gem refused as platform_gem_unsupported — lockfile auto-fetch for gems was dead Crawler (crawlers/ruby_crawler.rs): - parse_dir_name_version prefers the last dotted-version boundary, so http-2-1.0.1 parses as (http-2, 1.0.1) instead of the ghost (http, 2) - vendor/bundle discovery enumerates engine dirs (jruby, truffleruby) instead of hardcoding ruby/ Scan/get plumbing: - run_nested_apply now threads --ecosystems: scan --ecosystems gem --sync no longer applies (or mutates) other ecosystems' patches - scan --vendor --dry-run --vex no longer writes the VEX file nor exits 1 on not-yet-vendored state Test hardening: - e2e_gem lifecycle harness: BUNDLE_PATH replaces bundle install --path (removed in bundler 3+; all 3 lifecycle tests green under 4.0.15) - e2e_hosted_production gem leg now asserts the CHECKSUMS digest CHANGED after redirect (client-verifiable today) and, in the success arm, verifies installed content against the published afterHashes — an inert gem patch can no longer stay green (the npm minimist blindspot) - docker_e2e_gem serves the true git-blob beforeHash so the chain exercises the default non-forced apply path, not just --force - setup_matrix_gem module doc: the with-setup Docker cases ARE still a baseline gap (bootstrap deadlock: plugin registration evaluates plugins.rb before any gems land; exit-semantics twin), doc corrected Verified: core 2460/0, cli lib 350/0, clippy+fmt clean; e2e_gem 11/11 (incl. live lifecycle under bundler 4.0.15), e2e_vendor_gem_build 6/6 (incl. real-bundler capstone), docker_e2e_gem 2/2, docker_e2e_vendor_gem 1/1, hosted production gem leg green (redirect verified; install still blocked by the known depscan#23630 compact-index 404 — server-side). Co-Authored-By: Claude Fable 5 <[email protected]>
…ll scan arms Addresses both Bugbot findings on #175: - The grant-agnostic Gemfile idempotency guard required LF (`do\n`), so a core.autocrlf checkout of a previously-redirected Gemfile was not recognized and the indented gem line inside the block got wrapped in a second, nested source block on re-run. The recognizer now accepts `do\r?\n`; pinned by gemfile_rerun_on_crlf_checkout_never_nests (verified red without the fix). - The dry-run VEX skip only covered the vendor JSON arm; the interactive `scan --vendor --dry-run --vex` path (embed_vex_human) and the JSON `scan --apply --dry-run --vex` path (embed_vex_into_json at the apply fall-through) still generated the document — exiting 1 on a not-yet-vendored project or writing the attestation during --dry-run. The guard now lives at the top of both embed helpers, covering every scan arm; pinned by scan_vendor_dry_run_with_vex_interactive_* and scan_apply_json_dry_run_with_vex_* (both verified red without it). Co-Authored-By: Claude Fable 5 <[email protected]>
d063cb1 to
99f9cd0
Compare
|
Rebased Conflicts resolved (1):
Auto-merges verified semantically:
Verification (all green):
🤖 Generated with Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
1640912
into
main
There was an error while loading. Please reload this page.
…dor coverage; gems.rb + CRLF redirect fixes (#177) * fix(gem): bundler vendored+hosted sweep — 13 confirmed bugs fixed, tests hardened ULTRACODE review + full test matrix over the gem/bundler ecosystem (vendored and hosted modes, every configuration). 24 findings survived adversarial verification; the 13 well-scoped ones are fixed here, each pinned by a test that fails without its fix. Hosted redirect (patch/redirect/mod.rs, gem section): - splice the Gemfile edit by regex byte range: a commented-out duplicate of the gem line no longer gets rewritten instead of the live line - grant-agnostic idempotency guard: re-running scan --mode hosted with a rotated grant token refreshes the source URL in place instead of nesting a second source block (new edit kind redirect_gemfile_source_url) - fail closed on gem-level git:/github:/path:/source: options (they override the enclosing source block, making the redirect an attested no-op); warn redirect_gem_source_option and skip the dep - fail closed on platform-suffixed CHECKSUMS siblings (redirect_gem_platform_unsupported) instead of inserting a duplicate bare-coordinate pin - recognize paren/tab/multi-space gem declarations; gate the append-branch on the gem being genuinely undeclared (no more duplicate declarations bundler rejects) - never pin the lock CHECKSUMS when the Gemfile source redirect did not land (mixed state guaranteed a checksum failure) - warn that a redirected pair breaks frozen/deployment installs - record the upstream sha256 line as original on the CHECKSUMS edit so a future revert can restore it (golden fixture updated) Vendored backend (vendor/gem.rs): - insert new PATH sections at bundler's sorted position (identifier order, verified against real bundler 4.0.15 bundle lock) — two or more vendored gems no longer churn the committed lock - fail closed on platform-suffixed GEM-specs siblings on no-CHECKSUMS locks (mirrors the existing CHECKSUMS guard) - re-vendor on a patch UPDATE (new uuid, same purl): recognize our own path: wiring and rewire in place instead of refusing with gemfile_declaration_not_editable — the documented automatic re-vendor contract now actually works for gem Auto-fetch (vendor/registry_fetch.rs): - stage fetched gems into the canonical <name>-<version> leaf instead of a dir literally named "gem", which vendor_gem refused as platform_gem_unsupported — lockfile auto-fetch for gems was dead Crawler (crawlers/ruby_crawler.rs): - parse_dir_name_version prefers the last dotted-version boundary, so http-2-1.0.1 parses as (http-2, 1.0.1) instead of the ghost (http, 2) - vendor/bundle discovery enumerates engine dirs (jruby, truffleruby) instead of hardcoding ruby/ Scan/get plumbing: - run_nested_apply now threads --ecosystems: scan --ecosystems gem --sync no longer applies (or mutates) other ecosystems' patches - scan --vendor --dry-run --vex no longer writes the VEX file nor exits 1 on not-yet-vendored state Test hardening: - e2e_gem lifecycle harness: BUNDLE_PATH replaces bundle install --path (removed in bundler 3+; all 3 lifecycle tests green under 4.0.15) - e2e_hosted_production gem leg now asserts the CHECKSUMS digest CHANGED after redirect (client-verifiable today) and, in the success arm, verifies installed content against the published afterHashes — an inert gem patch can no longer stay green (the npm minimist blindspot) - docker_e2e_gem serves the true git-blob beforeHash so the chain exercises the default non-forced apply path, not just --force - setup_matrix_gem module doc: the with-setup Docker cases ARE still a baseline gap (bootstrap deadlock: plugin registration evaluates plugins.rb before any gems land; exit-semantics twin), doc corrected Verified: core 2460/0, cli lib 350/0, clippy+fmt clean; e2e_gem 11/11 (incl. live lifecycle under bundler 4.0.15), e2e_vendor_gem_build 6/6 (incl. real-bundler capstone), docker_e2e_gem 2/2, docker_e2e_vendor_gem 1/1, hosted production gem leg green (redirect verified; install still blocked by the known depscan#23630 compact-index 404 — server-side). Co-Authored-By: Claude Fable 5 <[email protected]> * fix(review): CRLF-tolerant source-block guard; dry-run VEX skip for all scan arms Addresses both Bugbot findings on #175: - The grant-agnostic Gemfile idempotency guard required LF (`do\n`), so a core.autocrlf checkout of a previously-redirected Gemfile was not recognized and the indented gem line inside the block got wrapped in a second, nested source block on re-run. The recognizer now accepts `do\r?\n`; pinned by gemfile_rerun_on_crlf_checkout_never_nests (verified red without the fix). - The dry-run VEX skip only covered the vendor JSON arm; the interactive `scan --vendor --dry-run --vex` path (embed_vex_human) and the JSON `scan --apply --dry-run --vex` path (embed_vex_into_json at the apply fall-through) still generated the document — exiting 1 on a not-yet-vendored project or writing the attestation during --dry-run. The guard now lives at the top of both embed helpers, covering every scan arm; pinned by scan_vendor_dry_run_with_vex_interactive_* and scan_apply_json_dry_run_with_vex_* (both verified red without it). Co-Authored-By: Claude Fable 5 <[email protected]> * test(gem): hermetic hosted capstone + CHECKSUMS/transitive/scan--vendor coverage; gems.rb + CRLF redirect fixes Closes the gem audit's top coverage gaps with hardened, independent-oracle tests, plus the two small redirect fixes the new tests exercise. Redirect fixes (patch/redirect/mod.rs gem section + hosted.rs candidates): - gems.rb/gems.locked support end-to-end: the candidate list now reads the modern pair and the gem rewriter keys on whichever pair bundler reads (gems.rb wins when both spellings exist — verified on bundler 4.0.15; same order as setup::gem). Diverging spellings fail closed with redirect_gem_gemfile_spellings_diverge. Before: a gems.rb project was a silent hosted-mode no-op. - CRLF Gemfile.lock tolerance: the CHECKSUMS matchers accept \r-terminated lines and edits preserve the file's CRLF endings byte-for-byte. Before: a CRLF lock (legal to bundler — verified via bundle check/frozen install) was misdiagnosed as bundler <2.6 (redirect_gem_no_checksums_section). New coverage: - e2e_redirect_gem_build.rs — FULLY HERMETIC hosted gem capstone: authored gems built with real `gem build`, one wiremock playing the upstream compact index, the Socket patch-registry compact index (production's /patch-registry/gem/<token>/<uuid>/ shape, real /versions md5s and /info checksums), and the patches API. scan --mode hosted, then a REAL fresh-checkout `bundle install` (host bundler) pulls the patched .gem: bytes match afterHash, the runtime dep installs because /info declares it, require probe + post-install verified VEX. A gems.rb twin pins the modern spelling end-to-end. A deps red-arm pins the compact-index dependency contract the production server currently violates: a deps-less /info breaks the install with bundler's APIResponseMismatchError (the live-CI signature). A KNOWN-LIMITATION canary pins that on a CHECKSUMS lock (bundler >= 4 default) the current rewrite makes the prescribed unfrozen install fail with "mismatched checksums" (empirically verified; the converged-lock fix shape that a frozen install accepts is documented in the test). - docker_e2e_vendor_gem.rs — lockfile_checksums twin (bundler 2.7 in the image, `bundle lock --add-checksums`): vendor swaps the registry sha256= CHECKSUMS line for bundler's bare path-gem form, a frozen --network none install accepts the rewritten lock byte-stably, and revert restores the registry sha256= line VERBATIM (the exit-16 hazard). - e2e_vendor_gem_build.rs — TRANSITIVE-dep capstone: vendoring rack via a rack-test project appends the managed block + sorted `rack (= v)!` DEPENDENCIES pin, a real frozen fresh-checkout install accepts the pair byte-stably, require probe through the dependent, revert round-trip. - in_process_vendor.rs — gem through `scan --vendor` (mock-proxy API, hermetic bundler layout, no ruby needed): JSON-arm end-to-end (pair edit + artifact + stub gemspec + already_vendored rerun), manifest-drop reconcile byte-restore, and the --detached variant (no manifest, embedded record, vendor --revert exit path). Every rewriter behavior change is pinned by unit tests that fail on the base commit (gems.rb pair routing, diverging-spelling fail-closed, CRLF pin-in-place/insert byte-preservation, CRLF rerun no-op); the gems.rb e2e leg fails without the candidate-list entry. Bundler behaviors (gems.rb preference, lockfile naming, compact-index formats, checksum enforcement, CRLF acceptance) were validated against real bundler 4.0.15 and the image's 2.7.2 before being baked into fixtures. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(redirect): judge gems.rb/Gemfile divergence on the redirect footprint residue, not raw bytes Run 1 on byte-identical gems.rb/Gemfile twins edits only gems.rb (the file bundler reads), so the raw-byte divergence guard trapped every later run — the rotated-grant URL refresh included — behind redirect_gem_gemfile_spellings_diverge, a divergence the rewriter itself created. Compare redirect-footprint residues instead: erase the managed Socket source block (rotating grant segments wildcarded) and the dep's own gem declaration line from both spellings before judging. Rebased onto d063cb1 (CRLF-tolerant block recognizer): the residue eraser now accepts `do\r?\n … end\r?` too, mirroring the recognizer — a core.autocrlf checkout rewrites run 1's LF block to CRLF, and a block the recognizer accepts must also be erased here or the re-run is trapped behind the divergence warning before it can reach the recognizer. Pinned by gems_rb_crlf_twins_rerun_is_no_op_and_rotated_grant_refreshes (verified red with an LF-only residue eraser). Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
…on gem/pypi/maven variants; variant-scope the rollback before-blob gate (#176) * fix(gem): bundler vendored+hosted sweep — 13 confirmed bugs fixed, tests hardened ULTRACODE review + full test matrix over the gem/bundler ecosystem (vendored and hosted modes, every configuration). 24 findings survived adversarial verification; the 13 well-scoped ones are fixed here, each pinned by a test that fails without its fix. Hosted redirect (patch/redirect/mod.rs, gem section): - splice the Gemfile edit by regex byte range: a commented-out duplicate of the gem line no longer gets rewritten instead of the live line - grant-agnostic idempotency guard: re-running scan --mode hosted with a rotated grant token refreshes the source URL in place instead of nesting a second source block (new edit kind redirect_gemfile_source_url) - fail closed on gem-level git:/github:/path:/source: options (they override the enclosing source block, making the redirect an attested no-op); warn redirect_gem_source_option and skip the dep - fail closed on platform-suffixed CHECKSUMS siblings (redirect_gem_platform_unsupported) instead of inserting a duplicate bare-coordinate pin - recognize paren/tab/multi-space gem declarations; gate the append-branch on the gem being genuinely undeclared (no more duplicate declarations bundler rejects) - never pin the lock CHECKSUMS when the Gemfile source redirect did not land (mixed state guaranteed a checksum failure) - warn that a redirected pair breaks frozen/deployment installs - record the upstream sha256 line as original on the CHECKSUMS edit so a future revert can restore it (golden fixture updated) Vendored backend (vendor/gem.rs): - insert new PATH sections at bundler's sorted position (identifier order, verified against real bundler 4.0.15 bundle lock) — two or more vendored gems no longer churn the committed lock - fail closed on platform-suffixed GEM-specs siblings on no-CHECKSUMS locks (mirrors the existing CHECKSUMS guard) - re-vendor on a patch UPDATE (new uuid, same purl): recognize our own path: wiring and rewire in place instead of refusing with gemfile_declaration_not_editable — the documented automatic re-vendor contract now actually works for gem Auto-fetch (vendor/registry_fetch.rs): - stage fetched gems into the canonical <name>-<version> leaf instead of a dir literally named "gem", which vendor_gem refused as platform_gem_unsupported — lockfile auto-fetch for gems was dead Crawler (crawlers/ruby_crawler.rs): - parse_dir_name_version prefers the last dotted-version boundary, so http-2-1.0.1 parses as (http-2, 1.0.1) instead of the ghost (http, 2) - vendor/bundle discovery enumerates engine dirs (jruby, truffleruby) instead of hardcoding ruby/ Scan/get plumbing: - run_nested_apply now threads --ecosystems: scan --ecosystems gem --sync no longer applies (or mutates) other ecosystems' patches - scan --vendor --dry-run --vex no longer writes the VEX file nor exits 1 on not-yet-vendored state Test hardening: - e2e_gem lifecycle harness: BUNDLE_PATH replaces bundle install --path (removed in bundler 3+; all 3 lifecycle tests green under 4.0.15) - e2e_hosted_production gem leg now asserts the CHECKSUMS digest CHANGED after redirect (client-verifiable today) and, in the success arm, verifies installed content against the published afterHashes — an inert gem patch can no longer stay green (the npm minimist blindspot) - docker_e2e_gem serves the true git-blob beforeHash so the chain exercises the default non-forced apply path, not just --force - setup_matrix_gem module doc: the with-setup Docker cases ARE still a baseline gap (bootstrap deadlock: plugin registration evaluates plugins.rb before any gems land; exit-semantics twin), doc corrected Verified: core 2460/0, cli lib 350/0, clippy+fmt clean; e2e_gem 11/11 (incl. live lifecycle under bundler 4.0.15), e2e_vendor_gem_build 6/6 (incl. real-bundler capstone), docker_e2e_gem 2/2, docker_e2e_vendor_gem 1/1, hosted production gem leg green (redirect verified; install still blocked by the known depscan#23630 compact-index 404 — server-side). Co-Authored-By: Claude Fable 5 <[email protected]> * fix(review): CRLF-tolerant source-block guard; dry-run VEX skip for all scan arms Addresses both Bugbot findings on #175: - The grant-agnostic Gemfile idempotency guard required LF (`do\n`), so a core.autocrlf checkout of a previously-redirected Gemfile was not recognized and the indented gem line inside the block got wrapped in a second, nested source block on re-run. The recognizer now accepts `do\r?\n`; pinned by gemfile_rerun_on_crlf_checkout_never_nests (verified red without the fix). - The dry-run VEX skip only covered the vendor JSON arm; the interactive `scan --vendor --dry-run --vex` path (embed_vex_human) and the JSON `scan --apply --dry-run --vex` path (embed_vex_into_json at the apply fall-through) still generated the document — exiting 1 on a not-yet-vendored project or writing the attestation during --dry-run. The guard now lives at the top of both embed helpers, covering every scan arm; pinned by scan_vendor_dry_run_with_vex_interactive_* and scan_apply_json_dry_run_with_vex_* (both verified red without it). Co-Authored-By: Claude Fable 5 <[email protected]> * fix(apply,rollback): singleton-variant mismatch policy + variant-scoped rollback blob gate apply (P1): for release-variant ecosystems (gem/pypi/maven) the variant loop's installed-distribution gate skipped ANY variant whose representative file mismatched — for a SINGLETON base (the common bare pkg:gem/name@ver record) that made the documented default mismatch policy (npm parity: warn content_mismatch_overwritten + apply the full verified patched content, --strict refuses, --force unchanged) unreachable: a locally-modified file failed the run with "no matching variant found" and left the file untouched. The gate now applies to multi-variant bases only — a singleton has no sibling to make a mismatch mean "different distribution", so it falls through to apply_package_patch's MismatchPolicy exactly like the npm branch. mismatch_blob_gaps mirrors the same singleton exemption so the Warn fall-through's afterHash blobs are prefetched under --download-mode diff. Multi-variant behavior is UNCHANGED (sibling mismatch = skip; no-variant-matches = fail closed), pinned by two guards. rollback/remove (P2): the before-blob gate ran over the WHOLE ecosystem-scoped manifest BEFORE select_installed_variants narrowed a broad multi-platform gem manifest to the installed platform, so an uninstalled sibling ?platform= variant's unfetchable before-blob aborted the entire run (--offline: wholesale; online: on any download failure) — and remove then refused with rollback_failed. Discovery + variant narrowing now run first and the gate covers only the (purl, path) pairs that will actually be attempted, plus in-scope purls the crawler could not resolve at all — those keep the pre-existing fail-closed gate (pinned by rollback_invariants / cli_rollback_silent / remove_network), and local-redirect purls stay excluded as before. Tests (each fails on the base commit without its fix): - cli_gem_variant_mismatch_policy.rs (new): singleton default warn+apply (human + JSON envelope), singleton --strict refusal; guards: --force unchanged, multi-variant sibling never overwritten, multi-variant no-match still fails closed. - in_process_gem_multi_platform.rs: rollback with unfetchable sibling before-blob succeeds online (404 mock, blob never requested) and --offline (installed variant's blobs only); remove ditto and clears both records. - apply.rs unit: mismatch_blob_gaps_singleton_mismatch_queued (RED vs the old gate); the sibling-skip test now uses a genuine multi-variant group. Verified: core 2460/0, cli lib 351/0, docker_e2e_gem 2/2, e2e_gem 8/8 non-ignored, gem/pypi variant + rollback/remove/apply suites green, clippy+fmt clean. Platform-gem dir layout re-validated against real RubyGems (ruby 3.4.10): gems/<name>-<version>-<platform> leaf. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(apply): keep qualified singletons and vendor-owned bases out of the mismatch fall-through Review fixes for the singleton-variant mismatch policy: Major: the singleton exemption covered QUALIFIED singletons too. A lone `pkg:gem/name@ver?platform=x86_64-linux` record names one specific distribution, and the representative-hash gate is the ONLY platform resolution (the ruby crawler drops the gem dir's platform suffix precisely because "the installed platform is resolved later" by that hash check) — falling through let a linux-only record silently overwrite a darwin install in the Bundler plugin's `--silent` auto-apply, where the warn half of warn-and-apply is invisible, with a later rollback restoring the LINUX before-bytes. The exemption now applies to UNQUALIFIED singletons only (`variants[0] == base_purl`); qualified singletons keep the pre-existing fail-closed "no matching variant found". `mismatch_blob_gaps` mirrors the same condition (its records now retain their manifest keys so the gate can see qualifiers). Minor: `mismatch_blob_gaps` now skips vendor-owned bases. The apply loop unconditionally skips them (their results are synthesized up front), so a drifted vendored release-variant file queued a spurious "Downloading N full patched blob(s)" fetch online and a spurious "will fail to apply" warning under `--offline`. Minor: refreshed the rollback_invariants comment made stale by the gate reorder — the offline missing-blob bail now aborts after discovery, before the rollback loop produces per-package results; the pinned contentless partial_failure shape is unchanged. Tests (each fails without its fix): - cli_gem_variant_mismatch_policy::qualified_singleton_wrong_platform_fails_closed: lone ?platform= record + wrong-platform install → exit 1, "no matching variant found", file byte-identical, no overwrite warning. - apply.rs unit mismatch_blob_gaps_qualified_singleton_gated_unless_forced: qualified singleton mismatch queues nothing by default, queues under --force. - apply.rs unit mismatch_blob_gaps_vendored_base_never_queued: vendored base queues nothing (even under --force), same fixture queues without the vendor claim. Verified: cli lib 353/0, cli_gem_variant_mismatch_policy 6/6, in_process_gem_multi_platform 9/9 non-ignored, apply/rollback/remove/ vendor/variant suites green, docker_e2e_gem 2/2, e2e_gem 8/8 non-ignored, core green, clippy+fmt clean. Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
… digest, project-scoped stamp (#178) * fix(gem): bundler vendored+hosted sweep — 13 confirmed bugs fixed, tests hardened ULTRACODE review + full test matrix over the gem/bundler ecosystem (vendored and hosted modes, every configuration). 24 findings survived adversarial verification; the 13 well-scoped ones are fixed here, each pinned by a test that fails without its fix. Hosted redirect (patch/redirect/mod.rs, gem section): - splice the Gemfile edit by regex byte range: a commented-out duplicate of the gem line no longer gets rewritten instead of the live line - grant-agnostic idempotency guard: re-running scan --mode hosted with a rotated grant token refreshes the source URL in place instead of nesting a second source block (new edit kind redirect_gemfile_source_url) - fail closed on gem-level git:/github:/path:/source: options (they override the enclosing source block, making the redirect an attested no-op); warn redirect_gem_source_option and skip the dep - fail closed on platform-suffixed CHECKSUMS siblings (redirect_gem_platform_unsupported) instead of inserting a duplicate bare-coordinate pin - recognize paren/tab/multi-space gem declarations; gate the append-branch on the gem being genuinely undeclared (no more duplicate declarations bundler rejects) - never pin the lock CHECKSUMS when the Gemfile source redirect did not land (mixed state guaranteed a checksum failure) - warn that a redirected pair breaks frozen/deployment installs - record the upstream sha256 line as original on the CHECKSUMS edit so a future revert can restore it (golden fixture updated) Vendored backend (vendor/gem.rs): - insert new PATH sections at bundler's sorted position (identifier order, verified against real bundler 4.0.15 bundle lock) — two or more vendored gems no longer churn the committed lock - fail closed on platform-suffixed GEM-specs siblings on no-CHECKSUMS locks (mirrors the existing CHECKSUMS guard) - re-vendor on a patch UPDATE (new uuid, same purl): recognize our own path: wiring and rewire in place instead of refusing with gemfile_declaration_not_editable — the documented automatic re-vendor contract now actually works for gem Auto-fetch (vendor/registry_fetch.rs): - stage fetched gems into the canonical <name>-<version> leaf instead of a dir literally named "gem", which vendor_gem refused as platform_gem_unsupported — lockfile auto-fetch for gems was dead Crawler (crawlers/ruby_crawler.rs): - parse_dir_name_version prefers the last dotted-version boundary, so http-2-1.0.1 parses as (http-2, 1.0.1) instead of the ghost (http, 2) - vendor/bundle discovery enumerates engine dirs (jruby, truffleruby) instead of hardcoding ruby/ Scan/get plumbing: - run_nested_apply now threads --ecosystems: scan --ecosystems gem --sync no longer applies (or mutates) other ecosystems' patches - scan --vendor --dry-run --vex no longer writes the VEX file nor exits 1 on not-yet-vendored state Test hardening: - e2e_gem lifecycle harness: BUNDLE_PATH replaces bundle install --path (removed in bundler 3+; all 3 lifecycle tests green under 4.0.15) - e2e_hosted_production gem leg now asserts the CHECKSUMS digest CHANGED after redirect (client-verifiable today) and, in the success arm, verifies installed content against the published afterHashes — an inert gem patch can no longer stay green (the npm minimist blindspot) - docker_e2e_gem serves the true git-blob beforeHash so the chain exercises the default non-forced apply path, not just --force - setup_matrix_gem module doc: the with-setup Docker cases ARE still a baseline gap (bootstrap deadlock: plugin registration evaluates plugins.rb before any gems land; exit-semantics twin), doc corrected Verified: core 2460/0, cli lib 350/0, clippy+fmt clean; e2e_gem 11/11 (incl. live lifecycle under bundler 4.0.15), e2e_vendor_gem_build 6/6 (incl. real-bundler capstone), docker_e2e_gem 2/2, docker_e2e_vendor_gem 1/1, hosted production gem leg green (redirect verified; install still blocked by the known depscan#23630 compact-index 404 — server-side). Co-Authored-By: Claude Fable 5 <[email protected]> * fix(review): CRLF-tolerant source-block guard; dry-run VEX skip for all scan arms Addresses both Bugbot findings on #175: - The grant-agnostic Gemfile idempotency guard required LF (`do\n`), so a core.autocrlf checkout of a previously-redirected Gemfile was not recognized and the indented gem line inside the block got wrapped in a second, nested source block on re-run. The recognizer now accepts `do\r?\n`; pinned by gemfile_rerun_on_crlf_checkout_never_nests (verified red without the fix). - The dry-run VEX skip only covered the vendor JSON arm; the interactive `scan --vendor --dry-run --vex` path (embed_vex_human) and the JSON `scan --apply --dry-run --vex` path (embed_vex_into_json at the apply fall-through) still generated the document — exiting 1 on a not-yet-vendored project or writing the attestation during --dry-run. The guard now lives at the top of both embed helpers, covering every scan arm; pinned by scan_vendor_dry_run_with_vex_interactive_* and scan_apply_json_dry_run_with_vex_* (both verified red without it). Co-Authored-By: Claude Fable 5 <[email protected]> * fix(gem): production-safe bundler plugin — tolerant bootstrap, honest digest, project-scoped stamp The generated Bundler setup plugin could deadlock a project on its own bootstrap, its design comment described trigger behavior bundler does not have, and its digest stamp was a machine-global file blind to the actual gem-file state. All trigger claims below were re-derived empirically against real bundler 4.0.15 (host) and 2.7.2 (docker image). plugins.rb template (+ published socket-patch-bundler twin): - [P0] bootstrap deadlock: bundler evaluates plugins.rb at plugin REGISTRATION, before any project gem is installed; the load-time SocketPatch.apply! got apply's exit 1 ("No packages found") and raised Bundler::BundlerError, so the FIRST `bundle install` of every fresh clone of a setup-wired project died (exit 29 under 4.0.15, exit 1 under 2.7) and every retry failed identically (registration never completed). Patch failures now warn once per process — naming what failed and the manual remediation — and let the install continue; SOCKET_PATCH_STRICT=1 restores the raise. The load-time and per-gem triggers are additionally stamp-gated so the bootstrap install stays quiet and defers to the forced after-install-all pass. - [P1] trigger reality: the header claimed plugins.rb runs during the Gemfile pass "on EVERY bundle invocation" — false; bundler evaluates a plugin when a subscribed hook event first fires. Measured surface (identical on 2.7 and 4.0): every `bundle install` — fresh AND fully cached — fires before/after-install(-all); `bundle pristine` fires ONLY the per-gem events; `bundle exec`/`bundle check`/`gem pristine` fire nothing. The plugin now also subscribes `after-install` (digest+stamp-gated), which catches `bundle pristine`'s patch reversion in the same run, and the digest folds in the on-disk CONTENT of every gem-patch target (resolved from the manifest purls under Bundler.bundle_path/gems), recomputed after apply — so out-of-band reversion flips the digest even when every committed input is byte-identical. Header documents precisely which flows re-apply and which cannot, including the stale .bundle/plugin/index caveat for checkouts registered by an older plugin version. - [P2] stamp location: the digest stamp was a fixed-name file under Bundler.bundle_path — the interpreter's machine-global gem dir when no bundle path is configured, shared and clobbered across every socket-patch project on the host. It now lives at .socket/gem-plugin-stamp (project-local, excluded from its own digest inputs); the legacy global stamp is deleted best-effort and never read. launcher.rb (gem/socket-patch): - Windows arm now propagates the child's real exit code instead of collapsing every non-zero exit to 1. - the binary-cache install is atomic: staged as a temp file in the destination dir, chmodded, then renamed into place (cross-run race on Windows rename tolerated when the winner already published). - first-run failures outside LauncherError exit with a clean one-line message instead of a raw backtrace; the PowerShell Expand-Archive fallback quotes paths containing single quotes; `version`'s documented from-a-checkout fallback never engaged because Gem::MissingSpecError is a Gem::LoadError (ScriptError family), not a StandardError — found by the new launcher guard. - socket-patch-bundler.gemspec: stale `git:` comment corrected to `path:` (the source has been path: since #150). setup-matrix driver (gem-scoped, npm-family byte-identical — verified by diffing the fixtures the old and new driver produce for npm across all patchsets): - the gem fixture now serves the REAL git-blob beforeHash probed from the published .gem (`gem fetch` + `gem unpack`, mirroring docker_e2e_gem's probe; verified against an independent oracle), so hash-gated gem apply passes the variant gate without --force. With the deadlock fix this turns the formerly-gapped gem with-setup docker cases green: the full 6-case gem matrix passes in BOTH host mode (bundler 4.0.15, real rubygems.org installs) and docker mode (rebuilt image, bundler 2.7) — no dependency on any sibling apply change. Tests (each red without its fix): - core template invariants: test_plugin_template_failure_policy_and_ stamp_location (new) + test_templates_are_well_formed (extended) pin the tolerant reporter, strict hatch, stamp constants, legacy cleanup, target-content digest, and the published twin's parity — 2 failures against the old template. - setup_matrix_gem::plugin_runtime drives the plugin generated by the REAL binary through REAL `bundle install` runs with a fake apply: first_bundle_install_survives_failing_apply (the P0 repro: red at exit 29 on the old template), strict_mode_fails_bundle_install_on_apply_ failure, successful_apply_stamps_project_scoped (stamp path + exactly one forced apply per cached install), digest_tracks_gem_file_content_ and_legacy_stamp_is_removed (plain-ruby drive; red on the old manifest-only digest and old stamp path). - setup_matrix_gem::launcher_guard drives launcher.rb with host ruby: windows_branch_propagates_child_exit_code (red: 7 collapsed to 1), unexpected_download_errors_exit_cleanly (red: raw backtrace), powershell_quote_doubles_single_quotes and install_executable_is_atomic_into_place (red: helpers absent). 7 of 8 runtime/launcher guards fail against the base-branch code. Verified: core 2461/0, cli lib 350/0, setup_matrix_gem 11/11 (incl. the docker-mode 6-case matrix on a fresh image AND host-mode 4.0.15 run), docker_e2e_gem 2/2, docker_e2e_vendor_gem 1/1, e2e_gem 11/11 (incl. live lifecycle), clippy+fmt clean on both crates. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(gem): bootstrap gate keys on target presence, not the committed-dir stamp; gitignore + lifecycle it The require_stamp gate trusted the EXISTENCE of .socket/gem-plugin-stamp — a file living in the directory every other workflow file tells users to COMMIT. Reproduced against bundler 4.0.15: a stale stamp that reaches version control passes the gate at plugin REGISTRATION on a fresh clone, digest-mismatches (targets absent), shells apply, and under SOCKET_PATCH_STRICT=1 resurrects the exact bootstrap deadlock this plugin exists to avoid (exit 29, "Failed to install plugin", no plugin index, every retry identical). Deleting the stamp had the inverse sharp edge: the gated triggers went dead, so `bundle pristine` left the patches reverted until the next `bundle install`. - plugins.rb template + published twin: the bootstrap gate now bails while NONE of the manifest's gem-patch targets exist on disk, reading the live gem tree and never the stamp (which is now a pure digest cache). Registration on a fresh clone stays quiet regardless of stamp state, and pristine heals in the same run even with the stamp deleted — both verified against real bundler 4.0.15. - report_failure: the trailer now states what the ACTIVE mode does — the strict raise says the install is failing because SOCKET_PATCH_STRICT is set, instead of claiming "`bundle install` continues". - setup wires /gem-plugin-stamp into .socket/.gitignore (append-only, sparing user lines) so the stamp never lands in git status or a blanket `git add .socket`; `--check` demands the entry (check/setup agreement); `--remove` best-effort deletes the stamp and strips our line. - matrix.json: the gem row records reality — hook_family bundler-plugin, baseline_supported true — so a future regression of the with-setup flow classifies as blocking regression, not a known gap. - launcher_guard::run_ruby scrubs RUBYOPT/BUNDLE_*/GEM_*/SOCKET_* like plugin_runtime::scrub, so the suite survives `bundle exec`. New pins: plugin_runtime::committed_stale_stamp_does_not_deadlock_strict_ fresh_clone (registration recorded, hook-only failure, retry converges), plugin_runtime::bootstrap_gate_keys_on_target_presence_not_stamp (both gate directions), strict-trailer asserts in the strict-mode test, gitignore and stamp-lifecycle asserts in host_guard + core gem tests. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(gem): normalize the platform-gem glob base to forward slashes Dir.glob treats backslash as an escape on EVERY platform, and Bundler.bundle_path carries Windows backslash separators through verbatim (verified: BUNDLE_PATH='vendor\bundle' yields <root>/vendor\bundle/ruby/3.4.0). The platform-install wildcard in patch_target_files (<gems>/<name>-<version>-*/<rel>) therefore escape-ate the separator and matched nothing on Windows: platform installs (nokogiri-1.15.0-x64-mingw-ucrt) dropped out of the digest, so a bundle pristine reversion of them left the stamp matching and the re-apply skipped. Fix: glob a slash-normalized base (forward slashes are valid separators on Windows); the direct non-glob join stays byte-faithful. Applied to both the setup template and the published gem twin, pinned by new needles in the core parity test. Regression test (verified red without the fix): plugin_runtime::backslash_bundle_path_still_digests_platform_gem_files drives the generated plugins.rb with plain ruby under a backslash-bearing BUNDLE_PATH while the real tree lives at the slash spelling (the two-spellings-one-directory situation Windows creates): the platform install must be enumerated as a patch target and its reversion must flip the digest and re-run apply. Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.