GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||
GeezerVison is showing github.com — v5 tests: one suite per command, retire #257 oracles by mikolalysenko · Pull Request #297 · SocketDev/socket-patch Conversation
|
| Finding | Outcome |
|---|---|
| F03: delete #257 "verbatim previous implementation" oracles after golden snapshots | done (partial). The rewriter and parser oracles are deleted: hosted npm, yarn classic, pnpm, cargo lock, composer, golang, uv/python lock, pdm plan, pdm/poetry parse reuse, and the reference POM parser. Each seeded sweep now replays against per-case golden digests (crates/socket-patch-core/src/golden.rs, tests/equivalence/*.golden). The goldens were blessed while the oracles still asserted equality (a19da46, then d61f6ba). Kept: the crawler oracles (cargo, composer, go, maven, npm, nuget, python). A crawler's output follows the filesystem's readdir order, and every crawler golden blessed on ext4 failed when replayed on tmpfs. A golden therefore can't pin them portably, whereas the oracle walks the same tree on every OS (912d005). Also kept: about 200 LOC of small memo-vs-fresh and pool-vs-serial oracles over real I/O (copy_tree, cargo_lock memo, reuse, go_mod_edit, common repack, verify, registry_fetch, pypi_wheel), plus group_equivalence's 38-line serial_oracle chain-order spec. |
| F04: collapse the covgap_/coverage_fix_/in_process_ categories into one suite per command | done (partial). Category files for apply, get, remove, rollback, repair, scan, vendor, update and the global CLI now live in tests/<cmd>/, one binary per command. Files keep their names as modules for traceability. Deferred: the 82 in-process or env-mutating binaries (in_process_*, most covgap_commands_*, cli_parse_*). They share process-global state (notice SILENT/JSON, the socket-cli config OnceLock, the API client's *_SHOWN flags), so merging them changes test isolation until those statics are made resettable. |
| F58: cut the CLI integration-test binaries by merging small ones | done. 67 subprocess-only binaries → 9 suites, so the CLI goes from 223 → 165 test binaries. Kept separate: every binary a workflow names (64), docker_e2e_*, and crawl_fd_limit_e2e (process-global walk pool). |
| F10: dedupe the CRLF/corrupt/dry-run/round-trip invariants into table-driven tests | deferred. Re-verified: the ~17 vendor backends each build their own fixture (blobs, installed tree, record, run fn), and no shared fixture interface exists at the LockModel level. Table-driving them needs a new cross-backend fixture trait, which is a design change for its own PR. It would save LOC, but no CI time: all of these tests live in the single core lib binary. |
Measured savings
.rs, this PR against the current base, merges excluded): +794 / −3334, net −2540, plus 21 golden data files (+4516 lines of digests).common/update_fixture/cache_env helper self-tests that ran again in every binary; they now run once per suite.--one-off tests from these files; that removal is v5: remove dead code and v3 compatibility shims #296's change.test jobs total 78.5 vs 78.7 min.cargo llvm-cov, on dbb5848): 97.35% → 97.35%, with missed lines 6161 → 6136.
CI status on fe293ed (after merging #292 and #296)
optional agent on dbb5848, and 2.17.3 direct hosted on 5e16953. PDM is green on fe293ed.yarn-classic 1.0.2 / 1.6.0 / 1.7.0 / 1.9.4. Also red on the base run 36474481876 and on v5: socket.yml patch rollout config and filtering #293 after v5 CI: build e2e binaries once and tier the PM matrix #291.install-proof: ubuntu 0.0.0-1, 0.0.0-1 on node 22.0.0, 0.0.0-11 and windows 0.0.0-11. On ubuntu the e2e_redirect_vlt_build hosted rollback legs fail; the same jobs are red on v5: cap new patches per scan, most critical first #294.DepOverride.berry_zip_url, so the goldens' input digests were re-blessed. 1848 lines in 12 files changed only their input digest, and every case key and output digest is unchanged (fe293ed).Test evidence (local, Linux)
socket-patch-core lib on fe293ed: 4492 passed as non-root, with TMPDIR on tmpfs as well as on ext4, so the goldens are filesystem-independent.
14a9cb0.14a9cb0 the stage_local_artifact peak-RSS test also fails as root; it inherits the parent's RSS under a full parallel run.repair passes 116/116 as non-root.#[path]/use crate:: edits.cargo clippy --workspace --all-features -- -D warnings is clean, and --all-targets is clean for core and CLI.scripts/tests (118 tests) pass.Notes for reviewers
Debug/JSON form of each result, so a dependency bump that rewords an error, a new lock fixture, or a changed DepOverride shape needs a re-bless (SOCKET_PATCH_BLESS_GOLDEN=1). Review that diff like any golden change: for a pure input-shape change, only the input-digest column should move.SOCKET_PATCH_{COMPOSER_FIXTURE,GO_FIXTURES,PY_LOCK_FIXTURES}. No CI job sets these.🤖 Generated with Claude Code
Sorry, something went wrong.
There was an error while loading. Please reload this page.
The randomized equivalence sweeps that compare the blocking-pool crawlers and single-pass rewriters with their pre-#257 oracles now also record, per case, a digest of the generated input and of the output both implementations agreed on, in crates/socket-patch-core/tests/equivalence/*.golden. The oracles still run, so every recorded output is proven equal to the oracle's. The next commit deletes the oracles and keeps the goldens. The crawler goldens were blessed on Linux as a non-root user: the sweeps strip permissions and plant symlinks, so they replay only there (other platforms and root still run the sweeps, without the golden comparison). Also moves the crawler sweeps' xorshift test RNG to crate::test_rng, and renames crawlers::oracle_support to crawlers::test_tree. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
The equivalence sweeps now replay against the golden digests pinned in the previous commit instead of re-running the verbatim previous implementations: the crawler oracles (cargo, composer, go, maven, npm, nuget, python), the hosted rewriter oracles and the reference POM parser are deleted. Every sweep still generates the same inputs from the same seeds, so a changed output is still caught per case. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
The pdm plan, pdm and poetry parse-reuse rewrites were checked against #257's verbatim fresh-parse implementations. Their sweeps now record per-case digests, blessed while those oracles still agreed with production, and the oracles are deleted. Golden keys with whitespace are normalized so a case label can never split a golden line. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
The subprocess-only integration tests for apply, get, remove, rollback, repair, scan, vendor, update and the global CLI now build as one binary per command instead of 67, so CI links and runs 58 fewer test binaries. Each file keeps its name as a module, and the shared helpers are declared once per suite, so their self-tests run once per suite instead of once per file. Binaries a workflow names, the docker e2e suites, crawl_fd_limit_e2e and the in-process suites (which share process-global notice and config state) stay separate. Doc links to the moved files are updated. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
Rows that named a moved test binary now name its suite and module. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
A crawler's output follows the directory order the filesystem hands back, which differs between filesystems: every crawler golden blessed on ext4 failed when replayed on tmpfs, and macOS and Windows skipped them. The per-call oracles walk the same tree as the crawler, so they check the parallel walk on every platform; restore them and drop the crawler goldens. The rewriter goldens are pure text and stay. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
There was an error while loading. Please reload this page.
|
[agent] On ubuntu, three hosted tests in This isn't this PR's failure:
I haven't found a fix to port yet. I merged the new base (#291) in dbb5848, and the jobs still fail there. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] I don't think this is from this PR. The same workflow passed on the previous head 912d005 (run 36497447814) and on #293 and #294, which have the same base. The only change since 912d005 is the base merge, which touches CI, scripts and Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] ready to land CI on dbb5848 is green apart from base-inherited failures:
The one PDM flake passed on re-run. There's no merge conflict. Coverage is 97.35% → 97.35%, with 25 fewer missed lines. The findings table, measured savings and test evidence are in the description. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
#292 landed on release/v5-prerelease as a7b0d00; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
|
[agent] Merged Local checks:
CI is re-running on the new head. The PR stays ready. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] This PR changes only test code and goldens, but the base merge brought in #292, which changes the hosted fetch path ( Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
#296 landed on release/v5-prerelease as 1e3ace6; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
#296 drops DepOverride's berry_zip_url, so the goldens' input digests (which serialize the deps) are re-blessed: 1848 lines in 12 files change only their input digest, and every case key and output digest is unchanged, so the rewrites behave exactly as before. The two env-gated fixture tests #296 edited stay deleted here. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB
|
[agent] Merged Conflicts: two, in Goldens re-blessed: #296 removes Local checks:
CI is re-running. The PR stays ready. Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Coordinator: #297 is next in the landing order (#287 is still a draft and is skipped). It is blocked only because there is no Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[agent] ready to land CI on fe293ed has finished. The branch contains the current base (1e3ace6, #296).
Generated by Claude Code |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
b97a1c2
into
release/v5-prerelease
There was an error while loading. Please reload this page.
Brings in #297, which groups the CLI tests into one binary per command and replaces the #257 oracles with golden files. It merges cleanly; this PR's test binaries stay at the top level, as most still do. Clippy and the policy, in-memory, parity, e2e policy, parser, help, scan and get suites pass. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01BKsyzefGhAnPkYmXCwq3H3
Brings in #297, which groups the CLI tests into one binary per command and replaces the oracle comparisons with golden files. The merge had no conflicts. This branch's own suites stay separate binaries. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01AvGSu5jGg1z3f3sePc2eHC
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.