GeezerVison

Problem

Architecture audit §3.B (purl equality). At least six relations answered "do these two purls name the same package release", and they disagreed:

• purl_eq folded percent-encoding and composer only;
• purl_identity_key stripped qualifiers and did composer only;
• two different functions were both named canonical_base_purl (one in vex::discover and one in rollout);
• policy::canon and package_spec_matches each had their own folding;
• the ledger had five ownership matchers with different canonicalization (covers_purl, purl_keys_cover, lookup_entry, apply's Go --check set, the hosted_pin_of / hosted_vendored_overlap keys);
• there were several local normalize_purl(strip_purl_qualifiers(..)) closures used as keys.

The canonical_purl doc comment also claimed an identity it doesn't provide.

User-visible effects:

• B20: scan --prune / --sync dropped a live manifest entry, and GC'd its blobs, when the API spelled the purl pkg:nuget/[email protected] and the NuGet global-cache crawl reported pkg:nuget/[email protected]. Update detection and redirect-candidate matching missed the same entry. docker_e2e_nuget.rs worked around this by serving a lowercase purl from the fixture. A PEP 503 spelling (typing_extensions vs typing-extensions) was pruned the same way.
• B73: remove / rollback pkg:pypi/[email protected] found nothing when the recorded key was typing-extensions. The same happened for a NuGet case variant.

Related: #553 (the takeover ledger lookup now uses the same key, but there's no e2e for it yet), #748 / C20 (this is the identity half; the builders are untouched), #484 (Go + vs %2B is folded by the key).

Change

• New utils::purl_key
  • PurlKey is the release identity. It strips qualifiers and subpath, percent-decodes, lowercases the type, PEP 503-folds PyPI names, case-folds NuGet names and versions, case-folds Composer names, and keys Composer versions by release identity (3.0.2 = v3.0.2 = 3.0.2.0).
  • PurlKey::qualified keeps release variants apart.
  • canonical_base_purl moved here. It is the display spelling the key is built from, and VEX product purls still use it.
• Call sites. Every purl identity comparison in core and the CLI now builds PurlKeys (the exceptions are listed under Deferred):
  • ledger ownership;
  • prune GC;
  • update detection;
  • redirect candidates;
  • lockfile-only checks (one predicate; the duplicate notInstalled check now calls it);
  • takeover ledger drop;
  • hosted pins;
  • VEX source matching;
  • rollout budget;
  • policy reports;
  • get presence;
  • Go-patches orphan prune;
  • vlt heal targets;
  • Gradle pin rows;
  • the hosted memory engine's batch-search owner map, lock_inventory::lookup, the vendored blob harvest, apply's mismatch-blob record filter, rollback's superseded_by_hosted, and the rollout RecordedIndex (review round, see below).
• Vendored key sets are now HashSet<PurlKey>, so the compiler routes every construction through the key.
• package_spec_matches compares the folded key case-insensitively. User filter specs stay as lenient as before, and a versioned Composer spec still matches by exact release identity.
• canonical_purl keeps only its display role. Its doc no longer claims identity.
• Docker e2e: docker_e2e_nuget.rs now serves the API's real mixed-case spelling for the patch, and scan --sync must keep that entry.

Duplicate copies deleted

Purl equality relations went from 11 to 1 (PurlKey, with canonical_base_purl as its display spelling). Deleted:

• utils::purl::purl_eq
• composer_version::{purl_identity_key, composer_purl_identity, composer_purls_equivalent, composer_bases_equivalent}
• rollout::canonical_base_purl
• policy::canon
• vex::discover::{canonical_base_purl, same_package}
• rollout::stage::qualified_key (a copy of PurlKey::qualified)
• lock_inventory::lookup's own PEP 503 / exact-version matcher
• vex_sources::same_package
• policy's canonical_pypi_purl

The ledger matchers (covers_purl, purl_keys_cover, lookup_entry_kv) are now one-line PurlKey comparisons. The ad-hoc key closures in scan/mod.rs, ledgers.rs, list.rs, hosted.rs, get.rs and apply.rs are gone.

Behavior notes

• Policy filtered[] / retained[] purls and rollout base_purls are the PurlKey spelling, so PyPI and NuGet names there now appear folded. Composer already showed its identity form.
• VEX output is unchanged except for one case: a Composer dev- branch name keeps its case in canonical_base_purl. Before this PR it was lowercased.

Review round (rebased onto main @ 431b818)

• Rebased onto 14 new main commits (Fix VEX attesting beside an unpatched same-lock copy (#935, #938, #939) #940, Fix gem takeover un-hosting a grouped gem (#775) #776, Fix hosted yarn classic pins missing berry warning (#907) #917, Fix gem crawler missing Bundler .bundle root (#967) #968, Fix pnpm lock/workspace readers missing a BOM (#903, #904, #905) #909, Fix rollback/remove of an agent record superseded by a hosted pin (#933) #934, Honor HTTP-date Retry-After on vendor-service retries through api::retry (#677) #889, then Fix composer e2e flake on packagist connect timeouts #1037, Fix yarn berry project gates drifting between modes (#628, #629) #657, Fix Hatch environments being invisible to stale-install checks and VEX (#335) #700, Fix .bundle/config values keeping a trailing # comment (#951) #953, Scan every vendored-write file for vendored references (#832, #958) #1015, Fix remove/rollback missing PyPI name spellings (#1024) #1025, Fix stale docs and comments, and the npm wrapper's schema and exit code #1022).
• Fix remove/rollback missing PyPI name spellings (#1024) #1025 (fixes remove and rollback don't PEP 503-normalise PyPI purl identifiers, so remove pkg:pypi/[email protected] exits 1 "No patch found" while get accepts the same identifier #1024, the PyPI half of B73) had brought back utils::purl::purl_eq with a PEP 503 rule. This branch deletes it again: patch_matches already folds PEP 503 through PurlKey. Fix remove/rollback missing PyPI name spellings (#1024) #1025's unit test now checks equality with PurlKey::qualified, and its remove/pypi_name_spellings.rs CLI tests pass unchanged. The get.rs conflict was resolved so Fix VEX attesting beside an unpatched same-lock copy (#935, #938, #939) #940's hosted-claim set is a HashSet<PurlKey> (not canonical_purl strings). The new gem-takeover pin lookup in vendor.rs uses PurlKey::same, and the new rollback::superseded_by_hosted uses PurlKey instead of the deleted helpers.
• scan --prune --dry-run now shares the wet vendor GC's manifest-key drop (vendor::unused_vendored_manifest_keys). Before this, the preview used a qualifier-strip relation, so NuGet case, PEP 503 and composer padding variants were pruned for real but missing from the preview.
• hosted::memory::discover::batch_search matches response packages to their asking roots by PurlKey. Before this, a respelled answer (such as Newtonsoft.Json) was credited to every root in the chunk.
• lock_inventory::lookup matches by PurlKey, so a composer API @3.0.2.0 finds the lock's 3.0.2.
• LockfileSupplement.purls is a HashSet<PurlKey> built once. The lockfile-only check is now one hash lookup, not an O(n) re-key on every miss.
• hosted_vendored_overlap deduplicates by PurlKey, so one release spelled two ways gives one takeover warning.
• Composer: PurlKey keeps the sentinel-free composer_version_key so its string stays printable in policy and rollout reports. A new test over the vector file checks that a rejected spelling can never key the same as an accepted one: every accepted key is itself accepted, bare and behind a v. The module docs now say so.

Core API break (for the release notes; CHANGELOG untouched)

The following socket-patch-core public items were removed or moved:

• utils::purl::purl_eq
• vex::canonical_base_purl (re-export) and rollout::canonical_base_purl; use utils::purl_key::canonical_base_purl
• composer_version::{composer_purl_identity, purl_identity_key, composer_purls_equivalent}
• rollout::stage::qualified_key

These signatures changed from HashSet<String> to HashSet<PurlKey>:

• vendor::vendored_purl_keys
• VendorState::purl_keys
• vendor::purl_keys_cover

lock_inventory::lookup now also matches composer padding and NuGet/composer name case. Nothing else in the workspace references the removed items.

Testing

Failing first: on main plus only the new tests, all three regression tests failed. On this branch they pass.

• cargo test -p socket-patch-core --lib -- folds_pep503_and_nuget_case nuget_case_spellings_are_one_vendored_package: 2 failed.
• cargo test -p socket-patch-cli --lib -- detect_prunable_keeps_case_and_pep503: failed. It pruned both Newtonsoft.Json and typing_extensions.

New tests:

• purl_key property-style table tests (hand-written loops, not proptest):
  • every spelling variant of a release shares one key, and no other release does;
  • the Composer key equals release equivalence over the whole composer-version-vectors.json;
  • qualified keys keep variants apart;
  • NuGet and PEP 503 cases.
• B20 prune test (gc.rs).
• B73 identifier test (purl.rs).
• NuGet scan --mode vendored over a hosted pin skips the takeover (purl case mismatch), keeps the hosted feed wired, writes no vendor ledger and still reports success #553-shape ledger test (state.rs).

Commands run on this branch (macOS):

• cargo test -p socket-patch-core -p socket-patch-cli --lib
  • CLI: 862 passed.
  • Core: 5580 passed. 1 failed, digest::production_digests_go_through_the_helpers, which was already red on main and is fixed by Fix main CI red on stale digest pending-list entries #1016 (now merged; I rebased onto it after the run).
• cargo test -p socket-patch-core --tests --no-fail-fast: all 36 integration suites pass. The lib failure is the same pre-existing one.
• cargo test -p socket-patch-cli with these integration tests: scan, remove, rollback, apply, get, vendor, cli, e2e_scan, e2e_nuget, e2e_composer, e2e_composer_version_identity, e2e_socket_yml_policy, scan_rollout_e2e, hosted_memory_{rollout,engine,parity}, covgap_commands_{scan_mod,scan_hosted,rollback,get,vendor,vex}, global_scope_project_state, in_process_{remove_repair_lifecycle,rollback_vendored,vendor}, e2e_vex, e2e_vex_vendor, e2e_vex_redirect, e2e_embedded_vex, coverage_fix_scan_hosted_dryrun_vendored. All passed.
• cargo clippy -p socket-patch-core -p socket-patch-cli --all-targets -- -D warnings: no findings in any touched file.
  • The local clippy 1.93 reports pre-existing lints in untouched files, such as jvm_jar.rs, prebuilt_common/mod.rs and a macOS-only unused variable in python_crawler.rs.
  • I rustfmt'd only the changed hunks. Files that were already unformatted on main keep their old formatting.
• After the rebase: focused unit tests for purl, gc, state, policy and rollout pass.
• After the review-round rebase and fixes (macOS):
  • cargo build -p socket-patch-core -p socket-patch-cli --all-targets
  • cargo clippy -p socket-patch-core -p socket-patch-cli --all-targets: 0 findings in touched files. The pre-existing lints in untouched files listed above remain.
  • cargo test -p socket-patch-core --lib: 5641 passed (the digest failure is gone now that Fix main CI red on stale digest pending-list entries #1016 is on main).
  • cargo test -p socket-patch-cli --lib --test scan --test vendor --test rollback --test apply --test get --test remove --test gradle_agent_cli --test in_process_rollback_vendored --test in_process_vendor: all passed (lib 872, scan 118, vendor 91, rollback 38, apply 106, get 81, remove 94 (incl. Fix remove/rollback missing PyPI name spellings (#1024) #1025's PyPI spelling tests), gradle_agent_cli 34, in_process_rollback_vendored 16, in_process_vendor 121).
  • New regression tests: batch_search_credits_a_respelled_response_only_to_its_asker, lookup_matches_by_purl_identity, overlap_reports_one_entry_per_release_across_spellings, covers_every_spelling_of_the_release (unused_vendored_manifest_keys), and sentinel_free_key_keeps_rejected_spellings_apart. The first three fail against the pre-fix code: a spelling-keyed owner falls back to every chunk root, exact-version lookup misses 3.0.2.0, and spelling dedup gives 4 entries.
• Not run locally: docker_e2e_nuget (Linux/Docker). It is the end-to-end proof for B20 (mixed-case manifest key against the lowercase global-cache crawl, through scan --sync, GC, apply and agent VEX). It must pass in CI before this lands.

Deferred

• Name-level PyPI comparisons inside lock parsers (canonicalize_pypi_name(a) == canonicalize_pypi_name(b)) are left alone. They compare names, not purls.
• Same-source qualifier grouping in vendor.rs / apply.rs (variant_groups, vendored_bases over manifest keys) is left as-is, since both sides already share one spelling.
• Moving Ecosystem and canonicalize_pypi_name to core (E39 / Move canonicalize_pypi_name and the PEP 508 name scanner out of crawlers and vendor into one PyPI name module #883) is a separate refactor.
• The four target grammars (scan --package, get fuzzy, remove/rollback, UUID shortcut) are untouched.
• Left as literal spelling comparisons, deliberately:
  • get.rs's failure dedup on normalize_purl is display-only.
  • ecosystem_dispatch::merge_qualified matches a crawler's echo of the exact base purl it was asked for, so both sides are the same string.
  • The Gradle/Maven scan keys use canonical_base_purl. Maven coordinates are case-sensitive, so the canonical spelling is the identity, and the lock-file GAVs they are checked against are built as strings.
• The vendored blob harvest matches PurlKey::new(base_purl), not lookup_entry. It keeps the old qualifier-insensitive match for manifest keys, whereas lookup_entry is qualifier-sensitive.
• No maintainer decision was needed.

🤖 Generated with Claude Code

Generated by Claude Code