GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Bound crawler and tool probes through one spawn deadline (#845) by mikolalysenko · Pull Request #886 · SocketDev/socket-patch

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 5, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #845

Summary

Every crawler probe (gem env gemdir, python3 --version, npm root -g, composer global config home, ...) now runs under a 10 s deadline through one utils::process::output_within primitive. A wedged toolchain shim used to hang scan, and every other crawling command, forever with no output. The three sites that hand-rolled their own tokio::time::timeout + kill_on_drop (pipenv, hatch, self-update sanity_exec) now use the same primitive, and their inline blocks are deleted.

Why

  • Issue #845, register row C48 (register), living document doc/07-infra-agent.md (process spawning).
  • Leverage: B 1 (p1 hang), U 1 (the async-CommandRunner and npm_dir slices reuse the primitive), D ≈3 (three duplicate timeout blocks plus the unbounded runner, all on one primitive), R M. Score ≈5.
  • Why this slice: it bounds run_resolved in utils/process.rs, so it fixes every probe without editing the crawler files that Fix hosted gem redirect ignoring Bundler mirror.all (#681) #684, sbt, Mill and scala-cli support in agent, hosted and vendored modes #690, Fix Hatch environments being invisible to stale-install checks and VEX (#335) #700 and Fix gem pair model ignoring custom lockfile and Bundler 1 twins (#749, #751) #768 change.

What changed

  • utils::process::output_within(Command, budget) -> Result<Output, BoundedError>: null stdin, captured stdout, discarded stderr. stdout is read on a thread under recv_timeout, then the child's exit is polled until the deadline. At the deadline the child is killed and reaped, and the call returns TimedOut without waiting for a grandchild that still holds the pipe (a #!/bin/sh shim's sleep).
  • PROBE_TIMEOUT = 10 s. That is the budget pipenv, hatch and sanity_exec already used.
  • run_resolved, behind SystemCommandRunner and GlobalProbeRunner, goes through output_within. A probe that times out answers None ("no information"), the same as a failed probe, and logs one --debug line.
  • pipenv::installed_major, pypi_hatch::require_environment_context_support_with and update::download::sanity_exec build a std::process::Command and run it through run_blocking(output_within). sanity_exec keeps its ETXTBSY retry loop around the call and its "hung during its --version self-check" error.
  • Base-red port (7b2eabd): Route Gradle digests through utils::digest #878's fix, verbatim, so main's failing digest ratchet passes here. It moves the inline digests in gradle_cache.rs, jvm_jar.rs and sidecars/maven.rs onto utils::digest, and no-ops once Route Gradle digests through utils::digest #878 merges.

Deleted

Refactor commit only (excluding the #878 port): 4 files, +229 / −34.

  • Production: ≈ +124 / −34. The deleted lines are the three tokio::time::timeout(..., cmd.output()) + kill_on_drop blocks and the unbounded command.output() in run_resolved.
  • Tests: ≈ +105.

Behavior

  • Crawler probes: used to wait with no limit. Now a probe that runs longer than 10 s is killed and treated as absent.
  • Children's stderr: now goes to the null device instead of a pipe that was captured and then dropped. pipenv and hatch already threw their stderr away, and sanity_exec already nulled it.
  • Unchanged: output, exit codes, JSON and the CLI contract.

Test evidence

  • New tests in utils::process:
    • a_hung_probe_answers_none_within_its_budget: a gem shim that runs exec sleep 30, under a 300 ms budget. This can't be expressed on main, which has no budget.
    • output_within_does_not_wait_for_a_grandchild_holding_stdout.
    • output_within_reports_status_stdout_and_spawn_errors: stdout, stderr dropped, nonzero exit, null stdin, missing program.
  • New test in pypi_hatch: a_hung_hatch_is_refused_within_the_probe_budget.
  • The existing pipenv, hatch, sanity_exec (hang, ETXTBSY retry, wrong program) and process tests still pass.
  • cargo test -p socket-patch-core --lib: everything passes except the 4 tests that fail on main too because the sandbox runs as root (copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_…, pypi_requirements::wire_failure_rolls_back_…). utils::digest::tests::production_digests_go_through_the_helpers is red on main @ 9c43dfc and green here after the Route Gradle digests through utils::digest #878 port.
  • cargo test -p socket-patch-cli --all-features --test in_process_gem_apply --test cli_global_args --test in_process_alternate_installers: all pass.
  • cargo clippy --workspace --all-features -- -D warnings: clean.
  • CLI repro (debug build): a Bundler project (rack 2.2.8), with a gem shim on PATH that logs its args and runs exec sleep 3600, and the API pointed at a closed port. scan --json exits 1 after 10 s with the full JSON envelope (scannedPackages: 1). The shim log shows env gemdir and env gempath, and no sleep 3600 process is left afterwards. The issue's run of the same setup on main hung past 90 s with empty stdout.

Risk

M. Every probe's spawn path changes. The budgets match the old per-site values, and the crawler probes only gain a bound.

Remaining (#845 later slices)

  • vendor/npm_dir.rs's git check-ignore exchange (after Fix vendored npm-family tarballs dropped by .gitignore (#831) #837, which edits that file).
  • An async CommandRunner.
  • The "no output() outside utils::process" architecture_tests guard.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mr9d6u7vNrWPgoibe1ZRXd


Generated by Claude Code

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@claude
Start refactor for #845
358792d
Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko Mikola Lysenko (mikolalysenko) added refactor Structural change: duplicated code or logic, missing abstraction, layering, dead code arch-refactor PR opened by the scheduled architecture refactor routine labels Oct 5, 2026
@mikolalysenko Mikola Lysenko (mikolalysenko) mentioned this pull request Oct 5, 2026
Crawler probes spawn gem, python and npm with no timeout, so a hung shim hangs scan forever #845
Closed
4 tasks
@claude
Bound crawler and tool probes by one deadline
b036f7d
A version-manager shim that never answers (`gem`, `python3`, `npm`,
`composer` behind rbenv/asdf, a Ruby waiting on a network gem home)
used to hang `scan`, `apply`, `vex` and every other crawling command
forever with no output: the crawler probes waited on `output()` with
no deadline.

Every probe now runs through one `utils::process::output_within`
primitive: null stdin, captured stdout, dropped stderr, and the child
killed and reaped at the deadline without waiting on a grandchild
that still holds the pipe. Crawler probes get the same 10 s budget
that the Pipenv and Hatch version probes and the self-update
`--version` check already used, and those three sites drop their
hand-rolled `tokio::time::timeout` + `kill_on_drop` blocks for it.

Refs #845.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 5, 2026 19:13
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 5, 2026
@claude
refactor: #886 for #845 (C48), re-rank queue
49b0ea0
Assisted-by: Claude Code:claude-opus-5-5
cursor[bot]
cursor Bot reviewed Oct 5, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@claude
Port #878: Gradle digests through utils::digest
7b2eabd
`main` fails `utils::digest::tests::production_digests_go_through_the_
helpers` because #646 left inline sha1/sha256 calls in
`gradle_cache.rs`, `jvm_jar.rs` and `sidecars/maven.rs`, which turns
`test`, `test-release` and `coverage` red on every PR. This is #878's
change verbatim; it no-ops once #878 merges.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

[agent] coverage, test (macos-latest), test (windows-latest) and test-release failed on b036f7d at a single test, utils::digest::tests::production_digests_go_through_the_helpers. It is red on main @ 9c43dfc as well: #646 left inline digests in crawlers/gradle_cache.rs, patch/jvm_jar.rs and patch/sidecars/maven.rs, and this PR touches none of them. I ported #878's fix verbatim in 7b2eabd. It no-ops once #878 merges. Locally the digest, gradle_cache, jvm_jar and sidecars tests pass, and clippy is clean.

The cancelled e2e (windows-latest, e2e_safety_vlt, …) cell ran no steps and left no logs, which points to a lost runner rather than a test failure. The new push re-runs it.


Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cursor[bot]
cursor Bot reviewed Oct 5, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 7b2eabd. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

[agent] lock-diff (vlt patch compatibility) failed on 7b2eabd with "no cross-OS lock set … locks from no OS". The cause sits upstream of it: build (ubuntu-latest) was cancelled after 30 minutes without ever getting a runner (no runner name, no steps ran). That skipped native, canary, downgrade and install-proof, so lock-diff had no locks to compare. Nothing ran from this diff, so this isn't a failure in this PR. I re-ran the failed jobs once.


Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

[agent] Update on lock-diff: the one re-run failed the same way. build (ubuntu-latest) was cancelled again after about 16 minutes with no runner assigned and no steps run, so lock-diff again compared only the Windows and macOS locks against a missing Linux set.

This isn't this PR's. Every recent vlt patch compatibility run shows the same cancelled, runner-less build (ubuntu-latest): #724, #777, #768, #657, #825 and others since about 20:10 UTC. It's a runner-capacity problem for that job, not a code failure. I've used the re-run and won't spend another. The check should go green once ubuntu runners pick that job up again; the next push or a maintainer re-run will show it.


Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Tanmay182003
Tanmay Singla (Tanmay182003) approved these changes Oct 5, 2026
View reviewed changes
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

[agent] gradle 7.6.6 / jdk 17 / vendor / ubuntu-latest failed on 7b2eabd in gradle_multi_project_vendor_locked_offline_tamper_and_byte_exact_revert. Gradle itself failed before socket-patch ran: Could not GET 'https://repo.maven.apache.org/maven2/org/apache/commons/commons-text/1.10.0/commons-text-1.10.0.pom'. Received status code 429 from server: Too Many Requests while resolving the buildscript classpath. That's Maven Central rate-limiting the runner, not this diff; the PR doesn't touch Gradle, Maven or the vendor flow beyond the already-merged digest port, and the other 7.6.6 cells (fake-central smoke, both DSLs) passed in the same job. I've already spent this PR's one re-run on the runner-less vlt build, so I'm not re-running this one. A maintainer re-run, or the next push, should clear it.


Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 6, 2026
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator Author

[agent] Ready for review at 7b2eabd. CI: the two infra failures are now green after one re-run each. The vlt lock-diff job had failed because build (ubuntu-latest) never got a runner, and the Gradle 7.6.6 vendor cell had hit a Maven Central 429. All workflow checks are now green. The only non-green items are the CodeQL default-setup Analyze (actions/javascript-typescript) jobs, which the outage cancelled. They aren't required and the API won't re-run them. Bugbot reviewed 7b2eabd with no findings, and there are no open review threads.


Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 6, 2026
@claude
refactor: re-rank queue, slots full
7f3a571
No slot free (#876, #886, #889 ready); main and ranking unchanged.

Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 6, 2026
@claude
refactor: re-rank queue, add #931 (C52)
8ce3bdf
Capacity full (#876, #886, #889 ready). New tracking issue #930
and child #931 ranked; both skipped on file overlap.

Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 6, 2026
@claude
refactor: re-rank queue (3/3 slots, #960 skipped)
ca54c9e
No free slot: #876, #886 and #889 are ready and approved. New #960
is skipped because commands/vendor.rs is changed by open PRs.

Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 7, 2026
@claude
refactor: re-rank queue, add #998 (C56)
a959eb7
Capacity is full (#876, #886, #889 ready), so this run only re-ranks.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit 6bba625 into main Oct 7, 2026
702 of 758 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the arch-refactor/845-bounded-probes branch October 7, 2026 12:03
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 7, 2026
@claude
refactor: record #886, open #1015 (#832, #958)
93a47ca
@mikolalysenko Mikola Lysenko (mikolalysenko) mentioned this pull request Oct 7, 2026
Route the remaining hand-rolled child-process deadlines through utils::process #1067
Open
2 tasks
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 8, 2026
@claude
Bound the PDM site probe through utils::process
7ec4ec7
On macOS, scan reads PDM's site config by asking PDM's own Python
for platformdirs' site path. That probe kept its own deadline: a
tokio timeout around a kill_on_drop child with a private 5 s budget,
beside the output_within spawn every other crawler probe uses since
#886. The two kill policies could drift apart again.

The probe now runs through output_within on the blocking pool, with
the shared 10 s PROBE_TIMEOUT. A hung PDM interpreter is killed and
reaped at that deadline. A probe that answers in time behaves as
before: the same argv, private cwd, scrubbed environment, 4 KiB
output limit and JSON check.

A new guard test fails when production code outside utils::process
spawns a kill_on_drop child. vendor/npm_dir.rs stays on its pending
list until its git exchange moves too (#1067).

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko Mikola Lysenko (mikolalysenko) mentioned this pull request Oct 8, 2026
Bound the PDM site probe through utils::process (#1067) #1106
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments
@Tanmay182003 Tanmay Singla (Tanmay182003) Tanmay182003 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

arch-refactor PR opened by the scheduled architecture refactor routine Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review refactor Structural change: duplicated code or logic, missing abstraction, layering, dead code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Crawler probes spawn gem, python and npm with no timeout, so a hung shim hangs scan forever

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@mikolalysenko @Tanmay182003 @claude
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.