GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Read NuGet and Cargo crawler project files through the FIFO-safe reader (#592) by mikolalysenko · Pull Request #602 · SocketDev/socket-patch

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 2, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #592

Summary

A FIFO in a checkout at obj/project.assets.json (NuGet) or vendor/<crate>/Cargo.toml (Cargo) blocked open(2) forever, so scan, get and apply hung on that repository. Those reads, and the Pipenv .venv read, now go through the shared FIFO-safe readers in utils::fs. A new architecture test stops crawler production code from using a bare file read again.

Why (leverage)

  • Issue #592, register row E06 (register), living document Part 6.6 "FIFO safety" (doc/06-discovery-vex.md).
  • B=1, U=0, D=0, R=L, score 3. This is the top eligible candidate. The higher-scored ones all touch files that open PRs change: Read and splice nuget.config through formats::nuget in hosted, vendored and restore #594/Hosted NuGet rewrites packages.lock.json entries at other versions of the patched id #593 (redirect/mod.rs, Unify hosted NuGet routing and XML splice anchors #597), Vendored→hosted takeover drops the vendored ledger entry when the revert drift-keeps the artifact #568 (scan/hosted.rs, Fix hosted scan from a workspace member pinning nothing or the wrong files (#590, #417) #598) and E37 (vendor/cargo.rs, Fix hosted scan from a workspace member pinning nothing or the wrong files (#590, #417) #598).

What changed

  • crawlers/nuget_crawler.rs parse_project_assets_package_folders: tokio::fs::read_to_string → utils::fs::read_regular_to_string.
  • crawlers/cargo_crawler.rs verify_crate_at_path (find_by_purls): tokio::fs::read_to_string → read_regular_to_string. read_crate_cargo_toml (blocking-pool crawl_all): std::fs::read_to_string → read_regular_to_string_sync.
  • crawlers/python_crawler.rs Pipenv .venv: std::fs::read_to_string → read_regular_to_string_sync. Before, only the is_file() check guarded this read, which left a TOCTOU window. The issue called this read out of scope, but the new guard needs it fixed; it is one line.
  • crawlers/mod.rs: architecture_tests::crawlers_read_project_files_fifo_safely fails on any fs::read_to_string(, fs::read( or File::open( in crawler production code. The only allowlisted reads are the two machine-repository Maven POM reads in maven_crawler.rs, which NuGet and Cargo crawlers hang on a FIFO at obj/project.assets.json or vendor/<crate>/Cargo.toml #592 left out of scope.

Deleted / diff

git diff --stat origin/main...HEAD: 4 files, +245 / −4.

  • Production: +4 / −4. These are call-site swaps; no new production code.
  • Tests: +241. That covers two FIFO regression tests, their helpers and the architecture guard.

Behavior

None for regular files. Unreadable entries are still skipped. A non-regular file (FIFO, device, directory) at one of these paths is now skipped right away instead of hanging.

Test evidence

  • Red on main's reads (the three call sites temporarily reverted on this branch):
    • fifo_crate_manifest_is_skipped_not_blocked_on FAILED (5 s deadline);
    • fifo_project_assets_is_skipped_not_blocked_on FAILED (5 s deadline);
    • crawlers_read_project_files_fifo_safely FAILED.
  • Green on the branch: all three pass. cargo test -p socket-patch-core --lib -- crawlers: 516 passed.
  • cargo test -p socket-patch-core --lib: 4843 passed. The 4 failures are the known root-sandbox ones that also fail on main: copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_maps_error_and_leaves_lock_untouched and pypi_requirements::wire_failure_rolls_back_already_written_files. They pass in CI.
  • cargo clippy --workspace --all-features -- -D warnings: clean.
  • cargo test -p socket-patch-cli --all-features with --test in_process_scan, in_process_cargo_apply, crawl_fd_limit_e2e, e2e_nuget and e2e_cargo: all green.
  • CI: all 341 checks green on a024719. Before that, ac9bece failed only test (windows-latest). On a CRLF checkout the guard's #[cfg(test)]\nmod tests marker never matched, so it counted a test-only Maven read as production code. The fix is a024719, which normalizes CRLF first. Reproduced locally with a CRLF maven_crawler.rs: before the fix it reports 3 reads against 2 allowed and fails; after, it passes.

Bugbot round 1 found that the test watchdog's blocking writer open could hang the suite. That is fixed in ac9bece (O_NONBLOCK), and later rounds, including one on a024719, are clean.

Risk

Low. Four one-line call-site swaps onto readers that every other crawler already uses. The FIFO regression tests are #[cfg(unix)]; the architecture guard runs on every platform.

🤖 Generated with Claude Code


Generated by Claude Code

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@claude
Start refactor for #592
d58a314
Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko Mikola Lysenko (mikolalysenko) added refactor Structural change: duplicated code or logic, missing abstraction, layering, dead code arch-refactor PR opened by the scheduled architecture refactor routine labels Oct 2, 2026
@mikolalysenko Mikola Lysenko (mikolalysenko) mentioned this pull request Oct 2, 2026
NuGet and Cargo crawlers hang on a FIFO at obj/project.assets.json or vendor/<crate>/Cargo.toml #592
Closed
2 tasks
Claude (claude) added 2 commits October 2, 2026 20:12
@claude
Read crawler project files FIFO-safely
7b1ef3f
A FIFO planted in a checkout at obj/project.assets.json or at
vendor/<crate>/Cargo.toml blocked open(2) forever, so scan, get and
apply hung on that repository. The NuGet assets read, both Cargo
crate-manifest reads and the Pipenv .venv read now go through the
shared FIFO-safe readers in utils::fs; an unreadable entry is skipped
exactly as before.

Fixes #592.

Assisted-by: Claude Code:claude-opus-5-5
@claude
Guard crawlers against bare file reads
a20aade
An architecture test now fails when crawler production code reads a
file with a bare read_to_string, fs::read or File::open, so the FIFO
hang fixed for #592 cannot come back. The two machine-repository
Maven POM reads are allowlisted by count.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 2, 2026 20:24
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 2, 2026
@claude
refactor: run 2026-10-02T19:55Z, PR #602 for #592
20ad6f8
Assisted-by: Claude Code:claude-opus-5-5
cursor[bot]
cursor Bot reviewed Oct 2, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread crates/socket-patch-core/src/crawlers/cargo_crawler.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.

@claude
Release FIFO test watchdog without blocking
ac9bece
The timeout cleanup in the #592 FIFO regression tests opened the FIFO
for writing in blocking mode. If no reader was blocked on it (the
timeout had another cause), that open itself hung the suite. Open it
with O_NONBLOCK so it fails with ENXIO instead.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cursor[bot]
cursor Bot reviewed Oct 2, 2026
View reviewed changes

@cursor cursor Bot left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@claude
Match test-module markers on CRLF checkouts
a024719
On a Windows core.autocrlf checkout the crawler guard's
"#[cfg(test)]\nmod tests" marker never matched, so it counted a
test-only Maven read as production code and failed the Windows
test job. Normalize CRLF to LF before locating the test module.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 2, 2026
@claude
refactor: note CRLF gotcha from #602
59a2607
Assisted-by: Claude Code:claude-opus-5-5
cursor[bot]
cursor Bot reviewed Oct 2, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a024719. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mikolalysenko Mikola Lysenko (mikolalysenko) mentioned this pull request Oct 2, 2026
Stream patch blob and diff downloads to disk (#571) #607
Merged
@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 2, 2026
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator Author

[burn-down agent] Labeled Ready for review at a024719ab281f48223153f1ec641e07aa450d585.

  • CI: 335/341 check runs green on this head (6 skipped by path/matrix filters), 0 failing.
  • Bugbot: reviewed this exact head, no findings; its one earlier thread (cargo_crawler.rs) is resolved.
  • Mergeable; 1 commit behind main (045d7ec, API client only, no overlap with these crawler files).
  • Reviewer focus: the new guard test in crawlers/mod.rs that forbids bare read_to_string in crawlers, and CRLF marker matching.

Generated by Claude Code

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator Author

Reviewed a024719ab281f48223153f1ec641e07aa450d585: ready to merge as-is; no actionable findings.

The four call-site changes preserve regular-file and symlink behavior while using the existing descriptor-based guard to reject FIFOs and other nonregular inputs. Errors still skip the affected entry, and discovery continues for readable siblings. The watchdog cleanup and CRLF-aware architecture guard also look correct.

Validation: 517 crawler tests passed. Independent public-API tests passed 33 file-kind scenarios across Cargo, NuGet, and Pipenv; the CRLF guard control and two shared-reader contract tests also passed. These local checks ran on macOS; I did not rebuild the full CLI/network matrix.

The exact head has 335 successful and seven skipped checks, a clean Bugbot review, and no unresolved threads. It merges cleanly with current main 045d7ec783d788bf3c5a1310724b51e09fb6505d.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Oct 3, 2026
Move the crawler coordinate guards and composer's leading-v rule into utils and delete the copies #630
Closed
Read Cargo.toml package name and version through one shared toml_edit reader #693
Closed
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 3, 2026
@claude
refactor: re-rank at capacity, compact history
ffe3ebc
Still at capacity with #574, #602 and #607 awaiting human merge;
queue unchanged. Compacts the hourly re-rank history.

Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 3, 2026
@claude
refactor: re-rank queue, #717 enters first
26c7a6e
Still at capacity (#574, #602, #607 ready). No new work.

Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 3, 2026
@claude
refactor register: re-rank with #726-#728
72c14a8
At capacity (#574, #602, #607 ready); #728 enters the queue
fourth, #663 drops out.

Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 4, 2026
@claude
refactor: re-rank at capacity (01:56Z)
babe413
Still at capacity with #574, #602 and #607 ready; nothing merged
and no new arch-audit or refactor issues, so the queue holds.

Assisted-by: Claude Code:claude-opus-5-5
Tanmay182003
Tanmay Singla (Tanmay182003) approved these changes Oct 5, 2026
View reviewed changes
@mikolalysenko Mikola Lysenko (mikolalysenko) mentioned this pull request Oct 5, 2026
Crawler probes spawn gem, python and npm with no timeout, so a hung shim hangs scan forever #845
Closed
4 tasks
@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit 2eae9a0 into main Oct 5, 2026
342 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the arch-refactor/592-crawler-fifo-reads branch October 5, 2026 11:17
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 5, 2026
@claude
refactor: record #602 merge (E06 fixed)
a394ed2
Assisted-by: Claude Code:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments
@Tanmay182003 Tanmay Singla (Tanmay182003) Tanmay182003 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

arch-refactor PR opened by the scheduled architecture refactor routine Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review refactor Structural change: duplicated code or logic, missing abstraction, layering, dead code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

NuGet and Cargo crawlers hang on a FIFO at obj/project.assets.json or vendor/<crate>/Cargo.toml

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@mikolalysenko @Tanmay182003 @claude
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.