GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Add an opt-in verification gate to watch dispatches by omercangumus · Pull Request #2360 · github/squad

Conversation

omercangumus commented Oct 28, 2026

Copy link

Copy Markdown

Contributor

Summary

Add an opt-in watch.verify capability that runs configured commands, or detected build, test, and lint scripts, after successful dispatches.
Run checks sequentially with fail-fast behavior and surface local verification failures separately from remote CI status.
Document the config and add focused coverage.

Closes #1712

Checks

npm run lint passed.
npm run build -w packages/squad-sdk and npm run build -w packages/squad-cli passed.
Focused watch tests passed (28/28).
Docs lint and spelling passed.
The full test suite could not be completed locally: workflow tests require jq, which is unavailable in this Windows environment, and unrelated integration tests timed out. This PR is draft pending GitHub CI.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

🟡 Impact Analysis — PR #2360

Risk tier: 🟡 MEDIUM

📊 Summary

Metric	Count
Files changed	12
Files added	4
Files modified	8
Files deleted	0
Modules touched	4
Critical files	2

🎯 Risk Factors

12 files changed (6-20 → MEDIUM)
4 modules touched (2-4 → MEDIUM)
Critical files touched: packages/squad-cli/src/cli/commands/watch/capabilities/index.ts, packages/squad-cli/src/cli/commands/watch/index.ts

📦 Modules Affected

docs (1 file)

docs/src/content/docs/features/ralph.md

root (1 file)

.changeset/watch-verification-gate.md

squad-cli (6 files)

packages/squad-cli/src/cli/commands/watch/capabilities/execute.ts
packages/squad-cli/src/cli/commands/watch/capabilities/index.ts
packages/squad-cli/src/cli/commands/watch/capabilities/verification.ts
packages/squad-cli/src/cli/commands/watch/external-loader.ts
packages/squad-cli/src/cli/commands/watch/index.ts
packages/squad-cli/src/cli/commands/watch/types.ts

tests (4 files)

test/cli/watch-execute.test.ts
test/cli/watch-process-tree.test.ts
test/cli/watch-verification.test.ts
test/cli/watch.test.ts

⚠️ Critical Files

packages/squad-cli/src/cli/commands/watch/capabilities/index.ts
packages/squad-cli/src/cli/commands/watch/index.ts

This report is generated automatically for every PR. See #733 for details.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit 02ac551

PR Scope: 📦🔧 Mixed (product + infrastructure)

⚠️ 2 item(s) to address before review

Status	Check	Details
✅	Single commit	1 commit — clean history
✅	Not in draft	Ready for review
✅	Branch up to date	Up to date with dev
❌	Copilot review	No Copilot review yet — it may still be processing
✅	Changeset present	Changeset file found
✅	Scope clean	No .squad/ or docs/proposals/ files
✅	No merge conflicts	No merge conflicts
✅	Copilot threads resolved	No Copilot review threads
❌	CI passing	15 check(s) still running

Files Changed (12 files, +309 −6)

File	+/−
`.changeset/watch-verification-gate.md`	+5 −0
`docs/src/content/docs/features/ralph.md`	+12 −0
`packages/squad-cli/src/cli/commands/watch/capabilities/execute.ts`	+1 −1
`packages/squad-cli/src/cli/commands/watch/capabilities/index.ts`	+2 −0
`packages/squad-cli/src/cli/commands/watch/capabilities/verification.ts`	+97 −0
`packages/squad-cli/src/cli/commands/watch/external-loader.ts`	+1 −1
`packages/squad-cli/src/cli/commands/watch/index.ts`	+38 −2
`packages/squad-cli/src/cli/commands/watch/types.ts`	+1 −1
`test/cli/watch-execute.test.ts`	+22 −0
`test/cli/watch-process-tree.test.ts`	+19 −0
`test/cli/watch-verification.test.ts`	+110 −0
`test/cli/watch.test.ts`	+1 −1

Total: +309 −6

This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

omercangumus marked this pull request as ready for review

October 28, 2026 23:44

omercangumus requested review from bradygaster and tamirdresher as code owners

October 28, 2026 23:44

omercangumus force-pushed the omercangumus/1712-watch-verification-gate branch from 9f564c9 to 075ebde Compare

October 31, 2026 22:32

tamirdresher commented Nov 1, 2026

Copy link

Copy Markdown

Collaborator

Thanks Omer, the sequencing here is solid: verification after post-execute, opt-in, sequential stop on first failure. Two things block this from the security side so I am holding it for now.

In verify: true mode the commands come from package.json in the current working tree, which is the tree the --yolo Ralph session just edited while acting on issue text. That turns any agent file edit, including a pretest hook, into unprompted shell execution with the watch process environment. Our probe hook ran and read an inherited env var. Please pin autodetect to the base ref (git show <base>:package.json) or make explicit commands the only source, and add a SECURITY note like the one external-loader.ts prints.
exec() timeout only kills the shell, so npm run grandchildren survive. Please use the existing runProcessTreeCommand so the tree is reaped.

Separately, the result currently only changes a board counter. Either let it act on the dispatched issue (block the next dispatch) or call it a report rather than a gate.

Happy to re-review at a new head. This one will not make preview.4 as is, no rush.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

omercangumus force-pushed the omercangumus/1712-watch-verification-gate branch from 075ebde to c6a16a5 Compare

November 1, 2026 11:32

omercangumus commented Nov 1, 2026

Copy link

Copy Markdown

Contributor Author

Good catches, thanks. I removed the package.json autodetect; verification now only runs commands already set in watch.verify.commands, with a warning before they run. Timeouts use runProcessTreeCommand, and a failed check stops watch before another dispatch. I updated the docs and tests too. Pushed as c6a16a5; all checks are green.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          Verify work after watch dispatches

02ac551

Run configured checks after successful dispatches and report local failures separately from remote CI.

Closes github#1712

Co-authored-by: Copilot <[email protected]>

omercangumus force-pushed the omercangumus/1712-watch-verification-gate branch from c6a16a5 to 02ac551 Compare

November 1, 2026 19:47

tamirdresher reviewed

Nov 9, 2026

View reviewed changes

Comment thread

packages/squad-cli/src/cli/commands/watch/index.ts

		const verifiedIssues = new Set<number>();
		let untrackedDispatches = 0;
		for (const result of execResults.values()) {
		if (!result.success \|\| !result.data) continue;

tamirdresher Nov 9, 2026

Copy link

Copy Markdown

Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you count dispatched issues even when the executor returns success: false? ExecuteCapability still returns the dispatched count and issue IDs after an agent fails or times out. This guard drops that result, so a round with only failed dispatches never runs the configured checks and can start another dispatch with partially edited files. Please derive the verification gate from dispatched work rather than executor success.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

This branch has not been deployed

No deployments

Reviewers

tamirdresher

tamirdresher left review comments

bradygaster

Awaiting requested review from bradygaster

bradygaster is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

watch: no verification gate - round success is the child process exit code, nothing builds, tests, or lints the resulting work

Uh oh!

There was an error while loading. Please reload this page.

2 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

omercangumus commented Oct 28, 2026

Summary

Checks

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟡 Impact Analysis — PR #2360

📊 Summary

🎯 Risk Factors

📦 Modules Affected

⚠️ Critical Files

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛫 PR Readiness Check

⚠️ 2 item(s) to address before review

Files Changed (12 files, +309 −6)

Uh oh!

tamirdresher commented Nov 1, 2026

Uh oh!

omercangumus commented Nov 1, 2026

Uh oh!

tamirdresher Nov 9, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.