GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — feat(coderd/x/chatd): resolve execute tool calls on user interrupt by mafredri · Pull Request #29976 · coder/coder Conversation
|
| Term | Meaning | Replaces |
|---|---|---|
| model | The LLM. Never called "agent" here. | |
| workspace agent (agent) | The Coder agent process running in the workspace. | |
| tool call | One call the model made, committed in an assistant message (existing term). | |
| tool call message ID | The chat message ID (chat_messages.id) of the assistant message that contains the tool call. |
|
| provider tool call ID | The ID the model provider gave the tool call. Not guaranteed unique across messages. | |
| unresolved tool call | A tool call without a committed result (existing code term, unresolvedToolCallsFromHistory). |
|
| task retry | chatd rerunning a generation attempt after an error or the 15m attempt timeout (existing, runTaskWithRetry). |
|
| ownership change | Another coderd replica takes over the chat (existing). | |
| tool call age | Time since the tool call was committed, computed from the database clock. | ID age, call_age |
| tool call headers | Coder-Tool-Call-Message-Id, Coder-Tool-Call-Id, Coder-Tool-Call-Age-Ms, sent next to the existing Coder-Chat-Id on every request with a side effect. |
Idempotency-Key, op_id |
| tool call UUID | A UUIDv5 of chat ID, tool call message ID, and provider tool call ID, computed by one shared workspacesdk function on both sides. Used as the process ID of a process the tool call starts, and as the ID in the cancel route. |
|
| latest message ID | Per chat, the highest tool call message ID the agent has received. | |
| agent start | When the agent began keeping tool call records (agent process startup). | |
stale_tool_call |
Error code: the tool call's message is older than the chat's latest message ID, so chatd has already resolved it. | |
agent_started_after_tool_call |
Error code: the agent has no record of the tool call and started after the tool call was committed, so it cannot tell whether an earlier agent ran it. | created_before_agent_start, unknown |
tool_call_canceled |
Error code: a request with tool call headers arrived after a cancel recorded the tool call as canceled. Only a stale request can get it. | |
| tool call middleware | Agent HTTP middleware in agenttoolcall that applies the decision table to the start, edit, and write routes, runs the handler at most once per tool call, and records its response. |
per-route records |
| recorded response | Status code, Content-Type, and body of the one run of a tool call's request. Repeated requests get it back unchanged. |
stored edit result |
| run age | Time since the agent first ran a tool call's request, measured by the agent from the record's creation. A process started for the tool call takes that instant as its start time, so for execute the run age is the process age. Returned in the Coder-Tool-Call-Run-Age-Ms response header and in cancel answers. |
process age, age_ms |
stop_if_run_age_below_ms |
Cancel parameter: stop a running process only if its run age is below this value. chatd passes the execute timeout for a foreground call and 0 (never stop) otherwise. | |
| execute deadline | When a foreground execute stops waiting: process start plus the timeout argument. |
budget, anchor |
| agent answer timeout | How long chatd waits for the agent to answer one start, edit, write, or cancel request before it reports unknown: 1 minute. It bounds the wait for the answer, never the work. | fallback deadline, backstop |
Spelling: canceled, matching the codebase (514 uses versus 5 of cancelled in coderd, agent, codersdk).
Each story ends with the observable condition that proves it.
execute. The process is killed and the result says it was canceled by the user, with the partial output. Proof: the process is gone after the interrupt; the result is not "interrupted before it produced a result".edit_files or write_file is in flight, the result is the real edit result, "not applied", or "unknown" with the reason. (Design requirement: the contract must support it. Implemented in the final PR of the stack, after execute.)process_signal or agent shutdown ends it.tool_call_canceled.| Trigger | What happens | Evidence |
|---|---|---|
| Task retry after the 15m attempt timeout while a tool runs | Every unresolved tool call in the batch runs again; execute starts a new process each cycle |
tasks.go:28, tasks.go:155-174; CODAGT-757 incident, 36 starts in 9h (Linear, 2026-08-27) |
Any task retry after tools ran, before CommitStep |
Same; results are committed together after the whole batch | ARCHITECTURE.md:884 |
| Ownership change mid-call | The new owner runs unresolved tool calls again | ARCHITECTURE.md:882 ("at least once") |
| User interrupt | chatd stops waiting; the process keeps running; unresolved tool calls get "tool call was interrupted before it produced a result" | tasks.go:316, tasks.go:696-773, message_conversion.go:27; ARCHITECTURE.md:880 |
| Response from the agent lost | chatd cannot refer to the process: the agent picks process IDs | agent/agentproc/process.go:120 |
Root cause: the agent picks the ID of every side effect, so chatd cannot refer to it again without the original response. And chatd treats "I stopped waiting" as "the work stopped".
Rejected:
PUT to client-chosen URLs (earlier revision). It replaced existing routes; headers keep the existing API.| Request | Route | Change |
|---|---|---|
| start a process | POST /api/v0/processes/start |
Existing route plus tool call headers, through the tool call middleware. The process ID is the tool call UUID. Response header Coder-Tool-Call-Run-Age-Ms. |
| wait for output | GET /api/v0/processes/{id}/output |
Unchanged. |
| signal a process | POST /api/v0/processes/{id}/signal |
Unchanged; used by the model's process_signal tool. |
| edit files | POST /api/v0/edit-files |
Existing route plus tool call headers, through the tool call middleware (final PR). |
| write a file | POST /api/v0/write-file |
Existing route plus tool call headers, through the tool call middleware (final PR). |
| cancel a tool call | POST /api/v0/tool-calls/{id}/cancel |
New, one route for every tool. Tool call headers plus body {"stop_if_run_age_below_ms": N}. {id} is the tool call UUID. Returns whether the request ever ran, its recorded response, and the state of the process it started, if any. |
Every tool with a side effect uses the same headers, the same middleware, and the same cancel route; only the process stop rule is tool specific.
For a start, edit, or cancel request in chat C, for a tool call in message M:
| Condition | Start, edit, or write | Cancel |
|---|---|---|
| M is older than C's latest message ID | 409 stale_tool_call, nothing happens |
409 stale_tool_call |
| The agent has a record of the tool call | Return the recorded response (waiting for a run in progress); 409 input_mismatch if the request differs; 409 tool_call_canceled if a cancel recorded it |
Wait for a run in progress. If it started a process that is running with run age below stop_if_run_age_below_ms, kill it; otherwise leave it running. Return the recorded response and the process state |
| No record, and the agent started more than the tool call age plus a small margin ago | Run the handler once, record its response, and raise C's latest message ID to M if higher | Record the tool call as canceled so a request still in transit does nothing; report that it never started |
| No record, and the agent started less than that ago | 409 agent_started_after_tool_call, nothing happens |
409 agent_started_after_tool_call |
When C's latest message ID rises, the agent drops the tool call records of older messages in C. Processes themselves are unaffected. A background process keeps running and stays addressable by its process ID for process_output and process_signal, and it is removed after exit as today.
unresolvedToolCallsFromHistory reads the last assistant message only (generation.go:295-305). A new assistant message is generated only after every local tool call of the previous one has a result (generation.go:207-218).chat_messages_id_seq, dump.sql:2082). Every transition locks the chat row before inserting (ARCHITECTURE.md:98), so IDs increase within a chat.agent_started_after_tool_call.State per chat: one latest message ID, plus records for that message's tool calls. It grows with the number of chats, not the number of tool calls.
context.Background(), process.go:126-129; edits do not check the request context.)api.go:167-177, api.go:228-237) stays. Cancel needs no ownership lookup: its path ID must equal the UUID derived from the request's own chat ID and headers, otherwise 400. Requests with tool call headers but without Coder-Chat-Id are rejected with 400. The tool call UUID includes the chat ID, so two chats never derive the same process ID.Coder-Tool-Call-Age-Ms and returns run ages (Coder-Tool-Call-Run-Age-Ms, and in cancel answers). Both are durations.agent_started_after_tool_call rather than a second run.StartInterrupt) sends one CancelToolCall for every unresolved tool call whose tool sends tool call headers (execute; edit_files and write_file in the final PR), with stop_if_run_age_below_ms set to the effective timeout for a foreground execute and 0 otherwise, and commits results built from the answers. It waits for the answers; its timeout exists only so an unreachable agent yields "unknown". A process past its execute deadline, or started in the background, is never stopped (US6).timeout - run age. Every start, edit, write, and cancel request waits at most the agent answer timeout for its answer; when that expires while the task still runs, the result is unknown with the tool call UUID. When the wait for output reaches the execute deadline and the output cannot be read either, the result says the outcome is unknown, not that the command timed out.| Agent answer | execute result |
edit_files result |
|---|---|---|
| exited / recorded edit response | exit code and output | the edit result |
| running at the execute deadline, or left running by cancel | timed out, still running, background_process_id (today's behavior); background: the background start result |
n/a |
| canceled after it started | canceled by the user after the run age, partial output | n/a (edits finish instead) |
| canceled before it arrived | not run (foreground and background) | not applied |
409 agent_started_after_tool_call |
unknown: the workspace agent restarted after this tool call; the command may have run | unknown: may have been applied; check the file |
409 stale_tool_call or tool_call_canceled |
none: only a stale goroutine can get this, and its commit fails the history-version fence | same |
| agent unreachable | unknown: the agent could not be reached | unknown, same reason |
I confirmed with a throwaway chi test, mirroring today's process routes, that the new paths return 404. The test has been removed.
The record store is one per agent, in agenttoolcall: keyed by chat ID, tool call message ID, and provider tool call ID, holding each chat's latest message ID, the agent start, and each tool call's recorded response. The tool call middleware wraps the start, edit, and write routes, so those handlers stay unaware of repeats. The one cancel handler lives next to the store and asks process management to stop a tool call's process under the run-age rule; file tools need no stop step. Process management keeps owning output, exit state, and signals.
execute (foreground and background) first, because it has the worst failure today: duplicate processes that keep running. edit_files and write_file in the final PR, using the same headers, decision table, and cancel route. write_file replaces the whole file (agent/agentfiles/files.go:302, files.go:574-629), so repeating it is less harmful than repeating edit_files, but it follows the same contract.read_file, ls, process_output, process_list. They have no side effect, so running them again is harmless.spawn_agent, attach_file (CODAGT-875), and create_workspace. The same idea (identity from the tool call, enforced by a database constraint) applies separately.| Keep | Change | Drop |
|---|---|---|
| Records in agent memory, scoped by chat | The tool call identity moves into headers, and the process ID is derived from it; no key-to-process registry | Signal-by-idempotency-key route |
| Identity from the tool call message ID plus the provider tool call ID | With no record, the agent starts only if it has been running since before the tool call was committed; otherwise agent_started_after_tool_call |
#27371 watchdog: a later attempt has no side effect |
input_mismatch |
Records are dropped when the chat's latest message ID rises, not after a time | Committed error result for start_pending |
| Task retries and ownership changes never cancel; background processes are never canceled | Cancel records unknown tool calls as canceled; #27372 reads a 404 as "already gone" | StartedAt compared against coderd's clock |
| A later attempt waits on the existing process for the rest of the timeout | Interrupt handling sends cancel and builds the result; #27372 kills from a deferred call in the tool goroutine being canceled, while the committed result stays generic | 60-minute time-based retention |
edit_files and write_file use the same shape (final PR) |
The message ID was questioned in #27370 review as possibly unnecessary. It is now required: it is how the agent knows which records it may drop.
agent_started_after_tool_call and report "unknown". Persisting records to disk would not bring the processes back.agent_started_after_tool_call even if no request ever reached an earlier agent. That takes an agent restart between committing the tool call and its first request, or a request landing within the margin right after workspace start./write-file still streams.edit_files can partially apply on a disk error (agent/agentfiles/files.go:452-454). The recorded result is the handler's existing error response, which does not list the files written before the failure. Unchanged behavior; the recording only replays it.ARCHITECTURE.md:880-882 ("best-effort" interrupts, "at least once") would change. Per repo rules, the implementing PR leaves TODOs there for the human author.Decided (operator, 2026-09-25):
execute only. A process_output wait on a background process has no side effect and is not canceled. If CODAGT-1108's combined process view confuses users later, extending C5 is a small follow-up. (From the final PR on, unresolved edit_files and write_file calls also get a cancel, but only to learn their state; an edit in progress still finishes.)PUT (operator, 2026-09-25).execute first; edit_files and write_file in the final PR of the stack. They are in the design so the contract is general, not execute-shaped. write_file gets no special exception.Coder-Tool-Call-* headers, not one structured header that the agent would have to parse.Open: none.
This section is the single source of truth for names, signatures, and PR boundaries. Anything not listed here is out of scope; a deviation needs a user story (section 1 style) that motivates it. Revision 2 (Q8) replaces the per-route cancel routes, the per-package record tables, and the age_ms body fields of revision 1. All six PRs are published drafts; rework lands as new commits and children merge their parents (no force-push).
| # | PR | Branch (mafredri/codagt-757/...) |
Base | Content after revision 2 |
|---|---|---|---|---|
| 1 | #29946 | 01-sdk-tool-call |
main |
codersdk/workspacesdk: tool call identity, headers (incl. run age), UUID, per-request headers, error codes, CancelToolCall client. |
| 2 | #29954 | 02-agent-tool-call-records |
1 | agent/agenttoolcall: the record store, the tool call middleware, the cancel handler, the error table. |
| 3 | #29958 | 03-agent-process-tool-calls |
2 | agent/agentproc and wiring: start runs through the middleware with the tool call UUID as process ID; the process stop rule; reaping rule. |
| 4 | #29965 | 04-chatd-execute-tool-calls |
3 | coderd/x/chatd: dispatch identity, tool call age, execute sends headers, attaches, deadline from run age, result mapping, the shared error classifier. |
| 5 | #29976 | 05-chatd-interrupt-cancel |
4 | coderd/x/chatd: interrupt handling sends CancelToolCall for every unresolved tool call and commits results from the answers. |
| 6 | #29978 | 06-file-tool-calls |
5 | edit_files and write_file through the same middleware and cancel route; SDK result rebuilding; chatd file tools. |
codersdk/workspacesdk)Unchanged from revision 1: ToolCall, SetHeaders, ToolCallFromHeaders, ToolCallUUIDNamespace, ToolCallUUID (test vector: chat aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee, message 42, tool call "toolu_01/%x y" gives 1a9cb501-66b5-51bd-a7c6-1bf0e500b268), WithToolCall, ToolCallFromContext, ToolCallErrorCode and its four codes, ToolCallError, and the coded-409 decoding helper.
Changes:
const CoderToolCallRunAgeMsHeader = "Coder-Tool-Call-Run-Age-Ms" // decimal int64 >= 0, set by the agent on the recorded or replayed response of a tool call's request and on a cancel answer for a tool call that ran // StartProcessResponse: the revision 1 AgeMs body field is removed. The // body of a replayed start is the recorded one, so a live value travels in // the response header instead. type StartProcessResponse struct { ID string `json:"id"` Started bool `json:"started"` RunAge time.Duration `json:"-"` // from CoderToolCallRunAgeMsHeader; zero when absent (old agent or no tool call headers) } // ProcessOutputResponse.AgeMs is removed (no consumer after revision 2). type CancelToolCallRequest struct { // StopIfRunAgeBelowMs stops a running process only if its run age is // below this value. 0 never stops. StopIfRunAgeBelowMs int64 `json:"stop_if_run_age_below_ms"` } type CancelToolCallResponse struct { // Started is false when the agent never ran the tool call's request. // The agent has recorded it as canceled, so it never will. Started bool `json:"started"` // StatusCode, ContentType, and Body are the recorded response of the // tool call's request when Started. Body round-trips byte for byte. StatusCode int `json:"status_code,omitempty"` ContentType string `json:"content_type,omitempty"` Body []byte `json:"body,omitempty"` // Process is the state of the process the tool call started, if any. Process *ToolCallProcess `json:"process,omitempty"` } type ToolCallProcess struct { Running bool `json:"running"` Canceled bool `json:"canceled"` // a cancel request killed it; repeated cancels report it too Output string `json:"output,omitempty"` Truncated *ProcessTruncation `json:"truncated,omitempty"` ExitCode *int `json:"exit_code,omitempty"` RunAgeMs int64 `json:"run_age_ms"` }
Age plus the time spent connecting, and send over that connection. Requests without a tool call are unchanged.AgentConn: CancelProcess and CancelProcessResponse are replaced by CancelToolCall(ctx context.Context, id string, req CancelToolCallRequest) (CancelToolCallResponse, error): POST /api/v0/tool-calls/{id}/cancel, headers from WithToolCall, coded 409s as *ToolCallError. StartProcess fills RunAge from the header. Regenerate agentconnmock.agent/agenttoolcall)Revision 1's generic Records[V] and PR 6's Chats merge into one store of recorded responses. The decision rules and their order are unchanged (stale; record exists; clock.Since(start) - ageMargin <= age means the agent started after the tool call; otherwise run once), ageMargin = 2 * time.Second, ages clamped at 0, per-chat state kept for the agent's lifetime, older messages' records dropped when a chat's latest message ID rises, waiters wake on success, error, and panic.
type Store, one per agent, func NewStore(clock quartz.Clock) *Store (the agent start is clock.Now()).func (s *Store) Middleware(next http.Handler) http.Handler:
next unchanged.agentchat.FromContext) or with malformed headers. Input hash = SHA-256 over method, URL path, raw query, and the full request body (the body is buffered; a new body reader is handed to next).next, runs next once with a response recorder, records status, Content-Type, and body, and writes them to the client. Repeated requests get the recorded response. Every answered response (new or replayed) carries Coder-Tool-Call-Run-Age-Ms, the time since the record was created on the store's clock.ToolCallError bodies from one table of codes and sentence-case messages in this package.func FromContext(ctx context.Context) (ToolCall, bool) for handlers, where ToolCall holds the Key and the tool call UUID.func (s *Store) Current(key Key) bool (the key is in its chat's latest message and has a record).POST /api/v0/tool-calls/{id}/cancel:
// ProcessStopper stops the process a tool call started, under the run-age rule. type ProcessStopper interface { StopToolCallProcess(ctx context.Context, processID string, stopIfRunAgeBelow time.Duration) (state workspacesdk.ToolCallProcess, found bool, err error) } func (s *Store) CancelHandler(stopper ProcessStopper) http.HandlerFunc
{id} must equal the UUID derived from the request's own chat ID and headers (400). No record and provable: record canceled, answer started: false. Pending run: wait. Started: ask stopper (nil allowed), answer the recorded response plus Process when found. Sentinel errors: 409 codes. Request context ended: return without writing.agent/agentproc, agent/agent.go, agent/api.go)store.Middleware (store passed to agentproc.NewAPI through an option). With a tool call in context, manager.start uses the tool call UUID as process ID and remembers the key; otherwise unchanged.agentproc implements ProcessStopper: find the process by ID; running with run age below the threshold: SIGKILL the process group, mark it canceled by a cancel, wait for exit (bounded by ctx); running at or above: leave it; exited: final state. Report Canceled for a process a cancel killed, on every later cancel too. Run age from the process's own start time on the manager clock.store.Current is not reaped; otherwise the 5-minute rule is unchanged.POST /processes/{id}/cancel, age_ms in bodies, WithToolCallChats.agent/agent.go builds one agenttoolcall.Store on the agent clock; agent/api.go mounts the cancel route with processAPI as stopper.Close kills, one git watcher notification, chat isolation) and add the run-age rule: below threshold kills, at or above leaves running, 0 never stops.coderd/x/chatd)Unchanged from revision 1 except:
StartProcessResponse.RunAge (clamped at 0).chattool holds the agent answer timeout (1 minute). Start requests wait at most that long; when it expires while the tool's context is still live, the result is unknown with the tool call UUID. The foreground start still also ends at the execute timeout, whichever comes first.processSnapshotResult removed (waitForProcess with an expired context gives the same result and texts; prove it with the existing tests); chatloop takes one batch-level ToolCallIdentity instead of three fields and the toolCallBatch type; small helpers for attaching headers, the connection-error check, age calculation, the default timeout, and message reload, each only where it removes duplication.execute keep today's result), input_mismatch wording, the regression tests, the one ARCHITECTURE.md TODO line.coderd/x/chatd)CancelToolCall per call, in parallel under the agent answer timeout, outside the locked transaction. stop_if_run_age_below_ms: the effective timeout for a foreground execute, 0 for background execute (and for file tools in PR 6).chattool; tasks.go only collects results and commits them. Foreground: not started gives "not run"; Process.Canceled gives canceled by the user after the run age with partial output; Process.Running gives the timed-out, still running result with background_process_id; exited gives the real result. Background: not started gives "not run" (US10); running or exited gives the background start result with background_process_id. A start recorded as successful whose process the agent no longer reports gives the background start result (background) or unknown with the tool call UUID (foreground). Errors go through PR 4's classifier; old agent (404 without a code), no workspace, deleted workspace, and no agent for execute keep today's result; unreachable and unreadable give unknown with the tool call UUID.ProcessOutput pre-checks, InterruptBackgroundExecute, the per-tool branching in tasks.go.ARCHITECTURE.md TODO line.POST /edit-files and POST /write-file are wrapped by the same store.Middleware (store passed to agentfiles.NewAPI through an option). Removed: agentfiles/toolcall.go, the fileResult refactor in files.go, per-route input hashing, the file cancel routes, WithToolCallChats, the Chats type. The file handlers go back to today's code apart from what the middleware needs.EditFiles and WriteFile return *ToolCallError for coded 409s; CancelToolCallResponse gets methods that rebuild exactly what EditFiles or WriteFile would have returned from the recorded response. CancelEditFiles and CancelWriteFile are removed.edit_files and write_file keep their identity, classifier, and wording from revision 1; interrupt handling uses PR 5's one path with stop_if_run_age_below_ms 0 and PR 6's result rebuilding. chattool.InterruptedCall decides per tool whether a call gets a cancel and maps the answer, so tasks.go has no tool switch and one converter for every committed interrupt result. The two ARCHITECTURE.md TODO lines cover file tools.chatloop.TruncateToolResult). Story: an interrupted edit_files call whose recorded result holds a multi-megabyte diff would otherwise be committed uncut and overflow the next prompt, while the same result arriving live is truncated.TestAgent_ToolCalls (real AgentConn through the agent router) covers the one cancel route for a process and an edit, a stale request, and the run-age rule.🤖 This PR was created with the help of Coder Agents, and will be reviewed by a human. 🏂🏻
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
ebecc64 to
62a44a8
Compare
2094e2f to
aae8893
Compare
… run An interrupt left the processes of unresolved execute calls running and committed "tool call was interrupted before it produced a result" for them. For a capable workspace agent, the interrupt task now runs the interrupt run before FinishInterruption: it dispatches every unresolved call of a tool that sends tool call headers through chatloop.ExecuteLocalTools with the interrupt cause, using the tools generation builds, so each request matches the one the agent recorded. In the interrupt run, execute cancels the tool call first. After a 204 it sends its start again and builds its normal result: canceled with partial output, completed, not run when the agent never received the start, or unknown. A background call reads its output once, and the output wait is bounded by the agent answer timeout. A failed cancel is never followed by the start: a plain 404 keeps the generic result, anything else is unknown. An agent that is not capable gets no request and every call keeps the generic result. A canceled task context commits nothing.
aae8893 to
7e709ed
Compare
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.