GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: add async ask_user_question behind a hidden experiment by mafredri · Pull Request #32085 · coder/coder

Conversation

@mafredri

@mafredri mafredri commented Nov 12, 2026

Copy link
Copy Markdown
Member

ask_user_question exists only in plan mode, where it ends the agent's turn. An agent left to run cannot ask something and keep working on what does not depend on the answer.

This adds the hidden experiment chat-async-questions. With it enabled, a root chat created by the web UI gets ask_user_question outside plan mode as an async question:

  • The tool is not a stop tool there, so the turn continues.
  • The result carries a new question_id and a status text that tells the model how the answer arrives.
  • The chat owner gets one web push, Question: <first question>, after the step that holds the result commits. A step whose commit fails sends no question push.

With the experiment disabled nothing changes. Plan mode and chats created through the API keep the blocking question in both cases. The experiment is not in ExperimentsSafe, so --experiments='*' does not enable it.

Nothing answers an async question yet: the follow-up PRs add the answer message part and the question panel in the web UI.

Verified with: go test ./coderd/x/chatd/chattool/ -run 'AskUserQuestion|AsyncQuestion', go test ./coderd/x/chatd/ -run 'TestBuildTurnEnvironmentToolPolicy|TestAsyncAskUserQuestionWebPush|WebPush|TestInterruptChat' and make pre-commit. The policy test asserts, per chat kind, whether the tool is offered, which description it has and whether it ends the turn.

Refs CODAGT-1460. Replaces #31310 by @ibetitsmike; the chatd push path is his.

Decision log
  • A minted question_id identifies the question, not the tool call ID. Providers can reuse call IDs, and feat: let web UI chats ask questions without ending the turn #31310's follow-ups needed occurrence matching in the UI because of it.
  • The option schema is unchanged: a label and a description. feat: let web UI chats ask questions without ending the turn #31310 made options label-only.
  • The async result is capped at 16 KiB, the smallest tool-result budget. A longer result would be truncated into invalid JSON and lose its question_id.
  • Question results are read only on turns that offer the async tool, so plan mode and deployments without the experiment run no new code on the tool-result path.
  • When one step asks more than once, only the last question is pushed.
  • Not in this PR: an open-question state in the API, clearing the push when a question is answered or replaced, and questions from subagents.

🤖 Created with the help of Coder Agents.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
mafredri and others added 11 commits November 12, 2026 13:09
@mafredri
feat(codersdk): add the chat-async-questions experiment
fb1636f
Keep async questions hidden until their web UI answer flow is ready.
The experiment is deployment-scoped and remains outside the safe set.
@mafredri
feat(coderd/x/chatd/chattool): add an async mode to ask_user_question
d9a87af
Let questions return an ID that a later user answer can reference.
Bound async result size so tool-result truncation cannot remove that ID,
and preserve the blocking question contract for plan mode.
@mafredri @ibetitsmike
feat(coderd/x/chatd): let web UI chats ask without ending the turn
eb2ad42
Let web UI root chats continue after asking for a user decision when
the hidden experiment is enabled. Send question pushes only after the
tool result commits, without changing plan-mode or API-chat behavior.

Co-authored-by: Michael Suchacz <[email protected]>
@mafredri
fix(coderd/x/chatd): read question results only on turns with the asy…
5e1432e
…nc tool

Keep blocking question results and tools from other sources out of the
async question push path when a turn does not offer the async tool.
@mafredri
test(coderd/x/chatd): assert which ask_user_question mode a turn gets
da85012
Distinguish blocking questions from async questions even when the tool
name and turn-stop policy match. Preserve the API plan-root coverage
while checking the turn environment flag for every policy case.
@mafredri
fix(coderd/x/chatd/chattool): accept pointer outputs in AsyncQuestion
91a0d90
Use the shared Fantasy conversion so a text result works in both its
value and pointer forms, as it does elsewhere in the generation path.
@mafredri
test(coderd/x/chatd): make the question push test state its scenarios
e7d974e
Expose the running-turn assertions in the continued-execution scenario
and distinguish the completion push from the question push. Use named
options for each test setup and remove the redundant error assertion.
@mafredri
refactor(coderd/x/chatd): clarify comments on the question push path
d9fb35b
Keep the comments precise about tool configuration and post-commit
question pushes, and separate body preparation from scheduling.
@mafredri
refactor(coderd/x/chatd/chattool): improve readability of ask_user_qu…
3478262
…estion

Move the async result into its own function so Run shows the async and
blocking branches side by side, and break up the "too long" message.
In the tests, write the JSON inputs as multi-line raw strings and
separate arrange, act and assert in each subtest.
@mafredri
refactor(coderd/x/chatd): improve readability of the question push test
b89bc80
Name the fake model's three behaviors, the tool call input, the state
passed to the held-call callback and the turn result, and move the
committed question_id lookup into a helper, so each subtest reads top
to bottom. The assertions are unchanged.
@mafredri
refactor(coderd/x/chatd): improve readability of the ask_user_questio…
199a4b4
…n policy

Name the client type and experiment conditions that select the async
tool, document the new tool-policy test fields, and separate the
ask_user_question assertions from the toolset assertions.
@mafredri
mafredri temporarily deployed to main November 12, 2026 13:10 — with GitHub Actions Inactive
@mafredri
mafredri temporarily deployed to main November 12, 2026 13:10 — with GitHub Actions Inactive
@linear-code

linear-code Bot commented Nov 12, 2026

Copy link
Copy Markdown

CODAGT-1460

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot assigned mafredri Nov 12, 2026

This branch was previously deployed

1 inactive deployment
main — 199a4b4c Deployed Nov 12, 2026 by mafredri via cla #254924
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@mafredri mafredri

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@mafredri
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.