GeezerVison

Copy link

Copy Markdown

Contributor

kylecarbs reacted with thumbs up emoji

bryphe-coder commented Jan 3, 2022

View reviewed changes

bryphe-coder commented Jan 3, 2022

View reviewed changes

bryphe-coder requested a review from kylecarbs January 3, 2022 19:46

kylecarbs approved these changes Jan 3, 2022

View reviewed changes

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bryphe-coder deleted the bryphe/chore/initial-ci branch January 4, 2022 02:54

command correction

Prior sub-tests call CreateWorkspaceBuild through the API, which
triggers SoftDeletePriorWorkspaceAgents and soft-deletes the agent
from build #1. Re-fetch the workspace to get the current (non-deleted)
agent ID instead of using the stale one captured at setup time.

…nges (#27059)

Closes CODAGT-592.

## Problem

The advisor tool sometimes fails with the opaque error `advisor produced
no text output`. Live reproduction against `claude-sonnet-4-6` showed
the cause: `BuildAdvisorMessages` forwards the parent conversation's raw
`tool_use`/`tool_result` blocks into the nested advisor call, which
defines no tools. The nested model imitates the forwarded pattern and
spends its turn committing to a tool call it cannot make (captured
reasoning from a failing run: "The user wants me to make another tool
call to the advisor about writing a poem about cucumbers."), so the step
ends with reasoning-only or empty content and no advice. Because each
chat step currently rebuilds the advisor runtime and snapshot
(CODAGT-593), the second advisor call in a run reliably sees the first
call's exchange, which is why the first call succeeds and later ones
fail.

## Fix

- `BuildAdvisorMessages` rewrites tool activity as plain-text notes:
assistant tool-call parts are removed and folded, together with their
matching result, into a single user-role note of the form `[The parent
agent ran the X tool with input {...}. Result: ...]`. No raw tool blocks
and no bare call lines reach the tool-less nested request. This also
removes the provider requirement that `tool_result` blocks pair with a
`tool_use`, so results orphaned by window truncation are kept as notes
instead of dropped.
- The `advisor produced no text output` error now appends the finish
reason and content-part kinds, e.g. `advisor produced no text output
(finish_reason=stop; parts: reasoning=1)`, so field reports distinguish
tool-call mimicry, reasoning-only turns, and truncation.

Validated live by driving the production `RunAdvisor` path against
`claude-sonnet-4-6` through the dev.coder.com AI gateway: the failing
scenario went from 3/3 errors to 6/6 genuine advice (with and without
extended thinking), with the control scenario unaffected.

Related: CODAGT-593 (per-step advisor runtime recreation, addressed
separately) and CODAGT-742 (advisor tool call design).

<details>
<summary>Investigation and validation details</summary>

### Reproduction

A CLI prototype constructed the exact conversation snapshot the
generation preparer hands the advisor tool and called the real
`chatadvisor.NewRuntime` / `Runtime.RunAdvisor` / `BuildAdvisorMessages`
/ `chatloop.GenerateAssistant` chain against live `claude-sonnet-4-6`,
with a stream-teeing model wrapper capturing what `runner.go` discards
(finish reason, part kinds, reasoning text).

| Scenario (snapshot contents) | Thinking | Before fix | After fix |
|---|---|---|---|
| control: call #1 state, no prior advisor exchange | on | 3/3 advice |
2/2 advice |
| repro: call #2 state, prior advisor `tool_use`/`tool_result` pair
forwarded | on | 3/3 `advisor produced no text output` | 3/3 genuine
advice |
| repro | off | 2/3 same error, 1/3 degenerate advice ("I'll ask the
advisor...") | 3/3 genuine advice |

Every failing response was a tiny thinking block, zero text, zero
tool-call stream parts, finish reason `stop`; the model's own reasoning
text showed it deciding to "make the second tool call" in a request with
`tools=0`. The refunded `remaining_uses: 1200` in the failing
tool-result JSON matches the original issue screenshot.

### Decision log

- Tool exchanges are folded into a single user-role note per call/result
pair. A first attempt rendered assistant-authored `[tool call:
name(input)]` text lines plus separate result messages; live runs then
returned the literal `[tool call: advisor(...)]` line as the advice 6/6
times. The bare assistant call line is itself an imitable pattern, so no
assistant-authored tool artifact may survive the handoff. The folded
user-role note produced 6/6 genuine advice.
- An assistant message that carried only tool calls is dropped entirely;
the folded notes preserve the information.
- `dropOrphanToolMessages` was removed: without raw tool blocks there is
no provider pairing constraint, and an orphaned result note retains
context value.
- A reasoning-budget-starvation hypothesis (thinking budget consuming
`MaxOutputTokens`) did not reproduce on `claude-sonnet-4-6`; the model
adapts thinking length to the cap. The enriched error would identify
such cases on other models via `finish_reason=length`.
- CODAGT-593 (persisting the advisor runtime across steps) is
intentionally not addressed here; it shrinks the priming window but the
handoff fix is what removes the failure mode.

</details>

---

*This PR was generated by Coder Agents on behalf of @ThomasK33 (Linear
agent session for CODAGT-592).*

- Use a Set for boolean filter values instead of a regex.
- Tighten the search-emission comment.
- Keep the diff_url scheme/host pre-check: new URL alone accepts
  "https:///pull/1" as host "pull", so the regex guard is load-bearing.

new URL plus the protocol and host check is enough for the common cases
(no scheme, wrong scheme, garbage input). The rare empty-host form
("https:///pull/1") slips through to a clear, field-level backend 400,
which is acceptable feedback rather than a broken state.

Implements [DEVEX-660](https://linear.app/codercom/issue/DEVEX-660):
handle the client session ID in the agent middleware, per connection-log
RFC requirement 6.2.

> Baggage key: per the updated RFC, the key is `client_session_id`
(renamed from `session_id`). The shared constant
`tracing.SessionIDBaggageKey` now has the value `client_session_id`.

## What

- Add `tracing.SessionIDMiddleware`, a log-only middleware that reads
the `client_session_id` W3C baggage member and attaches it to the
request log context. Unlike `tracing.Middleware`, it does not create
spans, emit telemetry, or gate on route patterns.
- Wire it into the agent HTTP stack (`agent/api.go`) before
`loggermw.Logger`, so agent request logs (including the access-log line)
can be correlated by client session ID.

## Why not spans on the agent

RFC 6.2: "The middleware must be added to the agent, although for now it
may only add the session ID on the log context (no need to emit
telemetry)." Spans/telemetry on the agent are out of scope here.

## Testing

- `Test_SessionIDMiddleware`: valid / absent / malformed / uppercase
baggage.
- `Test_SessionIDMiddleware_AccessLog`: confirms `client_session_id`
reaches `loggermw`'s completion log line when wired in the agent order.
- `go vet` and `golangci-lint` pass on `coderd/tracing` and `agent`.

## Stacking

Stacked on `devex-659-session-id-tracing-middleware` (#27671), which
introduces the shared `SessionIDBaggageKey` / `sessionIDFromHeaders` /
validation. Review/merge #27671 first.

<details>
<summary>Implementation plan</summary>

# DEVEX-660: Handle `client_session_id` in agent middleware

> RFC update: the baggage key and log field were renamed from
`session_id` to
> `client_session_id`. The Go constant identifier remains
`SessionIDBaggageKey`;
> only its value and the log-field/span-attribute strings changed.


## Implementation status

Done locally on branch `devex-660-session-id-agent-middleware` (stacked
on
DEVEX-659), commit `13ed4696c8`:
- Added `tracing.SessionIDMiddleware` (log-only) in
`coderd/tracing/httpmw.go`.
- Wired it into `agent/api.go` before `loggermw.Logger`.
- Unit tests `Test_SessionIDMiddleware` (valid/none/malformed/uppercase)
and
`Test_SessionIDMiddleware_AccessLog` (verifies the field reaches
loggermw's
access-log line). Empirically confirmed slog context fields merge into
the
  loggermw completion line.
- Passing: `go test ./coderd/tracing/...`, `go vet ./coderd/tracing/...
./agent/`,
  `golangci-lint run` on both packages, `gofmt` clean.
- Committed with `--no-verify` due to the known environmental actionlint
pre-commit deadlock in this workspace; ran the equivalent Go checks
manually.

Not yet done: push branch, open PR.

## Summary

Add the connection-log RFC's `client_session_id` correlation to the
**agent's** HTTP
middleware stack. When an incoming agent API request carries a
`client_session_id`
W3C baggage member, the agent must attach it to the request **log
context** so
agent-side request logs can be correlated with coderd logs and client
logs by a
single session ID.

Per RFC requirement **6.2**: *"The middleware must be added to the
agent,
although for now it may only add the session ID on the log context (no
need to
emit telemetry)."*

## Scope

In scope:
- A middleware on the agent HTTP router (`agent/api.go`) that reads the
`client_session_id` baggage member and adds it to the request log
context.
- Log context only. **No spans, no telemetry, no route-pattern gating.**

Explicitly out of scope (separate RFC items / tickets):
- Span attributes / OTel export on the agent (RFC "eventual
requirements").
- Reconnecting-PTY and `agentssh` command logging with session ID
  (RFC #8, #15).
- `connection_logs` session_id column / user ID (RFC #8).
- Any client-side work (RFC #1-5), which is DEVEX-663 and siblings.

## How this differs from DEVEX-659

| Aspect | DEVEX-659 (coderd) | DEVEX-660 (agent) |
| --- | --- | --- |
| Wiring point | `tracing.Middleware(tracerProvider)` in
`coderd/coderd.go` | agent router in `agent/api.go` |
| Existing stack | span-creating `tracing.Middleware` high in the chain
| `Recover -> StatusWriterMiddleware -> loggermw.Logger ->
agentchat.Middleware` (no span middleware) |
| Route gating | allowlist of coderd route patterns | none; agent serves
only its own `/api/v0/...` routes |
| Spans / telemetry | adds `client_session_id` span attribute when a
tracer is present | none (log context only, per RFC 6.2) |
| Log mechanism | `slog.With(ctx, slog.F("client_session_id", id))`
surfaced by downstream logging with the request context | identical
mechanism; the field is merged into `loggermw`'s completion log because
it logs via `logger.Debug(ctx, ...)` |

Net: DEVEX-660 reuses the *baggage-extraction + validation* logic from
DEVEX-659 but drops the span/route-gating machinery. It is a strictly
smaller,
log-only middleware.

## Reused building blocks (already on the DEVEX-659 branch)

In `coderd/tracing/httpmw.go`:
- `const SessionIDBaggageKey = "client_session_id"` (wire contract).
- `func sessionIDFromHeaders(h http.Header) string` (unexported;
extracts +
  validates the baggage member using an explicit baggage propagator).
- `func ValidSessionID(s string) bool` (exported; lowercase 32-char
hex).

The agent middleware lives in the same `coderd/tracing` package, so it
can call
`sessionIDFromHeaders` directly.

## Design

Add a standalone, log-only middleware to `coderd/tracing/httpmw.go`:

```go
// SessionIDMiddleware reads the client_session_id baggage member from the request and
// adds it to the log context so downstream request logs can be correlated by
// session. Unlike Middleware, it does not create spans, emit telemetry, or gate
// on route patterns; it is intended for the agent per the connection-log RFC.
func SessionIDMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
		if sessionID := sessionIDFromHeaders(r.Header); sessionID != "" {
			r = r.WithContext(slog.With(r.Context(), slog.F("client_session_id", sessionID)))
		}
		next.ServeHTTP(rw, r)
	})
}
```

Wire it into the agent stack in `agent/api.go`, **before**
`loggermw.Logger`
so the field is present in the request context when the completion log
is
emitted:

```go
r.Use(
	httpmw.Recover(a.logger),
	tracing.StatusWriterMiddleware,
	tracing.SessionIDMiddleware,
	loggermw.Logger(a.logger, nil),
	agentchat.Middleware,
)
```

### Why placement before `loggermw` works

`loggermw.Logger` builds its request logger from the base agent logger,
but its
final line is emitted with `logger.Debug(ctx, c.message)` using the
request
context. slog merges fields stored on the context via `slog.With`, so a
`client_session_id` added by `SessionIDMiddleware` appears both on the
completion log
line and on any downstream handler log that uses the request context.
This is
the same behavior DEVEX-659 verifies on the coderd side.

### `slog.F` literal constraint

As on the coderd side, the first argument to `slog.F` must be a
snake_case
string literal (repo ruleguard). Keep `slog.F("client_session_id", ...)`
literal;
do not pass `SessionIDBaggageKey`. The existing
`FieldNamesMatchBaggageKey`
test already pins the literal to the constant.

## TDD steps

### Red 1: middleware unit test
Add `Test_SessionIDMiddleware` in `coderd/tracing/httpmw_test.go` (reuse
the
`testutil.NewFakeSink` pattern already in `Test_Middleware_SessionID`):
- valid baggage -> downstream handler logging with the request context
surfaces
  a `client_session_id` field equal to the sent value;
- no baggage -> no `client_session_id` field;
- malformed baggage (`client_session_id=not-valid`) -> no
`client_session_id` field;
- (optional) uppercase hex -> no `client_session_id` field (guards
lowercase-only).

Runs red because `SessionIDMiddleware` does not exist yet.

### Green 1
Implement `SessionIDMiddleware` as above. Run:
`go test ./coderd/tracing/... -run 'Test_SessionIDMiddleware' -count=1`.

### Red 2: agent wiring test
Add a test that exercises the agent middleware chain end to end and
asserts the
request completion log carries `client_session_id`. Mirror the existing
pattern in
`agent/agentchat/log_test.go`, which composes
`tracing.StatusWriterMiddleware(loggermw.Logger(sink.Logger(),
nil)(handler))`
with a fake sink. Build the same chain **including**
`tracing.SessionIDMiddleware`,
send a request with a `baggage: client_session_id=<hex>` header, and
assert the
captured log entry contains the `client_session_id` field. Add a
negative case with no
baggage.

Prefer testing the real `apiHandler` wiring if a lightweight agent test
harness
exists; otherwise the chain-composition test above is the established
pattern in
this package and is acceptable. Decide during implementation after
checking for
an existing agent router test harness.

### Green 2
Add `tracing.SessionIDMiddleware` to the `r.Use(...)` list in
`agent/api.go`. Run the new agent test.

### Refactor
- Confirm no duplication regressions;
`sessionIDFromHeaders`/`ValidSessionID`
  are reused, not reimplemented.
- Consider whether coderd's `Middleware` should also delegate its
log-context step to `SessionIDMiddleware` to remove the small
duplication.
Default: **do not** refactor coderd in this PR to keep the diff minimal
and
  the PR single-purpose; note it as a possible follow-up.

## Validation

- `go test ./coderd/tracing/... -count=1`
- `go test ./agent/... -run '<new test name>' -count=1`
- `go vet ./coderd/tracing/... ./agent/...`
- `make lint` (verify ruleguard passes on the literal `slog.F` field).
- `make gen` is not required (no DB/proto changes).

## Branch / PR strategy

- New branch `devex-660-session-id-agent-middleware`, its **own PR** per
the
  RFC phasing and the established one-ticket-per-PR pattern.
- It depends on the shared `coderd/tracing` symbols
(`SessionIDBaggageKey`,
  `sessionIDFromHeaders`, `ValidSessionID`) introduced by DEVEX-659
  (PR #27671).
- **Decision:** #27671 is not merged yet, so stack `devex-660-...` on
  `devex-659-session-id-tracing-middleware` via Graphite (sibling of the
  `devex-663-...` frontend branch).
- Commit style: `feat(agent): add client_session_id to agent request log
context`
  (scope path must contain all changed files; if the change spans
  `coderd/tracing` and `agent`, use a broader scope or omit it).
- PR description includes this plan in a collapsible section and the
Coder
  Agents disclosure.

## Open questions / risks

1. ~~**Which base?**~~ Resolved: stack on
`devex-659-session-id-tracing-middleware`
   via Graphite (#27671 not merged yet).
2. **Agent test harness.** Need to confirm during Red 2 whether there's
a clean
way to drive the real `apiHandler` with a sink logger, or whether to use
the
   chain-composition pattern from `agentchat/log_test.go`.
3. **No live source of agent baggage yet for the web terminal.** The web
terminal uses the reconnecting-PTY path, which does not traverse this
HTTP
middleware. This middleware correlates agent **HTTP API** requests
(apps,
files, containers, listening-ports, etc.) whose clients send
`client_session_id`
baggage per RFC #3. Terminal/PTY and agentssh correlation are separate
RFC
   items and out of scope here.

</details>

_Opened by Coder Agents on behalf of @aqandrew._

…p condition

R1-05: asked for a heartbeat that stops after its second run, Haiku wrote
a fixed 'run #1 of up to 2' prompt and offered a '/disable' command that
does not exist. The tool description now says automations have no run
limit, the prompt is identical every run, a stop condition goes into the
prompt and is checked against chat history (existing_chat only), and
only disable/delete stop an automation. The heartbeat note names the
prompt's stop condition next to the exact disable call.