GeezerVison is showing github.com — chore: report Agent Relay builds in telemetry by Emyrk · Pull Request #31407 · coder/coder
Conversation
Emyrk
commented
Oct 28, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Telemetry builds now report whether Agent Relay created them. Before this, relay workspaces looked like any other build.
A start build counts as a relay build when it has the agent_relay_session_id parameter. Stop and delete builds never get the field. For those builds, the new agent_relay field on WorkspaceBuild reports:
provider: claude_code, cursor, ambiguous, or unknownclient_platform: Claude Code only. Lowercased, characters outside [a-z0-9_] become _, and cut to 15 charactersattempt: omitted when missing or not an integer, so 0 is sentsession_id_hash and pool_hash: SHA-256 hashes
No other parameter values are sent. Other builds leave the field out.
The snapshot fetches these parameters with one new batch query. The workspace-create event reads the stored parameters with one extra query, only when telemetry is enabled, so it reports the same data as the snapshot. docs/admin/setup/telemetry.md lists the parameters that are read.
The collector side is coder/coder-telemetry-server#56. Merge this PR first.
Refs RELAY-9
Decision log
- Detect relay builds from parameters. This needs no relay changes. Relay registration or CORE-426 provenance metadata can come later.
- Builds only, not workspaces.
GetWorkspaces doesn't return the latest build's parameters, so a workspace field would cost another query. The collector can join on workspace_id.
- Provider comes from parameter names only.
agent_relay_claude_code_* means Claude Code and agent_relay_cursor_* means Cursor. A build with both reports ambiguous; a build with neither reports unknown. coderd stores every declared parameter on every build (ephemeral ones with their default), so a Claude Code start after a stop still reports claude_code.
- Start builds only. Stop and delete builds of a relay workspace carry the same parameters but are not dispatches, so the snapshot skips them before querying parameters.
- Hash session ID and pool. Pool names are user content, and sessions only need counting.
- Known gaps. Cursor has no client platform. Relay version and health would need relay registration.
- Parameter names are shared with coder/agent-relay. If agent-relay renames one, telemetry silently stops reading it.
Manual test on develop.sh
Ran on 8ad7cd491f with CODER_TELEMETRY_ENABLE=true CODER_TELEMETRY_URL=http://127.0.0.1:8081 and go run ./scripts/telemetry-server --port 8081 as the collector.
Pushed three templates with no infrastructure, each declaring the same parameters as the real modules:
relay-claude: agent_relay_session_id, _delivery_id, _pool, _client_platform, plus ephemeral agent_relay_attempt and agent_relay_claude_code_lock_to_accountrelay-cursor: agent_relay_session_id, _delivery_id, _pool, agent_relay_cursor_pool_name, _idle_release_timeout, _repo_urlrelay-plain: region
Created workspaces with POST /api/v2/users/me/workspaces and rich_parameter_values, the same call the relay makes. Then ran stop and start builds on the relay workspaces, and restarted coderd to get a periodic snapshot.
| Workspace |
Build 1 (create event and snapshot) |
Builds 2 and 3 (stop, start) |
Claude (claude_in_slack, attempt 1) |
claude_code, platform, attempt 1 |
claude_code, platform, no attempt |
| Cursor |
cursor |
cursor |
| Plain |
field omitted |
n/a |
| Claude template, no session ID |
field omitted |
n/a |
- Stop and start builds keep
agent_relay_claude_code_lock_to_account, with an empty value, so Claude Code rebuilds still report claude_code.
- No session ID, pool name, or
region value appears in the captured telemetry. The hashes match sha256 of the inputs.
- Fed the captured snapshot through the conversion code in coder/coder-telemetry-server#56: all 8 builds converted, 6 of them with
agent_relay set.
- No telemetry errors or panics in the coderd logs.
This run predates the start-only change: build 2 (stop) carried the field then. Stop and delete builds are now omitted, which TestTelemetry_WorkspaceBuildAgentRelay covers.
Generated by Coder Agents on behalf of @Emyrk.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Tag telemetry workspace builds dispatched by Agent Relay using the
agent_relay_* rich parameters it stamps on each build. Session ID and
pool are hashed; no raw parameter values are sent.
Refs RELAY-9
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Docs preview
Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Detect the provider only from provider-specific parameter prefixes. Drop
the client platform fallback.
coder-agents-review[bot] reacted with rocket emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Chat: Review posted | View chat
Requested: 2026-11-03 15:43 UTC by @Emyrk
Spend: $8.96 / $100.00
Review history
- R1 (2026-11-03): 13 reviewers, 2 Nit, 1 P2, 5 P3, 3 P4, COMMENT. Review
deep-review v0.13.0 | Round 1 | 161b723..8ad7cd4
Last posted: Round 1, 11 findings (1 P2, 5 P3, 3 P4, 2 Nit), COMMENT. Review
Finding inventory
Findings
| # |
Sev |
Status |
Location |
Summary |
Round |
Reviewer |
Posted |
| CRF-1 |
P3 |
Dropped by orchestrator (merged into CRF-4; CRF-4 fix asks for the test if the path is kept) |
workspaces.go:891 |
createWorkspace telemetry AgentRelay path has no test |
R1 |
Netero |
No |
| CRF-2 |
P4 |
Open |
workspacebuildparameters.sql:27 |
No test checks the name-prefix filter of the new query |
R1 |
Netero |
Yes |
| CRF-3 |
Note |
Dropped by orchestrator (merged into CRF-4) |
workspaces.go:891 |
Create event and snapshot can report different agent_relay for one build |
R1 |
Netero |
No |
| CRF-4 |
P3 |
Open |
workspaces.go:891 |
Create event builds agent_relay from unvalidated request names; forgeable, untested, and differs from snapshot |
R1 |
Kurapika, Luffy, Netero, Gon |
Yes |
| CRF-5 |
P2 |
Open |
agentrelay.go:109 |
agent_relay marks every build of a relay workspace; stop builds report claude_code, not unknown |
R1 |
Pariston, Mafuuu, Melody, Zoro, Leorio, Gon |
Yes |
| CRF-6 |
P3 |
Open |
agentrelay.go:57 |
Attempt int omitempty drops attempt 0 and conflates it with missing or invalid |
R1 |
Melody, Leorio |
Yes |
| CRF-7 |
P3 |
Open |
agentrelay.go:101 |
Unsalted SHA-256 of guessable pool names does not hide them |
R1 |
Mafuuu P3, Kurapika Note, Ryosuke Note, Leorio Note |
Yes |
| CRF-8 |
P4 |
Open |
agentrelay.go:97 |
client_platform is sent as raw user-controllable text, contradicting the PR description |
R1 |
Kurapika, Ryosuke, Pen Botter, Luffy Note, Mafuuu Note |
Yes |
| CRF-9 |
P3 |
Open |
telemetry.md:21 |
Docs still say parameters are not collected, contradicting the new paragraph |
R1 |
Leorio P3, Pen Botter P4 |
Yes |
| CRF-10 |
P3 |
Open |
agentrelay.go:92 |
No test covers an empty stored agent_relay_session_id |
R1 |
Bisky |
Yes |
| CRF-11 |
P4 |
Open |
agentrelay.go:71 |
Prefix query loads agent_relay_credential JWTs into the reporter |
R1 |
Pariston |
Yes |
| CRF-12 |
Nit |
Open |
agentrelay.go:60 |
Several agentrelay.go doc comments restate the signature or code |
R1 |
Gon |
Yes |
| CRF-13 |
Nit |
Open |
agentrelay_test.go:100 |
Test case UnknownProvider tests a non-numeric attempt; name does not say so |
R1 |
Gon |
Yes |
| CRF-14 |
P3 |
Dropped by orchestrator (merged into CRF-5; premise that stop builds lose the ephemeral parameter is false) |
agentrelay.go:112 |
Count client_platform as a Claude Code signal |
R1 |
Zoro |
No |
| CRF-15 |
P3 |
Dropped by orchestrator (merged into CRF-5) |
agentrelay.go:110 |
Claude Code share depends on module version |
R1 |
Melody |
No |
| CRF-16 |
P3 |
Dropped by orchestrator (merged into CRF-5) |
agentrelay.go:49 |
ClientPlatform comment and test comment disagree on provider inference |
R1 |
Leorio |
No |
| CRF-17 |
P3 |
Dropped by orchestrator (merged into CRF-5) |
agentrelay.go:46 |
Exported Provider doc does not explain unknown |
R1 |
Leorio, Gon (:110) |
No |
| CRF-18 |
P4 |
Dropped by orchestrator (merged into CRF-6) |
agentrelay.go:56 |
Attempt comment omits the non-integer case |
R1 |
Leorio |
No |
| CRF-19 |
Nit |
Dropped by orchestrator (merged into CRF-5) |
telemetry.md:26 |
Docs say "creates", code says "dispatches" |
R1 |
Gon |
No |
| CRF-20 |
P2 |
Dropped by orchestrator (merged into CRF-4) |
workspaces.go:888 |
"stand in for" comment claims request values equal stored values |
R1 |
Gon |
No |
| CRF-21 |
P2 |
Dropped by orchestrator (agent-relay's own contract comment uses the same phrasing for the name set; provider split is visible from the prefix constants) |
agentrelay.go:16 |
Constant-block comment says "every build" |
R1 |
Gon |
No |
| CRF-22 |
P2 |
Dropped by orchestrator (contradicts Leorio's request to expand it; CRF-5 fix rewrites provider docs) |
agentrelay.go:46 |
Provider field comment lists the constants |
R1 |
Gon |
No |
| CRF-23 |
P2 |
Dropped by orchestrator (collector author is the same team; low value) |
agentrelay.go:51 |
SessionIDHash comment omits hex encoding |
R1 |
Gon |
No |
| CRF-24 |
P2 |
Dropped by orchestrator (one-line accurate comment; two-word overlap; requires make gen churn) |
workspacebuildparameters.sql:19 |
starts_with comment repeats itself |
R1 |
Gon |
No |
| CRF-25 |
Nit |
Dropped by orchestrator ("Merge this PR first" follows the collector link; reading is unambiguous enough) |
PR description |
Merge-order sentence ambiguous |
R1 |
Leorio |
No |
| CRF-26 |
Note |
Dropped by orchestrator (premise false: wsbuilder.go:683 is secrets, ephemeral parameters are stored with defaults) |
agentrelay.go:130 |
Claude Code builds undercounted as unknown |
R1 |
Pen Botter |
No |
| CRF-27 |
Note |
Dropped by orchestrator (Melody verified names against agent-relay f02bbf2) |
agentrelay.go:16 |
Parameter-name contract unverified |
R1 |
Pen Botter |
No |
| CRF-28 |
Note |
Dropped by orchestrator (informational, no action) |
agentrelay.go:45 |
Collector side not reachable for verification |
R1 |
Melody |
No |
| CRF-29 |
Note |
Dropped by orchestrator (informational, no action) |
workspacebuildparameters.sql:27 |
Query plan uses unique index; verified 47 ms at 25k IDs |
R1 |
Knuckle |
No |
| CRF-30 |
Note |
Dropped by orchestrator (informational, no action) |
dbauthz.go:6118 |
dbauthz scoping correct |
R1 |
Kurapika |
No |
Contested and acknowledged
Round log
Round 1
Round type: round 1 (full panel). Netero first pass: 1 P3, 1 P4, 1 Note; no P0, panel proceeded. Law not run (effective additions 340).
Panel: ging-go, knuckle, kurapika, ryosuke, melody, luffy, pen-botter (domain specialists), pariston, mafuuu (core), bisky, gon, leorio (pool), zoro (wildcard). 13 reviewers: the 7 triggered specialists plus core and the required Bisky and the Gon/Leorio floor reached 12; the wildcard made 13.
Cross-check: orchestrator verified CRF-5's premise in codersdk/richparameters.go ValidateResolve (ephemeral falls back to default), dynamicparameters/resolver.go (default saved), workspaces_test.go TestWorkspaceWithEphemeralRichParameters, and registry module agent-relay-claude-code main.tf (lock_to_account and attempt are ephemeral with default ""). Telemetry ProvisionerJob has no transition field. Result: 1 P2, 5 P3, 3 P4, 2 Nit posted. Remaining reviewer items merged or dropped with reasons in the ledger. Reviewed against 161b723..8ad7cd4.
About deep-review
CRF = Coder Review Finding (P0-P4, Nit, Note)
| Reviewer |
Focus |
| Bisky |
tests |
| Chopper |
ops/errors |
| Churn-guard |
change verification |
| Ging |
language modernization |
| Gon |
naming |
| Hisoka |
edge cases |
| Killua |
perf |
| Kite |
change integrity |
| Knov |
contracts |
| Knuckle |
SQL |
| Komugi |
flake/determinism |
| Kurapika |
security |
| Law |
decomposition |
| Leorio |
docs |
| Luffy |
product |
| Mafu-san |
process |
| Mafuuu |
contracts |
| Melody |
dispatch/pairing |
| Meruem |
structural |
| Nami |
frontend |
| Netero |
mechanical checks |
| Pariston |
premise testing |
| Pen-botter |
product gaps |
| Razor |
verification |
| Robin |
duplication |
| Ryosuke |
Go arch |
| Takumi |
concurrency |
| Zoro |
shape |
🤖 Managed by Coder Agents.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR adds an agent_relay field to telemetry workspace builds, derived from the build's agent_relay_* parameters. Findings: 1 P2, 5 P3, 3 P4, 2 Nit.
docs/admin/setup/telemetry.md:21
P3 [CRF-9] Line 21 still says "we do not collect parameters", and the new paragraph at line 26 reports values from five parameters, one of them as raw text. (Leorio, Pen Botter)
An administrator who reads line 21 to decide whether to keep telemetry enabled gets a rule that the following paragraph breaks. Add the exception to line 21, for example "we do not collect parameter values, except the Agent Relay fields described below".
🤖
🤖 This review was automatically generated with Coder Agents.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Normalize client_platform, send attempt 0, load only the parameters the
converter reads, and clarify the docs and comments.
|
CRF-9: fixed in ac0ac07. The docs sentence now reads "we do not collect environment variables, user email addresses, or parameter values, except the Agent Relay fields described below."
🤖 Generated by Coder Agents on behalf of @Emyrk.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
matifali
changed the title
feat: report Agent Relay builds in telemetry
chore: report Agent Relay builds in telemetry
Nov 5, 2026
Reviewers
BobbyHo
BobbyHo approved these changes
+1 more reviewer
Reviewers whose approvals may not affect merge requirements
Uh oh!
There was an error while loading. Please reload this page.
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|
BobbyHo
left a comment