GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: persist agent-pushed workspace context snapshots in coderd by kylecarbs · Pull Request #26145 · coder/coder

Conversation

@kylecarbs

@kylecarbs kylecarbs commented Jun 8, 2026

Copy link
Copy Markdown
Member

Replaces the v2.10 PushContextState stub with a real coderd write path. Phase 1 of the chat-side persistence story; nothing reads these rows yet.

Follows #25983 and unblocks CODAGT-569.

What ships

Schema (000517_workspace_agent_context.{up,down}.sql)

Two new tables plus api_key_scope enum extensions:

  • workspace_agent_context_snapshots (PK workspace_agent_id to workspace_agents(id) ON DELETE CASCADE): one row per agent, overwritten per push. Holds version, schema_version, aggregate_hash, snapshot_error, received_at.
  • workspace_agent_context_resources (PK (workspace_agent_id, source)): per-resource state. body_kind and status are TEXT + CHECK so adding new wire kinds (the RFC's reserved PLUGIN/HOOK/SUBAGENT/COMMAND) is a one-line CHECK update plus a Go switch case.

SQLC queries (coderd/database/queries/workspaceagentcontext.sql)

  • UpsertWorkspaceAgentContextSnapshot
  • UpsertWorkspaceAgentContextResource
  • DeleteStaleWorkspaceAgentContextResources (delete-where-source-not-in)
  • GetLatestWorkspaceAgentContextSnapshot
  • ListWorkspaceAgentContextResources

Handler (coderd/agentapi/context.go)

ContextAPI is a new sub-API. PushContextState:

  1. Rejects schema_version > 1 with a non-Unimplemented error so a forward-incompatible agent fails loudly during rollout instead of slipping into the permanent fallback path the Unimplemented translation reserves for old coderd deployments.
  2. Validates resources: no empty/duplicate sources, every variant maps to a known body kind, every status maps to a known enum value, the Body oneof is set (even when status is non-OK, mirroring the wire guarantee so coderd can attribute failures to a known kind).
  3. Inside Database.InTx, reads the existing snapshot. If the push is not initial and version is not strictly greater, returns accepted = false and leaves stored state untouched. Otherwise upserts the snapshot row, upserts each resource, then runs the stale-source prune so the snapshot and resource rows always agree.
  4. Returns accepted = true on success.

Resource bodies are stored as protojson(body oneof variant) in body JSONB with body_kind as the discriminator. Adding a new field to an existing variant is zero work since protojson tolerates new fields; adding a new variant is a CHECK + switch case.

RBAC + dbauthz

  • New ResourceWorkspaceAgentContext (Create/Read/Update/Delete).
  • New SubjectTypeAgentContext plus subjectAgentContext system role and dbauthz.AsAgentContext helper. The push handler elevates to this subject; the agent's own role does not get direct write access to the table.
  • New workspace_agent_context:* API key scopes registered in the enum migration; internal-only (not added to externalLowLevel).

Audit

These rows are agent-pushed state, not user-authored. They are intentionally not added to AuditActionMap and not enumerated in enterprise/audit/table.go, matching boundary_logs, workspace_agent_memory_resource_monitor, etc. enterprise/audit tests pass unchanged.

Tests

  • coderd/agentapi/context_test.go: 12 subtests covering accepts/rejects (schema version, empty/duplicate source, unknown status, missing body), version semantics (stale dropped, same-version replay dropped, initial=true overwrites lower version), variant coverage, non-OK status persistence, and the empty-active-set prune case.
  • coderd/database/dbauthz/dbauthz_test.go: 5 MethodTestSuite cases covering the new queries.
  • coderd/rbac/roles_test.go: WorkspaceAgentContext permission row asserting no human role currently has access.
  • coderd/database/migrations/testdata/fixtures/000517_workspace_agent_context.up.sql: one snapshot + one resource per known body kind plus a non-OK status, so the migration test suite never lands with these tables empty.

Out of scope (later phases)

  • Chat hydration (chats.context_aggregate_hash, last_injected_context).
  • Dirty-bit fan-out and PUT /chats/{id}/context.
  • Agent-side POST /api/v0/context/resync barrier and the coder exp chat context CLI.
  • codersdk chat-context wire types and the dashboard Sources drawer.
  • Removal of the chatd per-turn pull fallback.

Compat property

This is a pure write path. If anything here returns errors the agent's RunPush loop backs off, no chat behavior changes, and the workspace keeps behaving exactly like it did before v2.10.

Implementation plan and decision log

Key design calls:

  1. Concurrency: Accept iff req.Initial || req.Version > existing.Version. The strict RFC reading ("version comparison is authoritative") locks restarted agents out because their per-process counter resets to 1; honoring initial=true reflects the real reboot reality while still rejecting steady-state replays/out-of-order pushes.
  2. Body encoding: protojson over the oneof variant body proto, stored in JSONB with body_kind discriminator. Structured at the API/Go layer, schema-tolerant at the storage layer, and Phase 2 readers round-trip back via protojson.Unmarshal.
  3. Schema version rejection: returns a normal error, not Unimplemented. The agent's RunPush loop only short-circuits on Unimplemented; that escape hatch is reserved for old coderd deployments. A forward-incompatible agent should retry-and-back-off, not flip the connection into permanent fallback.
  4. Validation strictness: empty sources, duplicate sources, STATUS_UNSPECIFIED, and missing Body oneof variants are rejected before any write so a misbehaving agent cannot poison the snapshot table. Phase 2 readers can trust every row maps to a known proto variant.

This PR was authored by Coder Agents on Kyle Carberry's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@kylecarbs
kylecarbs requested a review from Emyrk as a code owner June 8, 2026 20:31
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Docs preview

📖 View docs preview for docs/reference/api/members.md

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot assigned kylecarbs Jun 8, 2026
@kylecarbs kylecarbs changed the title feat(coderd/agentapi): persist agent-pushed workspace context snapshots feat: persist agent-pushed workspace context snapshots in coderd Jun 8, 2026
@kylecarbs
kylecarbs force-pushed the feat/agentapi-context-push-persistence branch from 2b23687 to 540dacf Compare June 8, 2026 20:33
@kylecarbs
kylecarbs requested review from johnstcn and spikecurtis as code owners June 9, 2026 01:48
@kylecarbs
kylecarbs removed request for Emyrk and spikecurtis June 9, 2026 01:57
DanielleMaywood
DanielleMaywood approved these changes Jun 10, 2026
View reviewed changes

@DanielleMaywood DanielleMaywood left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer better dbauthz usage here but that can be iterated on

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread coderd/agentapi/context.go Outdated
Comment on lines +60 to +61
//nolint:gocritic // We need permissions to write to the DB here and we are in the context of the agent.
ctx = dbauthz.AsProvisionerd(ctx)

@DanielleMaywood DanielleMaywood Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Create a proper dbauthz boundary for this

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@spikecurtis spikecurtis Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to clean this up before merging. History tells me that if we don't get this right before main we won't come back around.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kylecarbs reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@kylecarbs kylecarbs Jun 10, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I'll pause before merging.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ae5b7ea69b. Dropped AsProvisionerd/ResourceSystem entirely. The handler relies on the agent token subject already in the RPC context, and each query authorizes against the workspace that owns the agent in dbauthz (fast path via the cached workspace RBAC object, slow path via GetWorkspaceByAgentID), same as UpdateWorkspaceAgentMetadata. Added TestWorkspaceAgentPushContextState, which pushes through the real agent RPC path (agent token middleware, DRPC v2.10, dbauthz, Postgres) to pin the boundary.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread coderd/database/dbauthz/dbauthz.go Outdated
}

func (q *querier) DeleteStaleWorkspaceAgentContextResources(ctx context.Context, arg database.DeleteStaleWorkspaceAgentContextResourcesParams) error {
if err := q.authorizeContext(ctx, policy.ActionDelete, rbac.ResourceSystem); err != nil {

@DanielleMaywood DanielleMaywood Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Proper resource tags for this, not ResourceSystem

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kylecarbs reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in ae5b7ea69b alongside the boundary change: these wrappers no longer use ResourceSystem. They now fetch the workspace by agent ID and authorize the action against that workspace object (with a fast path through the cached workspace RBAC object from the agent connection).

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
spikecurtis
spikecurtis reviewed Jun 10, 2026
View reviewed changes
Comment thread coderd/database/migrations/000517_workspace_agent_context.up.sql Outdated
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (workspace_agent_id, source),
CONSTRAINT workspace_agent_context_resources_body_kind_chk
CHECK (body_kind IN (

@spikecurtis spikecurtis Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For most other enumerated types in the database we use a SQL type, which then generates a Go type via sqlc --- why aren't we doing that here?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No good reason; fixed in ae5b7ea69b. body_kind and status are now proper SQL enum types (workspace_agent_context_body_kind, workspace_agent_context_resource_status) and the handler uses the sqlc-generated constants. The reserved plugin RFC kinds are pre-seeded in the enum, so future variants are still just an ALTER TYPE ... ADD VALUE plus a Go switch case.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread coderd/database/migrations/000522_workspace_agent_context.up.sql
-- deletes any sources absent from the latest push in the same
-- transaction.
CREATE TABLE workspace_agent_context_resources (
workspace_agent_id UUID NOT NULL REFERENCES workspace_agents(id) ON DELETE CASCADE,

@spikecurtis spikecurtis Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only soft-delete agents, so these will accumulate in the database forever. I think we should at least update SoftDeletePriorWorkspaceAgents and SoftDeleteWorkspaceAgentsByWorkspaceID to clean this up, or define a dbpurge job to do so.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ae5b7ea69b. Updated both SoftDeletePriorWorkspaceAgents and SoftDeleteWorkspaceAgentsByWorkspaceID to hard-delete the context rows for the agents being soft-deleted (CTE in the same statement), with a querier test (TestSoftDeleteWorkspaceAgentsPurgesContext). Went with this over a dbpurge job since the rows only describe live agents and agents are never un-deleted.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 14, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the spirit of this, I found a third soft-delete path that would have leaked: DeleteWorkspaceSubAgentByID (the DeleteSubAgent RPC), which removes a devcontainer sub-agent mid-build without a full workspace rebuild. Because the rebuild-time queries both filter deleted = FALSE (so they skip an already-deleted sub-agent), the FK ON DELETE CASCADE only fires on hard delete, and we never hard-delete agents, that sub-agent's context rows would have orphaned forever.

Fixed in de2c1be by applying the same purge CTE to DeleteWorkspaceSubAgentByID, and extended TestSoftDeleteWorkspaceAgentsPurgesContext to cover the sub-agent case.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread coderd/agentapi/context.go Outdated

accepted = true
return nil
}, &database.TxOptions{TxIdentifier: "push_agent_context_state"})

@spikecurtis spikecurtis Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need repeatable read isolation here to avoid situations where two different updates are sent simultaneously and klobber each other.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ae5b7ea69b. Switched to database.ReadModifyUpdate, which runs the transaction at repeatable read and retries serialization failures (40001). The loser of a conflict re-runs the version gate against the winner's committed state, so stale pushes get dropped rather than clobbering. Added a unit test (SerializationConflictRetries) covering the retry path.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread coderd/agentapi/context.go Outdated
// resourceRow is the validated, ready-to-persist form of a single
// incoming ContextResource. It collapses the proto oneof to (kind,
// JSONB) so the DB upsert is uniform across kinds.
type resourceRow struct {

@spikecurtis spikecurtis Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not reuse database.UpsertWorkspaceAgentContextResourceParams here instead of defining a new type?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reason, fixed in ae5b7ea69b. resourceRow is gone; validation now produces database.UpsertWorkspaceAgentContextResourceParams directly and the handler fills WorkspaceAgentID/Now at upsert time.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread coderd/agentapi/context.go
}
seen[r.Source] = struct{}{}

kind, body, err := marshalContextResourceBody(r)

@spikecurtis spikecurtis Jun 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to validate the resource sizes and numbers server-side, otherwise a compromised Workspace can DoS Coderd.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kylecarbs kylecarbs Jun 11, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ae5b7ea69b. Added server-side caps, validated before the transaction starts: at most 1000 resources per push, 256KiB per marshaled body, 4MiB aggregate body size (aligned with the DRPC message cap), 1KiB source/source_path (source is a btree PK column, which Postgres limits to ~2704 bytes per index entry), 4KiB error strings, 64B hashes, and explicit uint64 to int64 overflow checks on version/size_bytes. Each cap has a unit test.

Posted by Coder Agents on Kyle's behalf.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@DanielleMaywood

DanielleMaywood commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

QQ on scope: Should we move the removal of SchemaVersion into its own change? It sort of feels unrelated to the rest of the PR

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kylecarbs
kylecarbs force-pushed the feat/agentapi-context-push-persistence branch from 03358ed to abc1985 Compare June 12, 2026 19:48
@kylecarbs
kylecarbs requested a review from spikecurtis June 15, 2026 15:28
@kylecarbs
feat: persist agent-pushed workspace context snapshots in coderd
056b3ff
@kylecarbs
kylecarbs force-pushed the feat/agentapi-context-push-persistence branch from de2c1be to 056b3ff Compare June 15, 2026 15:59
@kylecarbs
kylecarbs merged commit b439b06 into main Jun 15, 2026
28 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@kylecarbs
kylecarbs deleted the feat/agentapi-context-push-persistence branch June 15, 2026 16:38
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@DanielleMaywood DanielleMaywood DanielleMaywood approved these changes
@johnstcn johnstcn Awaiting requested review from johnstcn
@spikecurtis spikecurtis Awaiting requested review from spikecurtis spikecurtis is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@kylecarbs kylecarbs

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@kylecarbs @DanielleMaywood @spikecurtis
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.