GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — feat: persist agent-pushed workspace context snapshots in coderd by kylecarbs · Pull Request #26145 · coder/coder Conversation
Member
Replaces the v2.10 Follows #25983 and unblocks CODAGT-569. What shipsSchema (
|
Docs preview📖 View docs preview for |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
2b23687 to
540dacf
Compare
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer better dbauthz usage here but that can be iterated on
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| //nolint:gocritic // We need permissions to write to the DB here and we are in the context of the agent. | ||
| ctx = dbauthz.AsProvisionerd(ctx) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
TODO: Create a proper dbauthz boundary for this
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
We need to clean this up before merging. History tells me that if we don't get this right before main we won't come back around.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Yup, I'll pause before merging.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Done in ae5b7ea69b. Dropped AsProvisionerd/ResourceSystem entirely. The handler relies on the agent token subject already in the RPC context, and each query authorizes against the workspace that owns the agent in dbauthz (fast path via the cached workspace RBAC object, slow path via GetWorkspaceByAgentID), same as UpdateWorkspaceAgentMetadata. Added TestWorkspaceAgentPushContextState, which pushes through the real agent RPC path (agent token middleware, DRPC v2.10, dbauthz, Postgres) to pin the boundary.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| } | ||
|
|
||
| func (q *querier) DeleteStaleWorkspaceAgentContextResources(ctx context.Context, arg database.DeleteStaleWorkspaceAgentContextResourcesParams) error { | ||
| if err := q.authorizeContext(ctx, policy.ActionDelete, rbac.ResourceSystem); err != nil { |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
TODO: Proper resource tags for this, not ResourceSystem
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Resolved in ae5b7ea69b alongside the boundary change: these wrappers no longer use ResourceSystem. They now fetch the workspace by agent ID and authorize the action against that workspace object (with a fast path through the cached workspace RBAC object from the agent connection).
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), | ||
| PRIMARY KEY (workspace_agent_id, source), | ||
| CONSTRAINT workspace_agent_context_resources_body_kind_chk | ||
| CHECK (body_kind IN ( |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
For most other enumerated types in the database we use a SQL type, which then generates a Go type via sqlc --- why aren't we doing that here?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
No good reason; fixed in ae5b7ea69b. body_kind and status are now proper SQL enum types (workspace_agent_context_body_kind, workspace_agent_context_resource_status) and the handler uses the sqlc-generated constants. The reserved plugin RFC kinds are pre-seeded in the enum, so future variants are still just an ALTER TYPE ... ADD VALUE plus a Go switch case.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| -- deletes any sources absent from the latest push in the same | ||
| -- transaction. | ||
| CREATE TABLE workspace_agent_context_resources ( | ||
| workspace_agent_id UUID NOT NULL REFERENCES workspace_agents(id) ON DELETE CASCADE, |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
We only soft-delete agents, so these will accumulate in the database forever. I think we should at least update SoftDeletePriorWorkspaceAgents and SoftDeleteWorkspaceAgentsByWorkspaceID to clean this up, or define a dbpurge job to do so.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Done in ae5b7ea69b. Updated both SoftDeletePriorWorkspaceAgents and SoftDeleteWorkspaceAgentsByWorkspaceID to hard-delete the context rows for the agents being soft-deleted (CTE in the same statement), with a querier test (TestSoftDeleteWorkspaceAgentsPurgesContext). Went with this over a dbpurge job since the rows only describe live agents and agents are never un-deleted.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Following the spirit of this, I found a third soft-delete path that would have leaked: DeleteWorkspaceSubAgentByID (the DeleteSubAgent RPC), which removes a devcontainer sub-agent mid-build without a full workspace rebuild. Because the rebuild-time queries both filter deleted = FALSE (so they skip an already-deleted sub-agent), the FK ON DELETE CASCADE only fires on hard delete, and we never hard-delete agents, that sub-agent's context rows would have orphaned forever.
Fixed in de2c1be by applying the same purge CTE to DeleteWorkspaceSubAgentByID, and extended TestSoftDeleteWorkspaceAgentsPurgesContext to cover the sub-agent case.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| accepted = true | ||
| return nil | ||
| }, &database.TxOptions{TxIdentifier: "push_agent_context_state"}) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I think we need repeatable read isolation here to avoid situations where two different updates are sent simultaneously and klobber each other.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Done in ae5b7ea69b. Switched to database.ReadModifyUpdate, which runs the transaction at repeatable read and retries serialization failures (40001). The loser of a conflict re-runs the version gate against the winner's committed state, so stale pushes get dropped rather than clobbering. Added a unit test (SerializationConflictRetries) covering the retry path.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| // resourceRow is the validated, ready-to-persist form of a single | ||
| // incoming ContextResource. It collapses the proto oneof to (kind, | ||
| // JSONB) so the DB upsert is uniform across kinds. | ||
| type resourceRow struct { |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
why not reuse database.UpsertWorkspaceAgentContextResourceParams here instead of defining a new type?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
No reason, fixed in ae5b7ea69b. resourceRow is gone; validation now produces database.UpsertWorkspaceAgentContextResourceParams directly and the handler fills WorkspaceAgentID/Now at upsert time.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| } | ||
| seen[r.Source] = struct{}{} | ||
|
|
||
| kind, body, err := marshalContextResourceBody(r) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
We need to validate the resource sizes and numbers server-side, otherwise a compromised Workspace can DoS Coderd.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Done in ae5b7ea69b. Added server-side caps, validated before the transaction starts: at most 1000 resources per push, 256KiB per marshaled body, 4MiB aggregate body size (aligned with the DRPC message cap), 1KiB source/source_path (source is a btree PK column, which Postgres limits to ~2704 bytes per index entry), 4KiB error strings, 64B hashes, and explicit uint64 to int64 overflow checks on version/size_bytes. Each cap has a unit test.
Posted by Coder Agents on Kyle's behalf.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
QQ on scope: Should we move the removal of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
03358ed to
abc1985
Compare
de2c1be to
056b3ff
Compare
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.