GeezerVison is showing github.com — feat: honor disable-model-invocation in skill frontmatter by ibetitsmike · Pull Request #31147 · coder/coder
Conversation
ibetitsmike
commented
Oct 21, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Extends #30882's model_invocable semantics to workspace skills and to stored skills created through the API.
Claude Code and Xum let skill authors set disable-model-invocation: true in SKILL.md frontmatter, so agents never pick the skill on their own and it runs only when a person invokes it. Coder Agents honored the key only when someone imported a SKILL.md file while adding a project skill (#30884).
What changes
- Workspace skills: when the pinned SKILL.md sets the key to the YAML boolean
true, chatd leaves the skill out of <available-skills> before merging sources, so a listed skill with the same name keeps its bare name, and read_skill reports it as not found. The skill stays in the / menu, and invoking it still loads its body. read_skill_file serves its supporting files under the bare name or workspace/<name> and refuses its SKILL.md (case-insensitive, after path.Clean). Skills without the key behave as on the base.
- Invoked workspace skills: invoking a workspace skill now stores the pinned skill directory on the invocation part, and the prompt renders it as
dir on <invoked-skill>, escaped as an attribute value. Before, only read_skill gave the model a skill's directory, and it refuses a hidden skill, so a hidden skill could not run the scripts it bundles. The part also stores the chat's workspace, and the prompt shows dir only while the chat stays bound to it, as it does for workspace file references. This changes feat: send skill invocations as structured message parts #30861's invocation code. Hook payloads drop the directory and its workspace like the body, and API responses strip both.
- Stored skills: a create request whose content sets the key and that omits
model_invocable stores model_invocable = false. An explicit value wins, and PATCH is unchanged. This covers personal, organization, and project skills, which share createSkill.
workspacesdk.SkillDisablesModelInvocation reads the key. The CreateSkillRequest.ModelInvocable doc comment, the generated API docs, and extending-agents.md (the SKILL.md format, a new "Turn off automatic use" section, and the notes on the skill index, the skill tools, and aliases) describe the new behavior.
Follow-ups
Testing
- Go tests cover the parser, the turn's skill index and aliases,
read_skill, read_skill_file, invocation of a hidden skill and the directory it carries (attribute escaping, no directory after the chat changes workspace, no directory in hook payloads), and the create default for personal and organization skills. Each new test fails when its code path is removed.
- Remote dogfood UAT with a real model failed rounds 1 to 3. Rounds 1 and 2 found the hidden SKILL.md still readable through
read_skill_file by symlinks and stale pins, plus a bare-name lookup bug and a docs gap that round 2 fixed. Round 3 found that the content check added against those bypasses refused a listed skill's file whose own frontmatter sets the key. After round 3 we narrowed the rule to the hidden skill's SKILL.md path. The key decides what agents are offered (the index and read_skill), not which files they can read: read_file and execute already open any workspace file, so a content check in read_skill_file protected nothing and kept producing edge cases.
- Round 4 passed on
c9ff25dc29f. Later commits added a behavior-neutral cleanup (cd617533c2e), one test case from the test audit (044c1e6e23a), and a comment fix (9ec3768b18b), which passed validation without another UAT round. Round 5 passed on 2b518c6c2b1, which adds the skill directory. A real model ran hidden skills' scripts from the dir on <invoked-skill>, including directories whose paths contain &, ', ", < and >, and API responses carried no directory. Merging main after feat: show project skill actions above project chat composers #30884 landed (2d7d3cb9674) did not change this PR's diff, so UAT was not rerun. 8f043b2a342, which shows the directory only while the chat stays bound to the skill's workspace, has Go tests but no UAT round.
- The doc-check skill ran in author self-check mode on
2d7d3cb9674, and 27de7f5c6d7 fixes its findings: the skill index sentence said every skill is listed, the stored-skill sentence claimed any project editor import honors the key and presented the API default as fixed, the API path came before the UI path, and some wording broke the docs style rules. The fix is docs-only, so UAT was not rerun.
- Polish and the test audit stayed inside the implementation diff. No new dependencies.
- Codex review found no major issues on
9ec3768b18b, then raised one P2 on the same head when the PR was marked ready: an invoked hidden skill gets no skill directory. A probe confirmed it, and 2b518c6c2b1 fixes it. Its earlier finding on that head (honor CODER_AGENT_EXP_SKILL_META_FILE) was refuted, because the pinned skill snapshot always reads SKILL.md. Both threads are resolved, and Codex found no major issues on 2b518c6c2b1 or on the merge 2d7d3cb9674.
- On
27de7f5c6d7, Codex raised two P2s. One was confirmed: an invoked skill's directory stayed in the prompt after the chat moved to another workspace, which a prompt test showed on that head, and 8f043b2a342 fixes it. The other was declined: on Windows, scripts\..\SKILL.md gets past the SKILL.md check in read_skill_file, but that check is not an access boundary (read_file opens any path), and Windows has more aliases (SKILL.md., SKILL.md::$DATA, SKILL~1.MD) that normalizing backslashes would not stop. On 8f043b2a342, Codex asked to bind the directory to the workspace build instead. That was declined: every workspace start is a new build with new agent IDs, so the directory would drop after each stop and start, and the pinned body and the live files can diverge without a rebuild anyway, as they do for read_skill. A later request on that head asked to limit a hidden skill's read_skill_file access to invocations in the current workspace. That was declined too: read_skill_file resolves names against the current pin for listed skills on main as well, and the key decides which skills agents are offered, not which files they can read. All four threads are resolved. a697dcd927f adds the format:"uuid" tag that TestSDKFieldsFormatted requires on the new field, and Codex found no major issues on a697dcd927f. ae70b2c533f is docs-only: it applies the doc-check suggestion to say that an invoked skill's directory is included only while the chat stays in that workspace. The author doc-check self-check found no blocking findings on it, and Codex found no major issues on ae70b2c533f.
Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
chatgpt-codex-connector[bot] reacted with thumbs up emoji
… focusable and cover submit settlement
…t-skills/project-skills-dialog
…ts through refetches and handle deleted skills
…oject-skills/project-skills-details
… and toggle tests to the Details row PR
…downloads, toggles, refetches and deleted skills
…roject-skills/invocation-api
…ll details after deletes and refresh after download 404s
…t-skills/project-skills-dialog
…oject-skills/project-skills-details
…permission check failures
…etail eviction and download 404 refreshes
…roject-skills/invocation-api
…t-skills/project-skills-dialog
# Conflicts:
# site/src/pages/AgentsPage/components/SkillsTable.tsx
# site/src/pages/AgentsPage/components/SkillsTableView.tsx
…kill refetches on save and respect the skill cap
…oject-skills/project-skills-details
# Conflicts:
# site/src/api/queries/chatProjectSkills.ts
…ed skill refetches on save and the skill cap
…kill permission check before opening
…roject-skills/invocation-api
…led while project skills load
…oject-skills/project-skills-details
…roject-skills/invocation-api
Skills gain two independent settings. action marks a project skill as a
one-click action above project composers; a check constraint keeps it on
project skills, and personal and organization routes reject the field.
model_invocable controls whether agents see and load a skill on their
own: skills with it off are left out of <available-skills> and read_skill,
but structured skill invocations still load them.
read_skill refuses a workspace skill hidden from the model, and before
this branch it was the only tool that returned a skill's absolute
directory. A person can still invoke such a skill, but its body could not
run the scripts it bundles, because the model never learned where they
are.
Invoking a workspace skill now stores the pinned skill directory on the
invocation part, and the prompt renders it as a `dir` attribute on the
<invoked-skill> block, escaped as an attribute value. The field is
internal only: API responses already strip it, and hook payloads now
drop it next to the skill body.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Codex Review: Didn't find any major issues. 🚀
Reviewed commit: 2b518c6c2b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
…odel-invocation
# Conflicts:
# coderd/apidoc/docs.go
# coderd/apidoc/swagger.json
# coderd/exp_chats.go
# coderd/userskills.go
# coderd/x/chatd/ARCHITECTURE.md
# coderd/x/chatd/chathooks/hooks_internal_test.go
# coderd/x/chatd/chathooks/trigger.go
# coderd/x/chatd/chatprompt/chatprompt.go
# coderd/x/chatd/chatprompt/chatprompt_test.go
# coderd/x/chatd/chattool/skill.go
# coderd/x/chatd/project_instructions.go
# coderd/x/chatd/skill_invocation.go
# coderd/x/chatd/skill_invocation_internal_test.go
# codersdk/skills.go
# docs/ai-coder/agents/extending-agents.md
# docs/reference/api/chats.md
# docs/reference/api/schemas.md
# site/src/api/typesGenerated.ts
# site/src/pages/AgentsPage/AgentCreatePage/AgentCreatePage.tsx
# site/src/pages/AgentsPage/components/ChatMessageInput/ChatMessageInput.test.tsx
# site/src/pages/AgentsPage/components/ChatMessageInput/ChatMessageInput.tsx
# site/src/pages/AgentsPage/components/ProjectPage/ProjectSkillsDialog.tsx
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Codex Review: Didn't find any major issues. Keep them coming!
Reviewed commit: 2d7d3cb967
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
…invocation
The skill index sentence said every discovered skill is listed, which a
skill that sets disable-model-invocation: true no longer is. The
stored-skill sentence claimed any project editor import honors the key
and presented the API default as fixed, but edit-mode import keeps the
current setting and an explicit model_invocable wins. A new "Turn off
automatic use" section states when the key applies to stored skills,
with the UI path first.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27de7f5c6d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
An invoked workspace skill stores its directory on the message, and every
later prompt replays it. After the chat was rebound to another workspace,
or unbound, the prompt still pointed the model at that path, which there
either does not exist or holds a different skill.
The invocation now also stores the chat's workspace, and the prompt shows
the directory only while the chat stays bound to it, as workspace file
references already do. The field is internal: API responses strip it and
hook payloads drop it.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f043b2a34
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f043b2a34
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TestSDKFieldsFormatted requires every uuid.UUID field in codersdk to carry
format:"uuid", including internal fields that Swagger ignores.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Codex Review: Didn't find any major issues. Hooray!
Reviewed commit: a697dcd927
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
doc-check: 1 non-blocking suggestion after re-review.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Documentation Check
Re-reviewed at a697dcd9. No blocking findings; every new claim checks out against the code. One non-blocking suggestion is raised inline.
Design notes
- The setting has four names across the API, UI, and frontmatter:
disable-model-invocation (frontmatter), model_invocable (API), Agents can use it automatically (project editor label), and Show as action. This makes the new #frontmatter paragraph and the docs/ai-coder/agents/extending-agents.md "Turn off automatic use" section harder to keep accurate.
Related open PRs
Findings for agents
- id: coverage-1
persona: coverage
class: suggestion
blocking: false
file: docs/ai-coder/agents/extending-agents.md
line: 140
quote: "When you invoke a workspace skill, the agent also gets the skill's directory, so the agent can run the scripts that the skill includes."
evidence:
- coderd/x/chatd/chatprompt/chatprompt.go:1368
- coderd/x/chatd/chatprompt/chatprompt.go:1355
rule: null
problem: The invoked-skill directory is only included while the chat stays bound to the owning workspace; the docs do not say this.
fix: "When you invoke a workspace skill, the agent also gets the skill's directory, as long as the chat stays in that workspace, so the agent can run the scripts that the skill includes."
verified: true
Automated review via Coder Agents. Reply to a finding's thread with the reason if the finding is wrong.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Codex Review: Didn't find any major issues. Keep them coming!
Reviewed commit: ae70b2c533
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
f0ssel
Awaiting requested review from f0ssel
+1 more reviewer
Reviewers whose approvals may not affect merge requirements
Uh oh!
There was an error while loading. Please reload this page.
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|