GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||
GeezerVison is showing github.com — docs(oracles): ship setup, design, and review skills, with a prompt to try them by KyleAMathews · Pull Request #7 · TanStack/oracles Conversation
|
| Skill | What the agent does with it |
|---|---|
oracles-setup |
Installs both packages at one pinned commit, configures lint, writes the AGENTS.md block, makes the skills findable, and takes a first oracle through the gap check. |
oracles-design |
Writes the oracle file's prose laws first and stops for review. Then it chooses a model, builds histories that reach the law, shows the evidence the change needs, and closes every gap. |
oracles-evaluate-review |
Acts on review findings. It keeps a lossless ledger, shows each bug on unchanged production, turns each bug into a family of histories, and closes the class. |
oracles-review |
Rebuilds one observation from the contract, checks the order of the work, and hunts weaknesses that tools cannot see. |
The skills make the right move the obvious one. They do not rely on the agent to judge when a rule applies.
observed and asks the user.Fresh agents ran the skills on 12 open-source libraries over five rounds. Each agent got only the packed packages and one library it had not seen. Each round's findings changed the skills, or went to the framework as an issue.
DataView copy loses its window, and a null-prototype object comes back with Object.prototype. In dequal, an own constructor key breaks equality. Nobody filed these upstream yet.A sixth round is in progress on three libraries that the maintainer owns.
docs/oracle-checklist.md holds ORC-001 to ORC-016, ported from TanStack DB's rubric as version 1. The IDs did not change, so existing citations still work. ORC-003 requires each oracle file to be a literate program. A table at the end lists every difference from the rubric and its reason.
docs/glossary.md defines the oracle terms, including law sheet, exit, framework bug, invalid mutant, allowed set, and boundary condition.docs/model-styles.md gives one small, type-checked example of each of six model styles.docs/guide.md holds the rubric's technique sections, made generic. It adds no requirements.docs/instruments/ holds six reasoning cards. The design grammar card names the parts of a model, records rule conflicts, and sketches adjacent designs, so the user can choose between them.Two framework changes wait for the maintainer's decision on the type domain. The first skips guards that TypeScript narrowing makes constant (F11). The second adds mutation operators (F17). Until they land, the design skill reports a constant guard as a CLI limitation, and it leaves production unchanged. That guard is either a branch after satisfies never or the test for the last member of an exhaustive union.
main already has everything that the skills teach, through 84cd34a (feat(oracles): owns on oracleProperty; a run that judges nothing fails; namespaced output #11 to feat(oracles): --config, wrapper configs that match the config's module format, and no listener warning with many contexts #26).docs/oracles-package-skills) bumps DB's pin to this commit and merges. It turns DB's oracle-review and oracle-authoring into addenda that load these skills.clamp oracle, built from the packed packages at this branch's head, passes tsc and 4 of 4 Vitest tests. The gap check on it reports 1 mutant, 1 killed, 0 gaps, and exits 0.git ls-remote returns the current commit, and both pkg.pr.new URLs resolve at its 7-character hash.tests/package.test.ts passes, so npm pack includes skills/ and docs/. All 53 relative links and anchors in the new docs resolve, and Prettier passes.Start with packages/oracles/skills/oracles-design/SKILL.md, then packages/oracles/docs/oracle-checklist.md. The setup skill and the package README show the API as an adopter uses it.
This pull request and its description were written by Isaac.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…checklist
The package now carries what an adopter's coding agent needs, inside
node_modules:
- skills/oracles-setup: install both pinned packages, configure lint,
write the AGENTS.md block, make the skills findable through repository
or global pointer skills, and run a first oracle and gap check.
- skills/oracles-design: law sheet first, model choice, a witness for
every distinction, histories and reach, driver and check, RED, GREEN,
hostile mutant, and closing every gap with no exits.
- skills/oracles-review: rebuild one observation, the shortcut table,
the weaknesses tools cannot see, and the finding format.
- docs/oracle-checklist.md: ORC-001 to ORC-016 (version 1), ported from
TanStack DB's rubric with the same IDs and a table of every change.
- docs/glossary.md, docs/guide.md, and five instrument cards.
- READMEs for both packages; `files` now includes skills and docs.
The skills teach `oracleProperty({ owns })` and no `law.owner`, so this
waits for those framework changes before it merges.
Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
There was an error while loading. Please reload this page.
🦋 Changeset detectedLatest commit: daab435 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
commit: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
`oracles-design` step 3 named six model styles but showed no example of any but a pure function. docs/model-styles.md gives a small, generic, type-checked example of each: a pure function, a naive reference structure, a transition table with derived reach, a pure step function, a relation between runs with its calibration, and an allowed set. It also states two rules from the model prototypes: each model rule is one sentence of the law, and legality lives in a separate pure guard. Co-authored-by: Isaac <[email protected]>
Oracle reviews were sometimes read as security work. The skills and docs now avoid words that suggest it: the adversarial-review card is the challenge-review card, attacks are challenges, a hostile mutant is a mutant (as TanStack DB's glossary already asks), a fault-injected step is a step that simulates a signal, and "bypass", "defeat", and "payload" are reworded. Links follow the renamed card. Co-authored-by: Isaac <[email protected]>
Design step 2 now asks the agent to name the model's parts and rules, map each rule to a sentence of the law, rebuild the known examples before trusting the model, and record rule conflicts as open questions. docs/instruments/design-grammar.md carries the heavier method for a hard law or a growing model: a stated preservation relation, parts admitted only for a job, overlaps, reconstruction and simplification by a different representation, a two-way comparison for every simplification with refactor, refine, and revise kept apart, and an optional TLA+ check with a weakened-configuration control. The design and review skills and the card index route to it. Co-authored-by: Isaac <[email protected]>
- New skill oracles-evaluate-review: a lossless ledger of review findings, enforcement gaps fixed and law changes brought to the user, one oracle plan for the whole review run against unchanged production, each bug turned into a family of histories with a nearby passing case, and a loss audit. - oracles-design: the approved law sheet becomes the law block at the top of the oracle file; a run-budget heuristic (critical states by construction and probes, starting runs, fast files, longer sweeps by the run multiplier, a lowered budget is a weakening); lint after each edit; a production cleanup step after GREEN; the evidence block goes in the pull request description. - oracles-review: findings for a lowered budget and for inserted rather than refactored code; a law block that disagrees with the law option. - oracles-setup: the AGENTS.md block names the new skill and asks for lint after each edit; pointer skills include the new skill. - Skill descriptions and a few sentences follow the STE structural rules. Co-authored-by: Isaac <[email protected]>
#6 made `recommended` turn off inline directives and widened the `production` glob to `src/**`, so setup now adds both presets as they are, without the workaround blocks. The plugin README lists no-oracle-disable and the new glob. The design skill now asks for at least one fixed seed (an empty list will be refused) and treats a random-only kill as a gap until its history is kept. Co-authored-by: Isaac <[email protected]>
Two fresh agents set up the package in vercel/ms (Jest and Biome, no ESLint) and developit/mitt (mocha, ESLint 7), using only the packed packages and the skills. mitt reached zero gaps. ms stopped correctly on a law that nobody can decide for a library the adopter does not own. Setup now covers what they hit: - the Node version against CI, the lint setup, and the command runner - `eslint@^9` when the repository has none, a stable parser major, and TypeScript 5 or newer for the oracle files, kept out of an older `tsc` - where oracle files go, and a Vitest config beside another runner - a separate ESLint 9 run for an ESLint 7 or 8 repository, with a check that the rules are live - `owns` entries that name one function - `--source` selecting files, an explicit `--seed`, `--out` inside the repository, and `.oracle-gaps/` in `.gitignore` - dead code that the gap check finds counts as a production change The design skill teaches function-level `owns`, dead-code removal as a production change, and what a law that kills nothing means. Co-authored-by: Isaac <[email protected]>
- Evidence by kind of change: a defect fix shows RED, GREEN, and the defect as a mutant; a behavior-preserving refactor shows the same oracle passing on old and new production with kill sets compared; a new oracle over correct code shows production passing and calibrations failing. Review no longer treats a refactor without RED as a shortcut. - Approval follows the semantics: strengthening an adopted law reuses it without a new sheet; a law change needs a law delta; a new law or missing authority needs the full sheet; an approval the task already gave stands. - A boundary-selection step before the law sheet, with an example. - Model state is justified by a legal continuation (one or more actions), not only the next action, in the design skill, ORC-008, the guide, and the law-discovery card. - Dead code needs a contract reason before removal; a surviving mutant only shows what the current histories did not expose. - A survivor that the adopted law already allows is reported as an allowed alternative, not sent back as a law question. - Setup: pointers and the AGENTS.md block read the chosen package's installed copy in a workspace; the TypeScript parser covers production source as well as oracle files. - The round-trip example compares tags element by element and names the two corruptions the joined comparison accepted as calibrations. Co-authored-by: Isaac <[email protected]>
The framework judges a mutant over the domain the operand types admit, so a narrower domain stated only in prose does not make a mutant invalid. Co-authored-by: Isaac <[email protected]>
Four fresh agents ran the branch's skills on four workflow branches: a behavior-preserving refactor, a review with two findings, a planted defect, and a package-local workspace. Each took the right branch. Their findings, plus the framework session's answers: - Refactor evidence follows the gate's rule: oracle files unchanged or only strengthened and passing on old and new code with the same seed; shared killed keys still killed; new keys in changed functions killed or acted on. - Design: an allowed alternative is its own outcome; a gap that existed before a change still counts in a changed function; widening histories inside the stated domain is strengthening; a grammar that covers part of the law's domain says so; constructed histories must fit the arbitrary; the driver records production errors; own the whole file when an owned function reads module data; several oracles may own one function; public state needs a law or a limit; an exhaustiveness-guard survivor is a known CLI bug, recorded without stopping or removing the guard. - Review: a law-dependent state in `measured` is an unreachable claim. - Setup: name the oracle config `vitest.config.ts` (the gap check loads only that name), pnpm 11's allowBuilds placeholder, refresh pointers on upgrade, the AGENTS.md block in a package-local workspace, keep oracle files out of the repository's own lint or match its style, install before a pointer gives up, and a path for runs with no user. Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
With #11 merged, the README sample type-checks and the gap check counts it as an oracle file. It still left one gap: `>` to `>=` differs at clamp(-0, 0), and the histories never generated -0. The example now covers its type's domain, as the design skill teaches: -0 is in the grammar, a constructed history enters "negative zero at max", and the gap check kills all three mutants. Co-authored-by: Isaac <[email protected]>
Three fresh agents adopted the package in p-limit (async JavaScript, xo, AVA), dequal (uvu with esm), and nanostores (TypeScript 7, oxlint). All three followed the method, and dequal's agent found real library defects. Their findings, plus the framework session's answers: - Setup: offer a separate Node 20 job when CI is older; use the repository's package manager for install commands; place oracle files where the existing runner will not load them (their own directory, or a `*.oracle.ts` name); probe-form for root-level source; xo repositories; the TypeScript 7 plugin bug as a temporary note; the live-rule check through stdin before any oracle file exists; choose a function with a documented promise; a first oracle that is RED found a defect (report it, never weaken the oracle, run the gap check on a scratch fix until the CLI judges a failing baseline); group the open gaps in the report. - Design: an analogy is not authority; laws people forget (no throw on valid input, the algebraic laws of the kind of function); constructed values derived from the generated history; whole-file entries that pull in another contract; owned code that serves another module; where `oracleProperty`'s options are typed; do not copy exit wording into the agent's own report; a synchronous-spin timeout is a CLI limitation. Co-authored-by: Isaac <[email protected]>
…mmar
Agents go from examples to a general law only when made to write it
down, and no agent in three validation rounds opened the design-grammar
card, because it triggered on judgment ("a hard law").
- Design step 1: before the law sheet, each law gets a written block:
the examples with sources, what varies and what stays, the law as one
"For every ..." sentence with its edges, a new case it predicts, and
every example rebuilt from the sentence alone.
- Five concrete triggers send the agent to the design-grammar card; the
evidence says which fired.
- Review: "Law as examples" and "Skipped design grammar" findings.
- Evaluate-review: the causal condition is written as a sentence over
histories, with a second history it predicts.
- Setup: TypeScript 7 is an environment limit (typescript-eslint), and
the plugin's `unparsedModule` report for a `.js` oracle file that
imports TypeScript (#13).
- Design: the interim exit-wording bullet is gone with #12; the rule not
to write those words in a report stays.
Co-authored-by: Isaac <[email protected]>
- Design step 1, item 6: sort each law into dynamics (how things change, stated by the model's rules), constraints (what holds in every history), and boundary conditions (the edges, where real behavior and most defects live). Every boundary gets a constructed history or an open question; it is never left out of the grammar. Every defect the validation agents found was at a boundary. - Adjacent forms keep design diversity: the design-grammar card's new step 8 sketches two to four forms that keep every adopted law, for the user to pick; design step 3 names a second fitting model style; design step 7 sketches alternative cleanup designs instead of keeping the first. - Glossary: dynamics, constraint, boundary condition, adjacent form. Co-authored-by: Isaac <[email protected]>
The law sheet is now a template that the agent fills in for each law. Its examples, law sentence, new case, and dynamics, constraints, and boundary conditions are fields, and the summary table is only an index. In round 4, one of four agents skipped the written block because it was prose before a table. Other fixes from round 4 (validation findings V64 to V96): - design: arbitraries must take histories back (`.map` needs its inverse); string units; observations are data; brand checks, not instanceof; methods of a returned object are owned through the enclosing function; the whole-file entry only when module data affects a law; authored mutants when operators are sparse; triggers checked again after each gap check, counting every row. - setup: the no-user default covers setup questions only; the end condition is the first oracle's own contract; keep an existing Vitest include; register the plugin in a custom probe-form block; xo and TS 7 installs; the live-rule probe needs the fast-check import; placeholders on a stopped run; scratch copy wording. - review: the shortcut rows match the template and the later trigger check. Co-authored-by: Isaac <[email protected]>
…Script The framework credits a kill to the first law that a history violates, so a later law can look idle. The design skill now says to aim hand mutants at that law before restating or deleting it. The setup skill reports a TypeScript copy that npm hoisted from @tanstack/oracles as a known packaging bug, so it is not taken as the repository's own TypeScript. Both are temporary: the framework fixes are queued. Co-authored-by: Isaac <[email protected]>
The law block in an oracle file is now prose for a reader who has not seen the sheet: a question for a title, how the code works, the documented promises, how the model states them, and what the driver observes. The sheet's fields stay in the review record. The AGENTS.md block (template 2) says who owns what. The maintainers own the laws. The agent owns the production code that the oracles cover, and makes it the smallest, clearest code that passes them. Round-5 validation fixes: - A stated limit no longer closes a gap. Setup's end condition, its report groups, design step 8, and the review's exit row now say so. - A block whose domain is its own example is a fixed history, not a law. - The string-unit advice names `binary` (every code point except lone surrogates), since `grapheme` has no control characters. - A guard that TypeScript narrowing makes constant (the last member of an exhaustive union) is a known CLI bug. Other mutants on its line still count. - A `not-observed` survivor can need a history where the branches diverge. - The design-grammar trigger after a gap check is a stop, not a note. - An idle law that another law covers is merged. No empty tests in oracle files. `vitest.config.mts` in a CommonJS package. The AGENTS.md lint line when lint cannot run. Co-authored-by: Isaac <[email protected]>
The law sheet is now the new oracle file before it has any code: an opening comment that states the laws as prose (ORC-003), and a review-notes comment with the per-law template that turns examples into a general law. After approval, the review notes are deleted, and the code is added under the prose. A law delta edits the prose in place. This removes the step that copied an approved sheet into the file. Co-authored-by: Isaac <[email protected]>
…ons as dated authority A potential law backed by a document is 'stated' until the engineer approves it, so 'adopted' no longer leaks into the chat before approval. A user's decision is written as a dated maintainer decision, not as the session's story. Co-authored-by: Isaac <[email protected]>
…ed them as peers Co-authored-by: Isaac <[email protected]>
…a probe Co-authored-by: Isaac <[email protected]>
…hority to quote Co-authored-by: Isaac <[email protected]>
…one in a scratch copy Co-authored-by: Isaac <[email protected]>
…he repository's own Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
…test 4.1 in a first session - oracles-design: each oracleProperty carries one law; properties may share the model, grammar, and driver. - oracles-start: the potential oracle always shows its contract block, keeps the worksheet's backing word private, and installs vitest@^4.1 (Node 20+), leaving CI Node versions to production. Co-authored-by: Isaac <[email protected]>
…GitHub login On a team repository a reader needs to know whom to ask. The login comes from gh, or the engineer; it is written without an @ so a copy into a pull request notifies no one. Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
…is removed Co-authored-by: Isaac <[email protected]>
…acle runs the gap check; the AGENTS.md block governs from the start - oracles-start: probes exist to learn the code's dynamics, boundaries, and constraints that the oracle needs; the agent writes down what it does not know and probes each point. Bugs are a welcome side effect. - When the first oracle finds no bug, run the gap check (output outside the repository) and present what the surviving changes show, before offering the next oracle. A failing oracle still stops at the RED. - Before the first message, read the block that production adds to AGENTS.md: it governs the agent from the start of the session. - Remove every mention of legacyCampaigns from the docs and skills. Co-authored-by: Isaac <[email protected]>
… chat, before probing Co-authored-by: Isaac <[email protected]>
From a semantic drift assay of cc88112..4749f61: - the gap check's --out follows oracles-setup step 7 (in .oracle-gaps/), and oracles-setup names step 7 as part of a first session - the session's ending and the walk-through cover the no-bug path - a decision records name, GitHub login, and date in start and checklist - one Vitest 4.1 / fast-check 4 pin across start, setup, and the README - a helper that forwards literal owns registers tests, in setup as in design - adopted means approved, in the glossary - 'gap check in CI' had no source; it is the gap check over all owned code Co-authored-by: Isaac <[email protected]>
…when the session ends Co-authored-by: Isaac <[email protected]>
…es/problems.md The skill keeps the main path and a pointer; the agent opens the reference when it hits another test runner, an older Vitest, a failed install, peers already on disk, or a probe that Node cannot load. Co-authored-by: Isaac <[email protected]>
…ciled; a preview feedback log - 'developer' replaces 'engineer' and 'user' for the person the agent works with; 'maintainer' is only the role with authority over the laws. A developer's approval adopts a law and is recorded; on a shared repository the maintainers confirm it in review. A library's end user is 'the caller'. - 'approved' is the act and 'adopted' the state; the glossary glosses owns as a public behavior in conversation; 'sketch' becomes 'draft'; the finding kind 'Test gap' becomes 'Untested behavior'. - While in preview, a first session keeps a feedback log outside the repository and asks the developer to send it to Kyle Mathews. Co-authored-by: Isaac <[email protected]>
… not the home directory Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
…dback log at the start Co-authored-by: Isaac <[email protected]>
…e repo's module-type warning is harmless The contract and law-block examples used klona, a library the skills are validated on; an agent noted that the example could prime its findings. They now use a fictional slugify. Co-authored-by: Isaac <[email protected]>
… lint; no gap check on a failing oracle Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
Co-authored-by: Isaac <[email protected]>
…offer a weaker one as a choice A first session asked the developer 'Full model, or checks only?'. The term meant nothing to them, and the answer should always be the strongest model. How the oracle is built is the agent's decision. Co-authored-by: Isaac <[email protected]>
While only invited people try it, the README holds a one-sentence overview and the prompt. The session itself explains the rest. Co-authored-by: Isaac <[email protected]>
…, not the skill's
From manual review: an opening that copied the skill's example ('a behavior
that's easy to break and has a simple rule behind it') sounded nonsensical,
and a probe plan listed the skill's categories ('dynamics, constraints,
boundaries, timing, and lifecycle') instead of questions about the code.
Co-authored-by: Isaac <[email protected]>
…pository under test Co-authored-by: Isaac <[email protected]>
…rst then fix, close gaps until something turns up From the 2026-11-11 UX test (Kyle Mathews and Tanner Linsley) and its evaluate-review ledger, validated with first sessions on klona, quick-lru, and p-queue: - Open with plain sentences about this repository (no quotable example), look around with impact signals (the most important API, README examples, user-facing code, strongly stated promises in old code, rare edge cases that take a system down, repeated fixes), present two or three spots, and stop for the choice. - A table of uncommon words, each explained in one line the first time it comes up, including gap check and gap. - Findings carry 'why it matters' and are ranked by impact; the first small one says why small bugs appear; trivial ones are left out of the findings and fixed later with the others. - Oracle first: a bug a probe finds goes into the oracle before any fix. Confirmed bugs are fixed right away, with the oracle failing before and passing after. - When the oracle passes, close gaps at once: ask why the oracle missed each one, widen the histories or the check under the approved laws, and keep going until a bug or a design question turns up. - The gap check's report goes to a temporary directory; no cleanup talk. - 'Take the oracle to production' becomes 'finish setting up the repository for oracles', offered when the developer wants to keep the work. oracles-setup is retitled to match. - problems.md: the repository's lint picking up the oracle file. Co-authored-by: Isaac <[email protected]>
…ized A finding names the rule the code breaks and the share of inputs that break it, with the smallest case as the example. No finding is called minor or trivial, and none is left out: the count and the enforced design say why each one counts, as in the oracle issue reports. Co-authored-by: Isaac <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.