GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||
GeezerVison is showing github.com — fix(eslint-plugin-oracles): rules never throw, and oracle rules cannot be disabled inline by KyleAMathews · Pull Request #6 · TanStack/oracles Conversation
Contributor
The oracle lint rules now hold their two promises. They never throw, and a comment in the file cannot silence them. Before this change, one bad setting made Why it mattersAn agent reads lint output after each edit. A rule that throws stops the run, so the agent gets no report for any rule in that file. A rule that a comment can switch off gives the agent an easy exit: it disables the rule and the report goes away. Both behaviors break the decision that oracle lint is mechanical and cannot be bypassed. What changesThe rules do not throw. Oracle files ignore inline config. The A new rule reports directives that would silence an oracle rule.
A directive can hide a report on the lines that it covers. This includes a directive that names this rule or has no rule list. The rule reports such a directive at the first token of the file. The message gives the line of the directive. // eslint-disable-next-line "oracle/probe-form" globalThis.__oracleProbe?.('x', 1) // now reported: the quoted rule name no longer hides the directive Probe-form also checks single-package repositories. The LimitsThese cases stay open. The rule header, the README, and the changeset document them:
How the oracles check itEach fix came after its oracle extension. The test commit is RED at its tip, and the fix commit makes it GREEN. Each fix commit has a hostile mutant that its oracle rejects. The commit messages contain the evidence blocks.
Verification
Merge with squash: the test commits are RED at their tips on purpose. This pull request and its description were written by Isaac.
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactions…edictions A rule that throws aborts ESLint's lint of the file, so the error is not a judged outcome. The four oracles that lint a rule now record a throw as an observation, and the predicted reports reject it. Law: a rule never throws on a program that parses. Authority: P8 of the frozen v20 design grammar (~/.cache/oracle-grammar/v20-design-grammar/freeze.md) and the maintainer decision of 2026-11-09, "lint rules never throw". AGENTS.md "Lint feedback while you work" needs the hooks to report on every edit, which a throw breaks. Oracles extended: probe-form (lintWithProductionConfig), fixed-seed (lintProgram: no-disabled and replayable-seed), replay-observation (reports), oracle-property-only (lintRendered). Reach: the law's states are the programs each driver lints. The oracle-property-only grammar reaches the throwing state through its malformed entries (bareString, fileNotString, reasonNotString, outsideGlob, implementationModule); the other three drivers reach it through their generated programs. RED (parent e1fb504 rules, this commit's oracle): cd packages/eslint-plugin-oracles && npx vitest --run tests/oracle-property-only-oracle.property.test.js 2 of 5 tests fail (fixed seed 20261109, random seed): "the rule threw, which aborts ESLint's run of the file: ... Error while loading rule 'oracle/oracle-property-only': ... legacyCampaigns entry 0 is not an oracle file". This RED is intentional: the next commit changes the rule and the model. GREEN (this commit, probe-form, fixed-seed, replay-observation unchanged on parent rules): all four test files pass except the oracle-property-only bad-entry tests above, which the next commit turns green. Mutant: a hostile `throw` at the top of probe-form's and no-disabled's create for oracle files. probe-form's four checked outcomes and no-disabled's fail with `threw: Error while loading rule 'oracle/probe-form': hostile mutant: rule threw`. Before this commit the same mutant ended the run with the thrown error instead of a checked outcome. Calibration: a throw injected into a clean history of oracle-property-only is rejected. Limits: a rule that never returns (a hang) is not covered. The oracle-property-only bad-entry tests stay red until the rule reports them. Part-reach oracles (no-unit-test) do not lint their drivers through a throw-observing path yet. Co-authored-by: Isaac <[email protected]> …yCampaigns entry The rule throws when it reads a malformed or unjudgeable legacyCampaigns entry, so ESLint aborts the file with "Error while loading rule". The model owed a configuration error for those entries. Under the never-throws law the owed outcome is a lint report, so the oracle now expects one, and it names the rule's report for each kind of bad entry. Law: a rule never throws on a program that parses. A malformed entry, or a legacyCampaigns value that is not an array, is reported as malformedEntry or malformedList at line 1; an entry that names a file outside the oracle glob (including README.md) or an implementation module is reported as refusedEntry. A reported entry exempts nothing; the file's other entries are judged as usual. Authority: P8 of the frozen v20 design grammar and the maintainer decision of 2026-11-09, "lint rules never throw". Grammar extended: entry kinds readme, missingFile, fileNotString, notAList (bareString, reasonNotString, outsideGlob, implementationModule were already generated). Constructed histories added for a README entry and a non-array list. The model's expected outcome is now a report list; the configuration-error branch is removed. Reach: each new state is entered by a constructed history (README entry, refused entry, malformed entry, non-array list) and by the generator's program (60 runs, fixed seed 20261109). RED (parent: the rule from e1fb504, this commit's oracle): cd packages/eslint-plugin-oracles && npx vitest --run tests/oracle-property-only-oracle.property.test.js 2 of 5 fail (fixed seed 20261109, random seed). First counterexample: {"entry":"outsideGlob", ...}; difference: "the rule threw, which aborts ESLint's run of the file: a rule never throws on a program that parses". The rule throws for every bad-entry kind, so each constructed history fails. GREEN: the next commit changes the rule. Mutant: the parent rule itself is the hostile mutant (it throws on every malformed or refused entry). The calibration "a rule's throw in a clean history" replaces a clean outcome with a throw and the check rejects it. Limits: implementationModules, which the rule also reads, is not generated; a non-array implementationModules still throws until a later change covers it. A rule that never returns is not covered. Co-authored-by: Isaac <[email protected]> …Campaigns entry, never throws A malformed or unjudgeable legacyCampaigns entry made the rule throw while it read the settings, so ESLint stopped with "Error while loading rule" and the hooks reported nothing for the file. The rule now reports the entry instead. Law: a rule never throws on a program that parses (P8 of the frozen v20 design grammar; maintainer decision 2026-11-09). Reported entries exempt nothing; the file's other entries are judged as usual. Change: - malformed entry -> malformedEntry at line 1 - legacyCampaigns not an array -> malformedList at line 1 (replaces the file's list) - outside the oracle glob (including README.md) or an implementation module -> refusedEntry at line 1 ESLint runs a rule once per file, so each oracle file that the configuration reaches reports the entry. README and the changeset text that said "the rule throws" now describe the reports. Reach: the extended oracle (previous commit) reaches each state: refused, malformed, and non-array lists, in 60 generated runs (fixed seed 20261109) and in constructed histories for a README entry and a non-array list. RED (parent: rule at e1fb504 with the previous commit's oracle): cd packages/eslint-plugin-oracles && npx vitest --run tests/oracle-property-only-oracle.property.test.js 2 of 5 fail (fixed seed 20261109, random seed); first counterexample entry "outsideGlob", difference "the rule threw, which aborts ESLint's run of the file". GREEN (this commit): the same command, 5 of 5 pass, every state reached (the oracle prints the reach counts for both seeds). Mutant: (a) drop the malformed entry's report: 2 of 5 fail, difference "the observed value differs from the owed value". (b) refused entry still judged (exempts): 2 of 5 fail, same difference. Both mutants were run on the fixed rule and then restored. Limits: implementationModules is not generated; a non-array implementationModules still throws. A rule that never returns (a hang) is not covered. The entry checks cannot tell a new baseline entry from an old one (unchanged). Co-authored-by: Isaac <[email protected]> … oracle rule P8 of the frozen v20 design grammar: lint rules cannot be disabled inline. The oracle-property-only oracle had no directive in its grammar, so a directive that silenced the rule was never generated. The grammar now writes a block directive on the alias import's line, which covers every site below it: `/* eslint-disable oracle/oracle-property-only */` or a bare `/* eslint-disable */`. The law: the reports are the same with or without the directive. Authority: P8 (frozen v20 design grammar, lint rules cannot be disabled inline), and the maintainer decision of 2026-11-09 that the oracle rules are mechanical. Grammar extended: directive kinds none, oracleBlock, allBlock. The state "a disable directive is written over a reported site, and it does not silence it" is measured and has a constructed history (a block directive over a raw campaign). The model is unchanged: the expected reports ignore the directive. Reach: the constructed history plus the generator (60 runs, fixed seed 20261109). The one-line form (eslint-disable-next-line) is not generated because it moves the site lines; the block forms are generated. RED (parent: the rule and config from e1fb504 plus the earlier fixes): cd packages/eslint-plugin-oracles && npx vitest --run tests/oracle-property-only-oracle.property.test.js 2 of 5 fail (fixed seed 20261109, random seed). First counterexample: {"directive":"oracleBlock", "sites":[{"kind":"prop","root":"fcTest"}]}; difference "the observed value differs from the owed value": the block directive silenced the rule. GREEN: the next commit sets noInlineConfig on the recommended config. Mutant: the next commit's config with `noInlineConfig` removed fails the same two tests (difference "the observed value differs from the owed value"). Limits: a one-line directive is not generated. The production config (probe-form on production source) is not covered by this law: it still honors directives, pending a maintainer decision (see the next commit's message). Co-authored-by: Isaac <[email protected]> …s of oracle rules P8 (frozen v20 design grammar): lint rules cannot be disabled inline. A `// eslint-disable-next-line oracle/oracle-property-only` (or a block directive) silenced the rule in any oracle file, so the rule was not enforced where the directive sat. The `recommended` config now sets `linterOptions.noInlineConfig`. ESLint then ignores the directive and warns that it has no effect. Side effect, stated: ESLint has no per-rule switch, so in an oracle file the setting also ignores directives for other rules. ESLint reports each ignored directive as a warning. The repository's own directive for no-sparse-arrays in packages/oracles/tests/oracle-property-scheduler-oracle.property.test.ts is in an oracle file, so it was replaced with an equivalent construction (a hole at index 0 made by Object.assign on new Array(2)). Not changed: the `production` config (probe-form on production source). Setting noInlineConfig there would also ignore every other rule's directive in production source, which is a maintainer decision, so probe-form can still be disabled inline in production files. Recorded as a limit. Law: in an oracle file a disable directive does not silence an oracle rule (the reports are the same with or without it). Authority: P8, frozen v20 design grammar. Reach and RED: the previous commit's oracle (directive dimension, constructed block-directive history, 60 generated runs at fixed seed 20261109). RED (parent: config without noInlineConfig, this commit's oracle): cd packages/eslint-plugin-oracles && npx vitest --run tests/oracle-property-only-oracle.property.test.js 2 of 5 fail, difference "the observed value differs from the owed value". GREEN (this commit): the same command, 5 of 5 pass, with every state reached. Full plugin suite: 14 files, 4290 tests pass. The scheduler oracle in packages/oracles passes (7 of 7) after the directive replacement. Mutant: the config with noInlineConfig removed (the parent) fails the same two tests: the block directive silences the rule. Limits: production config not covered (see above); a one-line directive (eslint-disable-next-line) is not generated by the oracle. Co-authored-by: Isaac <[email protected]> The production config's files glob is packages/*/src/**, so a single-package repository with src/ at its root gets no probe-form reports. Production source is src under a package or at the root of a single-package repository, and the oracle must generate both layouts. Law: a state probe in production source is reported exactly where the form is broken, in either layout; the root config enables the rule for production source alone. Authority: the probe-form law's existing authority (the maintainer's decisions of 2026-11-08 on the probe form) and P8 of the frozen v20 design grammar for the production config's coverage of the rule. Grammar extended: programArbitrary draws a layout (monorepo or singlePackage); the drivers lint the program under packages/oracles/src/ (monorepo) or src/ at the root (singlePackage). The census gains a single-package history, and the state "single-package layout" is measured, with a constructed history. Reach: the constructed history plus the generator (fixed seed 2234, 40 runs, and the census). The layout does not change the model's reports, only the file the drivers name. RED (parent: the plugin's production config from e1fb504, this commit's oracle): cd packages/eslint-plugin-oracles && npx vitest --run tests/probe-form-oracle.property.test.js 3 of 9 fail: census with a guard, single-package layout; fixed seed 2234; random seed. First counterexample: layout "singlePackage", a "let-guard" site, difference "the observed value differs from the owed value": no probe-form reports, and the root config has no rule for src/ at the root. GREEN: the next commit adds the single-package glob. Mutant: the parent config (no src/ glob) is the hostile mutant for this law; the same three tests fail. Limits: the gap-report oracle (packages/oracles/oracle-gaps/ gap-report-probe-form-oracle.property.test.mjs) shares the grammar and does not read the layout, so its own declared reach does not include the single-package layout. Recorded here for the owner. Co-authored-by: Isaac <[email protected]> …ge layout The production config's files glob was packages/*/src/**, so a single-package repository with src/ at its root got no probe-form reports. The config now names both layouts: packages/*/src and src, with the same source extensions. Law: the probe form is enforced in production source in either layout (see the previous commit's oracle; authority: the probe-form law's maintainer decisions of 2026-11-08 and P8 of the frozen v20 design grammar). RED (parent: the production glob from e1fb504, the previous commit's oracle): cd packages/eslint-plugin-oracles && npx vitest --run tests/probe-form-oracle.property.test.js 3 of 9 fail (census with a guard, single-package layout; fixed seed 2234; random seed), difference "the observed value differs from the owed value". GREEN (this commit): the same command, 9 of 9 pass, with the single-package layout reached (generator, census, and constructed history). Full plugin suite: 14 files, 4291 tests pass. Mutant: the parent glob (without src) fails the same three tests; the layout-specific reports are the states the mutant loses. Limits: the oracle draws `.ts` file names only, so an extension missing from the glob (for example .tsx under src/) is not caught by this oracle. Recorded as a gap for the owner. The gap-report probe-form oracle shares the grammar and does not read the layout. Co-authored-by: Isaac <[email protected]> …e is reported
P8 of the frozen v20 design grammar says lint rules cannot be disabled inline.
The noInlineConfig setting covers oracle files, but the production config cannot
take it without ignoring every other rule's directives in production source. The
maintainer chose to report the directive instead. This oracle owns the new rule
oracle/no-oracle-disable, which the next commit adds to both configs.
Law: a disable directive that names an oracle/* rule is reported on its own line,
in an oracle file and in production source, in the line, next-line, and block
forms (block and block-line), with or without a description, and whether the list
names only oracle rules, only other rules, or both. A directive that names no
oracle rule is not reported. A throw is a violation: the rule never throws.
Authority: P8 (frozen v20 design grammar) and the maintainer's decision on the
production-config limit (option b).
Limits written in the oracle: a blanket directive with no rule list is not seen
(ESLint suppresses the report itself), and a directive naming the rule itself can
hide the report on the line below it. Neither is generated.
Grammar: five directive forms, five rule-list kinds, a description flag, and two
locations (oracle file, production source). Constructed histories cover every form,
list kind, location, the described form, and the silent case.
Reach: each state is entered by a constructed history, and the generator (60 runs,
fixed seed 20261110) draws the rest.
RED (current tip, before the rule exists; the rule is not in either config):
cd packages/eslint-plugin-oracles && npx vitest --run tests/no-oracle-disable-oracle.property.test.js
3 of 5 fail (fixed seed 20261110, random seed, calibration "a report dropped").
First counterexample: {"location":"oracleFile","sites":[{"form":"nextLine",
"list":"oracleOne"}]}, difference "the observed value differs from the owed
value": the owed report is missing. The calibration fails because no clean
history has a report yet.
GREEN: the next commit adds the rule and enables it in both configs.
Mutant: the next commit reports line and next-line forms only when the next-line
or line form is ignored; the same tests fail.
Co-authored-by: Isaac <[email protected]>
… oracle rule
Oracle rules cannot be disabled inline (P8, frozen v20 design grammar). The
recommended config's noInlineConfig covers oracle files only. The production
config cannot take it without ignoring every other rule's directives in
production source, so the maintainer chose to report the directive instead.
The new rule oracle/no-oracle-disable reports each comment whose directive lists
an oracle/* rule: the eslint-disable, eslint-disable-line, and
eslint-disable-next-line forms, as line or block comments, with or without a
description, and whether the list is oracle only, other only, or mixed. The
report sits on the comment's own line. It is enabled in both recommended and
production. The rule does not throw: it reads comment text only.
Law: as in the oracle commit before this one (no-oracle-disable-oracle.property.test.js).
Authority: P8 and the maintainer's decision on the production-config limit (option b).
Limits: a blanket directive with no rule list is not seen, because ESLint
suppresses the report itself on the lines it covers. A directive that names
oracle/no-oracle-disable can hide the report on the line below it (its own
report is still made unless suppressed by a further directive). Neither form is
generated.
Reach: the oracle's constructed histories and generator (60 runs, fixed seed 20261110)
enter each form, each list kind, the described form, the silent case, both
locations, and the reported state.
RED (previous commit, on the tip before this rule):
cd packages/eslint-plugin-oracles && npx vitest --run tests/no-oracle-disable-oracle.property.test.js
3 of 5 fail (fixed seed 20261110, random seed, calibration "a report dropped").
First counterexample: {"location":"oracleFile","sites":[{"form":"nextLine",
"list":"oracleOne"}]}, difference "the observed value differs from the owed value".
GREEN (this commit): the same command, 5 of 5 pass. Full plugin suite: 15 files,
4296 tests pass. pnpm typecheck and pnpm format:check pass. pnpm lint:check passes
with no new warnings (the 2 warnings in packages/oracles are unchanged).
Mutant (a): the rule reads Block comments only, so line comments are missed.
Same command: 2 of 5 fail (first counterexample a nextLine form).
Mutant (b): the directive pattern drops the next-line keyword, so next-line forms
are missed. Same command: 2 of 5 fail.
Both mutants were run on this rule and the rule was restored.
Co-authored-by: Isaac <[email protected]>
…d, config, self-naming, and non-directive comments The oracle now generates the directive spellings that the rule must report and the comments it must not: quoted rule names, inline `eslint <rule>: "off"` config comments, a list that names oracle/no-oracle-disable itself, a description that names an oracle rule, a non-directive comment, JSDoc, and a string holding directive text. Its driver charges only errors that ESLint tags with a rule id to the rule; harness errors propagate. Law: an ESLint directive that disables an oracle/* rule is reported on the line it begins, however spelled; a comment that ESLint does not read as a directive is not reported. Authority: P8 of the frozen v20 design grammar (lint rules cannot be disabled inline; maintainer chose option b, a plugin rule that reports the directive). P10: a harness failure is never charged to the rule. The directive grammar is ESLint's. Reach: the constructed histories declare each form and list kind, and the counterexamples below are drawn from them. Per-state hit counts were not measured in this commit. RED (unchanged production, HEAD 65e1c85): `npx vitest --run tests/no-oracle-disable-oracle.property.test.js` in packages/eslint-plugin-oracles: 2 failed, 3 passed. Shrunk counterexamples: a `config` form (finding 2) and a double-quoted and a single-quoted `oracle/probe-form` directive (finding 1). Per-finding probe on the same production: quoted, config, and block/line self-naming directives are not reported, and they silence oracle/probe-form. A bare directive is reported. GREEN: not applicable. This commit changes no production code. The oracle stays RED until the production rule is repaired. Mutant: a substring mutant (`comment.value.includes('oracle/')`) fails the oracle on a `jsdoc` history, where HEAD production is silent. The mutant was applied in place and restored; no production file is in this commit. The old oracle passed this mutant 5/5 per the review. Limits: blanket directives (no rule list) are not generated. The report location for a self-naming directive is open: ESLint hides a report at or after the directive's start, so the rule's choice of line is a maintainer decision. Four other drivers (oracle-property-only, probe-form, fixed-seed, replay-observation) still charge all errors to the rule. The model's `isOracleRule` predicate is still a copy of production's and needs an independent judgment. Co-authored-by: Isaac <[email protected]> … first-token, covered, and config forms Extends the no-oracle-disable oracle to the review's remaining directive law: a blanket directive with no rule list is reported; a directive that names this rule, or has no list, sits at the file's first token; a directive under an earlier block directive in production source sits at the first token; an inline config comment that names an oracle rule is reported. The model names the plugin's rules itself instead of testing a prefix, and it generates a header comment before the first token. Law: every ESLint directive that would silence an oracle/* rule is reported once; the report sits on the directive's line, or at the first token when the directive names this rule, has no list, or is covered by an earlier block directive in production source. Authority: P8 of the v20 design grammar (option b); maintainer decision that blanket directives are reported wherever ESLint lets the report through, and that a self-naming directive is reported at the first token. Reach: the constructed histories include blanket line, next-line and block, a directive covered by a self-naming block, a directive covered by a blanket block, a directive under a block that names other rules, a config comment, and a header comment. Per-state counts not measured. RED (unchanged production, HEAD 10d7c1e): `npx vitest --run tests/no-oracle-disable-oracle.property.test.js` in packages/eslint-plugin-oracles: 2 failed, 3 passed. Shrunk counterexamples: a config-comment history (F2), and a block-line history with single-quoted names. GREEN: not applicable here; the production repair follows in its own commit. Limits: a directive before the first token is not generated; enable directives are not generated. Co-authored-by: Isaac <[email protected]> …acle files too The rule cannot tell whether an oracle file's noInlineConfig is set, so an earlier block directive covers a later report in both locations; the model now says so. Law unchanged otherwise. RED (unchanged production, HEAD 478d9e7): oracle, 2 failed / 3 passed. A block-covered report in an oracle file is expected at the first token and the rule places it on its own line. Co-authored-by: Isaac <[email protected]> …t at the file's start under either parser The Program node is the report location for a self-naming or blanket directive, but its start depends on the parser: typescript-eslint starts it at a header comment, espree at the first statement. The oracle now generates the parser (typescript or espree) and a header comment, and places the report at line 1, column 0: the file's start, which precedes every directive under either parser. Law: a report that cannot sit on its directive's line sits at the file's start, line 1 column 0. Authority: maintainer decision (F8, option 2). Limit recorded: a config comment that names this rule turns it off for the file, so ESLint runs no rule and none reports it. The grammar does not write that comment; the rule cannot close it, and the README and changeset record the residual. RED (unchanged production, HEAD 311ddad): oracle, 2 failed / 3 passed. Counterexample: an oracle file parsed by espree with a header comment; the rule reports line 2 where the law places the report at line 1. Co-authored-by: Isaac <[email protected]> … that would silence an oracle rule The rule now reads the directives and inline config comments through sourceCode.getDisableDirectives() and sourceCode.getInlineConfigNodes() instead of a hand-written directive regex. - F1: quoted rule names (bare, single, double) are the rule's names. - F2: an inline `eslint oracle/x: "off"` config comment is reported. - F3: a directive that names this rule, or has no list, is reported at the file's start (line 1, column 0): it hides its own report otherwise. A directive or config comment under an earlier block directive is reported at the file's start as well. - F10: a blanket directive with no rule list is reported, in every form. A file-wide block directive is reported too, when code precedes it. - F8 (rule side): the report is at an explicit file-start location, so it does not depend on whether typescript-eslint or espree gives the Program node its start. Law: every ESLint directive that would silence an oracle/* rule is reported once; the report sits on the directive's line, or at the file's start when the directive names this rule, has no list, or lies under an earlier block directive. Authority: P8 (option b), maintainer decisions on blanket and self-naming directives and on the file-start location. Reach: the constructed histories cover blanket line, next-line and block, self-naming block and line, covered by a self-naming block, covered by a blanket block, under a block naming other rules, config, header comment, espree and typescript parsers. Per-state counts not measured. RED (oracle at cdfdd1e, rule at 65e1c85, unchanged by the oracle commits): oracle 2 failed / 3 passed before this commit (blanket, covered and parser histories). Probe on the unchanged rule: quoted, config, and self-naming directives are not reported and silence oracle/probe-form. GREEN (this commit): oracle 5/5 passed. Plugin suite: 15 files, 4296 tests passed. tsc, eslint, prettier clean on the changed files. Mutant (each applied to the rule, run through the oracle, restored): - quotes not stripped: killed (2 failed) - no covered relocation: killed (2 failed) - self-naming not relocated: killed (2 failed) - blanket not reported: killed (2 failed) - description substring: killed (2 failed) - config ignored: killed (2 failed) - Program-node location: killed (2 failed) Limits: a self-naming or blanket directive on line 1 covers the file's start, so ESLint hides its report and no location exists. A production config comment that turns this rule off silences it for the file; no rule can report that. Directives that cover later lines through enable are not generated. Co-authored-by: Isaac <[email protected]> … importOf and report copies - S-1: importedName already yields undefined for default and namespace specifiers, so the ImportSpecifier test is removed. - S-2: rawCampaignOf returns the report's messageId and data, and the handler spreads it instead of copying field by field. - S-5: malformedList has no placeholders, so it takes no data object. Behavior-preserving: same reports, same messageIds, same data. GREEN: plugin suite 15 files, 4296 tests passed. Co-authored-by: Isaac <[email protected]> …e header location and non-array module settings Extends the never-throws and location grammar: - parser (typescript or espree) and a leading header comment, so the Program node's start differs between parsers (F8): a report at line 1 must sit at the file's start under either; - implementationModules as a non-array value, which the rule must report and never throw on (F6); a non-array value is read as an empty list, so the implementation exemption is lost. Narrows the F9 law: a refused entry names another file or an implementation module, and no legal history lets it exempt the file under test, so the "exempts nothing" half is not observable and is said so in Limits. Law: the rule never throws on a program that parses; a configuration that it cannot judge is reported at line 1 of the file, whatever the parser. Authority: P8, and the maintainer's option 2 for the report location (F8). RED (unchanged production, HEAD 08b6887): oracle 2 failed / 3 passed. - espree with a header comment: the rule reports line 2 where the law places the report at line 1. - implementationModules as a string: the rule throws a TypeError (modules.some is not a function), which the oracle's check counts as a violation. Co-authored-by: Isaac <[email protected]> …d as an empty list The model judges an implementationModule entry as refused only while that list is an array. A non-array value is read as an empty list (the law written with the rule), so the entry is judged instead and is not refused. RED: none (model correction); the implementation change follows in the next commit. Plugin suite on this model alone: one failed (the history above). Co-authored-by: Isaac <[email protected]> …dule settings and places its reports at the file start - F6: a non-array settings.oracles.implementationModules (or replayWitnessModules) is read as an empty list and reported at the file start as malformedSetting. The rule no longer throws on it. - F8: the configuration and stale/without-reason reports sit at an explicit file-start location (line 1, column 0), not the Program node, whose start depends on the parser (typescript-eslint starts it at a leading comment, espree at the first statement). Law: the rule never throws on a program that parses; a configuration it cannot judge is reported at line 1 of the file, whatever the parser. Authority: P8, and the maintainer's option 2 for the location (F8). RED (oracle at 5cf1450 + model fix 75b9201-parent, rule at HEAD before this commit): oracle 2 failed / 3 passed: espree with a header comment reports line 2 where the law places line 1; a string implementationModules throws. GREEN: oracle 5/5 passed; plugin suite 15 files, 4296 tests passed; tsc and eslint clean. Mutant (applied, oracle run, restored): program-node location: killed (2 failed); malformed setting silent: killed (2 failed); stale at program node: killed (1 failed); setting still throws (no normalization): killed (2 failed). Not done in this commit: no-disabled and replayable-seed read the same settings and now normalize them (no throw), but do not report the malformedSetting yet. Their oracles do not generate a non-array setting, so that report is unproved and stays open. Co-authored-by: Isaac <[email protected]> …rule threw on
The two report calibrations read observed.value as the rule's report list.
When the rule throws, the observation is { threw } and has no length, so the
calibration called .map on an object and the harness raised a TypeError
instead of the law's violation (M1, verified in this run).
RED: a probe-form mutant that throws in Program:exit gives
"TypeError: o.value.map is not a function" and "o.value.slice is not a
function" from the calibrations (7 failed).
GREEN: the same mutant now fails only on the never-throws law (no TypeError,
7 failed); the clean run passes 9/9.
Co-authored-by: Isaac <[email protected]>
… and the file-start location - README and changeset describe every directive the rule reports: quoted names, blanket directives, inline config comments naming an oracle rule. - The report location: the directive's line, or line 1 column 0 when the directive names this rule, has no list, or is under an earlier block directive (F3, F8). - Residuals the rule cannot close: a directive on line 1 that names this rule or has no list covers the file's start; a production config comment that turns this rule off silences it for the file (F10). - noInlineConfig in oracle files ignores every rule's directives, not only the oracle rules, and the changeset says so as a behavior change (F5). - A non-array implementationModules or replayWitnessModules is reported at line 1 as malformedSetting (F6). - The probe-form production globs move into their own paragraph (S-11), and the long README paragraph is rewrapped (S-13). Co-authored-by: Isaac <[email protected]> …campaign message tail (S-3) rawCampaign and rawProperty differ only in their first clause, so the shared advice is one constant. Message ids and the rendered text are unchanged. GREEN: plugin suite 15 files, 4296 tests passed; prettier clean. Co-authored-by: Isaac <[email protected]> …oracle drivers (F7, S-14, S-15)
Each driver wrapped its lint run in a try that recorded every error as the
rule's throw. Only an error that ESLint tags with a rule id is the rule's
throw; any other error is a harness failure, which must propagate (P10). The
shared helper tests/lint-observation.js returns { messages } or { threw },
charging only the tagged error. The five drivers use it:
probe-form, fixed-seed, replay-observation, oracle-property-only, and
no-oracle-disable (which already had the same logic inline).
Authority: P10 and decision 13 ("a harness failure stops the run with one
diagnostic and is never charged").
RED (HEAD driver, probe-form test, parser replaced by an invalid value): the
harness error is recorded as "threw: Key parser: Expected object ..." and the
rule is blamed (7 failed, each a census violation).
GREEN (new driver, same injection): the TypeError propagates and the run stops
(9 failed, all with the harness error).
Limit: the threw text changes from error.message to String(error), so the
observation carries the error's name too. The suite is green on this change
(15 files, 4296 tests).
Co-authored-by: Isaac <[email protected]>
…er (S-8) Module evaluation has no side effects, so the order is cosmetic. The import now sits in the same order as the rules object. GREEN: plugin suite 15 files, 4296 tests passed. Co-authored-by: Isaac <[email protected]>
All reactions
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page.
Click here to learn what changesets are, and how to add one. Click here if you're a maintainer who wants to add a changeset to this PR All reactions
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page.
All reactions
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. Uh oh!
There was an error while loading. Please reload this page.
KyleAMathews
added a commit
that referenced
this pull request
#6 made `recommended` turn off inline directives and widened the `production` glob to `src/**`, so setup now adds both presets as they are, without the workaround blocks. The plugin README lists no-oracle-disable and the new glob. The design skill now asks for at least one fixed seed (an empty list will be refused) and treats a random-only kill as a gap until its history is kept. Co-authored-by: Isaac <[email protected]>
KyleAMathews
added a commit
that referenced
this pull request
…#13) * fix(eslint-plugin-oracles): load and parse with the consumer's parser `oracle/no-unit-test` imported `@typescript-eslint/parser` at module load. That parser refuses TypeScript 7, so importing the plugin, and every one of its configs, threw in a repository that pins TypeScript 7.0.2 (field report round 3, nanostores). It also threw when `typescript` was absent. The rule now parses each imported test module with the parser and parser options that ESLint uses for the file it lints, through ESLint's own Linter, so any configured parser works, espree included. The parse cache is keyed on those language options too. typescript-eslint's type-aware options are left out of that parse: the rule reads syntax and scope only, and a nested `project` parse changed the program that type-aware rules then read (single- run mode). `@typescript-eslint/parser` moves to devDependencies. Also fixed, found by the extended oracle: under espree the rule dropped every reference (espree scopes have no `isValueReference`) and reported reaching tests as undeclared. And a module that did not parse was skipped silently: when a test reaches no part, each import of a module that the file's parser cannot read is now reported (unparsedModule) with the parser's error. The rule never throws on it. Evidence - Law: the plugin loads whatever the consumer's TypeScript (field report); rules never throw (maintainer decision 2026-11-08, #6); an imported module is parsed as ESLint parses the linted file (consumer-parser task). The narrowing of the report to files with an unreached test, and the strip of type-aware options, need maintainer confirmation. - Reach: new lint.part-reach.parsers family gates espree-oracle, espree-oracle-test-uses-part, oracle-imports-type-syntax-under-espree (fixed seed 82, 24, 56 of 103); unparseable-companion gates oracle-imports-unparseable-companion (20); new parser-switch law gates the type-syntax state (22). The TS environment is a focused test (consumer-typescript.test.js, 3 child-process cases). - RED (d1b508a rule): 13 of 84 fail. TS 7 stub: "typescript-eslint does not support TS 7.0."; absent: "Cannot find module 'typescript'"; espree: "unit:t0" read as "undeclared:t0"; broken import: "unparsed:broken.ts" missing. - GREEN: plugin suite 17 files, 4310 tests; typecheck, eslint, prettier, sherif, publint clean. A copy of the nanostores repro loads the plugin and lints an espree-readable oracle file. - Mutant: top-level parser import (consumer-typescript), fixed espree or fixed typescript-eslint (parsers, parser-switch), a throwing parse error (charged as threw:), the old isValueReference test (parsers), a cache that ignores the parser (parser-switch), kept type-aware options (typed-lint-isolation), eager reports (parsers, unparseable-companion). - Limits: the linted file's options, not the imported module's own, so a JS oracle importing a TS module with type syntax gets the report when a test is unreached; transitive unparseable modules are not reported; two parsers; a stub of TS 7's shape; ESLint 10 not run. Under TS 7, typescript-eslint itself cannot load, so .ts oracle files still cannot be linted there. no-unit-test.js grows 269 bytes min+gzip; the install drops the @typescript-eslint/parser tree. Co-authored-by: Isaac <[email protected]> * fix(eslint-plugin-oracles): report only parse failures as unparsed; ignore inline config in imported modules Review round on the consumer-parser change: - Only reading and parsing an imported module is caught. An error in the rule's own analysis of a parsed module is a rule bug and is no longer reported as a parse failure. - The nested Linter ignores the imported module's eslint comments (noInlineConfig), so they run no core rules and print no env warnings. - The changeset states the two behaviors to expect: a module is parsed with the linted file's parser, and only direct imports report a parse failure. Co-authored-by: Isaac <[email protected]> --------- Co-authored-by: Isaac <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ReviewersNo reviewsUh oh!
There was an error while loading. Please reload this page. AssigneesNo one assignedLabels
None yet
ProjectsMilestoneNo milestoneDevelopmentSuccessfully merging this pull request may close these issues. Uh oh!
There was an error while loading. Please reload this page. Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later. | |||||||
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |