GeezerVison is showing github.com — Let a dialect own its reserved keywords, starting with DuckDB by guizmaii · Pull Request #2787 · JSQLParser/JSqlParser
Conversation
guizmaii
commented
Oct 13, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Purpose
Implements #2786: a dialect owns its reserved keywords, in both directions, starting with DuckDB.
Consumer<CCJSqlParser> duckDb = parser -> parser.withDialect(Dialect.DUCKDB);
CCJSqlParserUtil.parse("SELECT settings, file FROM script s WHERE s.top > 1", duckDb); // accepted, rejected without the dialect
CCJSqlParserUtil.parse("SELECT a AS from, t.where FROM t", duckDb); // any keyword after AS or "."
CCJSqlParserUtil.parse("SELECT desc FROM t", duckDb); // rejected: DESC is reserved in DuckDB
CCJSqlParserUtil.parse("SELECT 1 name", duckDb); // rejected: a bare alias cannot be a keywordModel
KeywordPolicy#isName(word, NamePosition): whether a word is an unquoted name at a position: NAME (column, table, CTE, ...), FUNCTION, LABEL (after AS or .), BARE_ALIAS.Dialect#getKeywordPolicy(): null for the default rules (nothing changes without a policy).Dialect.DUCKDB (dollar-quoted string tags, adjacent string literals on a new line) uses DuckDbKeywordPolicy: DuckDB 1.5.5's 489 keywords with their categories, from duckdb_keywords(), and the PostgreSQL rules DuckDB applies to them. Function names only reject reserved keywords: DuckDB's grammar special-cases functions named by other keywords (COALESCE, NULLIF, IF, ...).
Grammar
RelObjectName() rejects the names the policy rejects at that position (position from the previous token: AS/., or a following ().DialectUnreservedWord() accepts the keywords reserved by default that the policy accepts as names; DialectLabel() accepts any keyword after AS or ..- A select item's alias without
AS follows the BARE_ALIAS rule; other aliases follow NAME.
- Constructs led by such words step aside when the word is a name: the
XOR operator (grammar and Pratt loop), INCLUDES/EXCLUDES, EXISTS, CONNECT_BY_ROOT, PRIOR, XMLSERIALIZE, window-frame BETWEEN/UNBOUNDED, CONNECT BY [NOCYCLE], with explicit lookaheads: checkGrammarAmbiguity passes.
Validation
- Java 17 Gradle
check: 13,489 tests, zero failures/errors; Spotless, Checkstyle, PMD, SpotBugs and checkGrammarAmbiguity pass. All existing tests are unchanged, apart from TaggedDollarStringTest, which now treats every dialect enabling dollar-quoted string tags (PostgreSQL and DuckDB) alike.
DuckDbKeywordPolicyTest compares, for each of DuckDB's 489 keywords, 7 name positions (column, qualified column, AS alias, bare alias, table, table alias, CTE) against what DuckDB 1.5.5 itself parses (duckdb-keyword-names.csv, generated with json_serialize_sql()): 3,391 cells agree. 32 cells are exempted, each documented: JSqlParser parses the word as dedicated syntax at that position (MAP {...}, STRUCT(...), TRY_CAST(...), COLUMNS(...), DEFAULT, WITH RECURSIVE, UPDATE ... SET, ...) or SELECT 1 day as a date unit.- 505 production DuckDB queries parse with and without
Dialect.DUCKDB.
Limits
- MySQL's SELECT modifiers (
DISTINCTROW, STRAIGHT_JOIN, SQL_CACHE, ...) cannot be names under a policy: Select()'s LOOKAHEAD(3) into PlainSelect() takes them as modifiers first.
- Other dialects keep the default rules; their policies can follow the same model.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
guizmaii
changed the title
Let a dialect declare which reserved keywords it accepts as names, starting with DuckDB
Let a dialect own its reserved keywords, starting with DuckDB
Oct 13, 2026
|
Thank you for your work and effort, but I am not a big fan of it:
- SQL:2016 is very broad on reserved keywords
- It is always good to write portable SQL and quote identifiers (as well as RDBMS specific constructs)
- I have performance concerns
- I have ongoing maintenance concerns
- Our current list of reserved keywords is very very short already
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
guizmaii
commented
Oct 13, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
Thanks for looking at it @manticore-projects. To clarify the problem this addresses:
Recent PRs added dialect-specific keywords to the reserved block for every dialect: ClickHouse's SETTINGS/PREWHERE, Exasol's CSV/FILE/SCRIPT/FBV/VERIFY, ... (and ERRORS, fixed in #2781). None of these is reserved in SQL:2016, so a query using settings or file as a column name is portable SQL: it parsed with 5.3 and is rejected by 5.4. The reserved list grew from words other dialects needed, not from the standard.
So this isn't about accepting non-portable SQL, but about one dialect's keywords not leaking into the others.
If the per-dialect model is too much, a smaller fix would be to make those dialect-only words non-reserved again, behind lookaheads where their clause starts, as #2781 did for ERRORS. Would you accept that instead?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reserved keywords are global: a keyword reserved for one dialect's syntax (ClickHouse's
PREWHERE and SETTINGS, Exasol's FILE and SCRIPT, MySQL's XOR, ...) is rejected as a name
in every dialect, and words a dialect reserves (DuckDB's CASE, TABLE, DESC, ...) are
accepted as names whatever the dialect.
- `KeywordPolicy`: a dialect's own rules, whether a word is an unquoted name in a
`NamePosition` (an object name, a function name, a label after AS or ".", a bare
alias). `Dialect#getKeywordPolicy()` is null for the default rules.
- `Dialect.DUCKDB` (dollar-quoted string tags, adjacent string literals on a new line)
uses `DuckDbKeywordPolicy`: DuckDB 1.5.5's 489 keywords with their categories, from
`duckdb_keywords()`, and PostgreSQL's rules for them. Function names only reject
reserved keywords, as DuckDB special-cases COALESCE, NULLIF, IF, ...
- The grammar consults the policy in both directions: `RelObjectName()` rejects the
names it rejects and `DialectUnreservedWord()` accepts the reserved-by-default keywords
it accepts; `DialectLabel()` accepts any keyword after AS or "."; a select item's bare
alias follows the bare-alias rule; the XOR operator, INCLUDES/EXCLUDES, EXISTS,
CONNECT_BY_ROOT, PRIOR, XMLSERIALIZE and the window-frame BETWEEN/UNBOUNDED step aside
when the word is a name, with explicit lookaheads (no new choice conflict).
- Without a policy nothing changes.
- `DuckDbKeywordPolicyTest` checks each DuckDB keyword in 7 name positions against what
DuckDB 1.5.5 parses (`duckdb-keyword-names.csv`), except 32 documented cells where
JSqlParser parses the word as dedicated syntax (MAP, STRUCT, TRY_CAST, COLUMNS, WITH
RECURSIVE, ...) or as a date unit (`SELECT 1 day`).
- `TaggedDollarStringTest` treats every dialect enabling dollar-quoted string tags, now
including DuckDB, like PostgreSQL.
If the per-dialect model is too much, a smaller fix would be to make those dialect-only words non-reserved again, behind lookaheads where their clause starts, as #2781 did for ERRORS. Would you accept that instead?
I would not object that at all: exotic and scarcely used features like this Exasol stuff can take longer to parse -- as long as it does not affect the performance of the regular Oracle/Postgres/MSQL/MariaDB queries.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
To make the point concrete, on current master:
| Statement |
no dialect |
Dialect.EXASOL |
IMPORT INTO t FROM LOCAL CSV FILE 'f.csv' |
fails |
parses |
EXPORT t INTO LOCAL CSV FILE 'f.csv' |
fails |
parses |
SELECT csv, file, script, verify, fbv FROM t |
fails |
fails |
The Exasol IMPORT/EXPORT statements are already gated behind Dialect.EXASOL (LOOKAHEAD({ Dialect.EXASOL... }) ( stm = Import() | stm = Export() )), but the keywords they introduced are reserved for every dialect. So anyone parsing PostgreSQL, DuckDB or plain SQL with a csv or file column gets rejected because of a syntax they can't even use without opting into Exasol.
That's the inconsistency I'd like to fix: keywords introduced for a dialect-gated syntax should only be reserved when that dialect is on.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
To make the point concrete, on current master:
Statement no dialect Dialect.EXASOL
IMPORT INTO t FROM LOCAL CSV FILE 'f.csv' fails parses
EXPORT t INTO LOCAL CSV FILE 'f.csv' fails parses
SELECT csv, file, script, verify, fbv FROM t fails fails
The Exasol IMPORT/EXPORT statements are already gated behind Dialect.EXASOL (LOOKAHEAD({ Dialect.EXASOL... }) ( stm = Import() | stm = Export() )), but the keywords they introduced are reserved for every dialect. So anyone parsing PostgreSQL, DuckDB or plain SQL with a csv or file column gets rejected because of a syntax they can't even use without opting into Exasol.
That's the inconsistency I'd like to fix: keywords introduced for a dialect-gated syntax should only be reserved when that dialect is on.
Well understood, I only object separate keyword lists (because it will be a nightmare to maintain).
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Opened #2793 with the smaller fix you suggested: the Exasol/ClickHouse clause words become non-reserved, with no separate keyword lists, and parse performance stays the same.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|