GeezerVison

AI disclosure: this change was prepared with AI coding agents.

What

WITH items accept ClickHouse's WITH <expression> AS <identifier> next to classic CTEs and WITH FUNCTION: scalar, column, and compound expressions, the three lambda shapes ClickHouse accepts (x -> e, (x, y) -> e, (x -> e)), and the reversed-CTE reading (SELECT ...) AS name. Parsing reuses the existing LambdaExpression grammar; the whole-lambda-in-parentheses form is modeled as a ParenthesedExpressionList, matching how (x -> e) already parses in select-item position. WithItem exposes getExpression().

Why

The query from #2632 fails at the first WITH item on master, and with it every documented WITH <expr> AS name form, including the lambda examples in the ClickHouse documentation.

How

1. WithItem() gains a third alternative WithItemExpression(), following the WithFunctionDeclaration precedent (Fix[2306] - Adds support for Trino UDF #2307). Lambda bodies reuse the existing LambdaExpression production.
2. isWithExpressionAliasAhead() distinguishes expression aliases from the CTE shape name [(cols)] AS [NOT] MATERIALIZED (statement). It walks the optional bracket group through the token chain until the matching closing bracket or EOF, and aborts on parser interruption. There is no token-count limit. Qualified names go directly to the expression branch because a CTE name is a single identifier token.
3. isImplicitCastAhead() no longer treats every DATA_TYPE / DATA_TYPE(N) as a typed-literal prefix: it starts one only when a trailing literal completes it, otherwise the tokens parse as an identifier or function call. INT '5', DECIMAL(10,2) '1.5', DOUBLE PRECISION '1' keep their cast reading. This second layer is needed for the issue's own SELECT.
4. The expression branch is wired through deparser (DmlDeParserSupport), TablesNamesFinder, SelectValidator, StatementFeatureVisitor, and the generic SelectVisitorAdapter traversal (subquery aliases stay on the select path, other expressions go to the expression visitor). SEARCH/CYCLE suffixes accepted by the tolerant grammar serialize on the expression branch too.

Root cause

Two layers. First, WithItem() only had the CTE and WITH FUNCTION forms; the whole WITH <expr> AS <ident> family was missing (verified against docker ClickHouse server 26.8.2.7: 16 shapes legal, including the undocumented bare x -> x*2 and chained total * 2 AS doubled). Second, isImplicitCastAhead() classified every DATA_TYPE head as a typed-literal prefix, sending SELECT double(5), SELECT number FROM t, and sum(number) into a branch that requires a following literal.

Testing

• Red to green on master dec8f5d: the issue query and SELECT double(5) fail there and pass with this change; classic CTE and typed-literal guards remain unchanged. Further guards cover nested WITH, INSERT, mixed expression aliases and CTEs, MATERIALIZED and invalid input.
• The review follow-up adds 13 regression cases for bracket groups at 4095, 4096 and 4097 tokens and beyond, large CTE column lists, nested calls, qualified function names, incomplete input and cancellation. Over-limit expressions and cancellation checks fail on the reviewed revision 06f1556 and pass with the follow-up. A mutation that always chooses the expression branch is rejected by the CTE guard.
• JDK 17: full mvn clean verify (7312 tests, 25 skipped) and gradlew check (7330 tests, 25 skipped) passed with the scanner fix. After incorporating the separate visitor-assertion follow-up, all 64 ClickHouse tests passed again with both build tools.
• Project JMH JSQLParserBenchmark.parseSQLStatements, standard 54-statement corpus, version=latest, one thread, 3 forks, 2 x 5-second warmups and 5 x 1-second measurements per fork: master/fixed/master measured 78.1 ± 25.6 / 57.1 ± 3.4 / 55.6 ± 2.2 ms/op (99.9% confidence intervals, 15 samples each). The intervals overlap; this benchmark shows no clear regression. Both versions produce the same serialized statements.
• The same benchmark and settings on a single 2047-argument WITH function (4095 tokens in its bracket group, accepted by both versions) measured 111.3 ± 25.6 ms/op at 06f1556 and 12.3 ± 2.8 ms/op with the fix. The parsed output matches. This improvement is specific to that long-expression input.

Behavior notes

• Tolerated shapes that now parse where master errored, each deparsing and re-parsing stably: WITH (a, b) AS c, WITH t(a, b) AS u, WITH ? AS p, WITH TRUE AS flag, WITH int(5) AS f, WITH number AS n, WITH (x, y) -> x + y AS m.
• In an alias position, PG's data -> 'x' reads as a lambda; master rejected both readings there. ->>/#> are unaffected.
• Not implemented: ClickHouse's trailing comma after the last WITH item. Single-parameter lambdas deparse without their parameter parentheses; the existing LambdaExpression.toString() convention, shared with function-argument lambdas.

Verification of the original issue

On master dec8f5d, WITH (x -> x * 2) AS double SELECT double(5) fails to parse. With this change the WITH item carries the lambda (ParenthesedExpressionList wrapping a LambdaExpression) with alias double, the select item parses as the function call double(5), and the statement round-trips. All ClickHouse syntax evidence comes from docker clickhouse-server 26.8.2.7; no claim is made about running the reporter's JSqlParser version.

Fixes #2632