GeezerVison is showing github.com — [SQL] Allow lambda functions to refer to columns of the current relation by mihaibudiu · Pull Request #7943 · feldera/feldera
Conversation
mihaibudiu
commented
Nov 12, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Previously you could write x -> x + 2. Now you can write x -> T.col + x as well, where T.col is a column of table T.
This makes lambdas significantly more powerful -- they become closures, not just pure lambdas.
Linked Issue
Fixes #6651.
Marking as draft, currently blocked on apache/calcite#5409.
Checklist
- Unit tests added/updated
- Documentation updated
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
|
@Override
|
|
|
public boolean allowLambdaClosure() {
|
|
|
return false;
|
|
|
return true;
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bug (critical, merge gate): with the Calcite commit currently pinned in calcite_version.env (4bf922b), turning this on gives silently wrong results. testCorrelatedSubquerySameType returns (1, 1), (7, 1) instead of (1, 2), (7, 0): T.x resolves to S.y. testCorrelatedSubquery also hits AssertionError: type mismatch INTEGER vs INTEGER ARRAY, and the subquery-in-lambda check in testBodyRestrictions does not fire. Please bump calcite_version.env to a commit that includes apache/calcite#5409 in this PR, so this flag can't land without the fix.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
What I ran: ./build.sh (passes), then mvn test -Dtest='LambdaClosureTests,ArrayFunctionsTests' against the pinned Calcite. Result: 70 tests, 3 errors, all in LambdaClosureTests (testCorrelatedSubquery, testCorrelatedSubquerySameType returns wrong rows, testBodyRestrictions). They depend on the unmerged Calcite change; see the inline comment. ArrayFunctionsTests passes. I also ran some extra QA cases:
| Case |
Result |
GROUP BY TRANSFORM(a, e -> e + x) |
✅ |
Captured nullable VARCHAR column (e || s, s NULL) |
✅ |
Captured struct field (p).v |
✅ |
HAVING ARRAY_EXISTS(ARRAY_AGG(x), e -> e = x + 0) |
❌ type mismatch INTEGER vs INTEGER ARRAY (same failure as the Calcite one) |
SUM(ts) OVER (PARTITION BY ARRAY_EXISTS(a, e -> e = x) ORDER BY ts) |
❌ Function 'COUNT' not yet implemented |
Same query with ARRAY_CONTAINS(a, x), no lambda |
✅ |
- bug (medium): a lambda that captures a column in a window
PARTITION BY/ORDER BY leaves a RexOver inside a LogicalProject, and the user gets a misleading "COUNT not yet implemented" error. Please either support it or reject it with a clear message, and document it next to the other restrictions in grammar.md.
- missing (medium): no tests cover window clauses or
HAVING with captured columns. Suggested additions to LambdaClosureTests:
@Test
public void testHaving() {
var ccs = this.getCCS(TABLES + """
CREATE VIEW V AS SELECT x FROM T GROUP BY x
HAVING ARRAY_EXISTS(ARRAY_AGG(x), e -> e = x + 0);""");
ccs.stepWeightOne(DATA, " x\n---\n 10\n 2");
}
@Test
public void testWindowPartition() {
var ccs = this.getCCS("""
CREATE TABLE T(a INT ARRAY, x INT, ts INT);
CREATE VIEW V AS SELECT x, SUM(ts) OVER
(PARTITION BY ARRAY_EXISTS(a, e -> e = x) ORDER BY ts) AS s FROM T;""");
ccs.stepWeightOne("INSERT INTO T VALUES(ARRAY[1], 1, 1), (ARRAY[3], 2, 2), (ARRAY[3], 3, 3);",
" x | s\n-------\n 1 | 1\n 2 | 2\n 3 | 4");
}
Docs and tests are otherwise in place. Not mergeable until the Calcite pin includes apache/calcite#5409.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This branch has not been deployed
No deployments
Reviewers
At least 1 approving review is required to merge this pull request.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|