GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [SQL] Compute using UTF-8 strings; support `STARTSWITH` and `ENDSWITH` by mihaibudiu · Pull Request #7729 · feldera/feldera

Conversation

@mihaibudiu

@mihaibudiu mihaibudiu commented Oct 30, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There are 3 commits:

  • first commit adds a new class Utf8String which wraps a string literal coming from a SQL program. This prevents accidentally computing on such strings using Java functions, which use UTF-16.
  • second commit hides string literals from Calcite, by wrapping a literal X into a function NON_BMP_LITERAL(X), to prevent Calcite from doing any computations with it
  • third commit solves the issue we started with, i.e., add two puny functions. Writing tests for these functions uncovered the need for the other two commits

Linked Issue

Fixes #7714

Checklist

  • Unit tests added/updated
  • Documentation updated

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
gz
gz approved these changes Oct 30, 2026
View reviewed changes
Comment thread ...ompiler/SQL-compiler/src/test/java/org/dbsp/sqlCompiler/compiler/sql/simple/StringTests.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread sql-to-dbsp-compiler/SQL-compiler/src/main/java/org/dbsp/util/Utf8String.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread ...-dbsp-compiler/SQL-compiler/src/main/java/org/dbsp/sqlCompiler/compiler/ProgramMetadata.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
mythical-fred-oss[bot]
mythical-fred-oss Bot reviewed Oct 30, 2026
View reviewed changes
Comment thread ...SQL-compiler/src/main/java/org/dbsp/sqlCompiler/ir/expression/literal/DBSPStringLiteral.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
mythical-fred-oss[bot]
mythical-fred-oss Bot reviewed Oct 30, 2026
View reviewed changes
Comment thread ...ler/src/main/java/org/dbsp/sqlCompiler/compiler/frontend/calciteCompiler/NonBmpLiterals.java Resolved

Uh oh!

There was an error while loading. Please reload this page.
@mythical-fred-oss

mythical-fred-oss Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Summary: the SQL compiler now stores string literals as UTF-8, string literals with non-BMP characters are hidden from Calcite's constant folding, and STARTSWITH/ENDSWITH are added for strings and binary values. The docs are updated and the tests are thorough.

What I ran

Check Result
cargo check --locked -p feldera-sqllib ✅
./build.sh ❌ the slt module no longer compiles (see inline comment)
mvn test -Dtest=StringTests,CastTests,FunctionsTest#testStartsEndsWith+testPositionNullLiteral ✅ 40/40
My own QA cases: non-BMP literal in a SQL UDF body, IN ('🐍','a') with GROUP BY '🐍', LIKE … ESCAPE with a non-BMP literal, CHAR padding/CASE with BMP vs non-BMP literals ✅ the optimized and runtime results agree

Gates

  • bug, critical: the build is broken because the slt module still uses the removed DBSPStringLiteral constructor. Please fix it before merge.
  • missing, medium (commit quality): the first commit, [SQL] Represent string literal values as UTF-8, rewrites how literals are represented and how casts are folded, but its body is only the sign-off. Please explain why the change is needed, for example that Java computes on UTF-16, which led to wrong constant folding.

Uncovered cases

  • Only views go through NonBmpLiterals, via sqlToRel. Column DEFAULT expressions, LATENESS, and SET values skip it. No test covers a non-BMP literal there, for example DEFAULT '🐍x' on a VARCHAR(1) column.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
mihaibudiu added 3 commits October 30, 2026 14:45
@mihaibudiu
[SQL] Represent string literal values as UTF-8
15a5638 
Signed-off-by: Mihai Budiu <[email protected]>
@mihaibudiu
[SQL] Hide from Calcite the string literals with characters above U+FFFF
6772dd7 
Such a literal is now wrapped in a call that Calcite cannot fold, typed
VARCHAR.  A literal with an unpaired surrogate, such as U&'\D83D', is an
error.  CHR is not folded by Calcite, since it can build such a character.

Signed-off-by: Mihai Budiu <[email protected]>
@mihaibudiu
[SQL] Add the STARTSWITH and ENDSWITH functions
b168073 
Fixes #7714

Signed-off-by: Mihai Budiu <[email protected]>
@mihaibudiu
mihaibudiu force-pushed the issue7714 branch from 633b957 to b168073 Compare October 30, 2026 21:47
@mihaibudiu
mihaibudiu added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 111d1cd Oct 31, 2026
1 check passed

Uh oh!

There was an error while loading. Please reload this page.
@mihaibudiu
mihaibudiu deleted the issue7714 branch October 31, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mythical-fred-oss mythical-fred-oss[bot] mythical-fred-oss[bot] left review comments
@gz gz gz approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[SQL] Add functions startswith and endswith for string processing

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@mihaibudiu @gz
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.