GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Add support for Apache Cloudberry by roseduan · Pull Request #1365 · sqlancer/sqlancer

Conversation

roseduan commented Sep 18, 2026

Copy link

Copy Markdown

Contributor

Apache Cloudberry is a PostgreSQL-derived MPP database, so the provider extends PostgresProvider and reuses PostgreSQL's expression generators and oracles. Cloudberry is a transparent MPP system: a single connection to the coordinator is enough, so there is no worker-node registration and no extension to create. createDatabase is inherited, wrapped only in a retry for an intermittent MPP-specific failure (see the comment on the override).

The Cloudberry-specific parts are the distribution policy, assigned per table via ALTER TABLE ... SET DISTRIBUTED with a distribution key chosen to be a subset of any PRIMARY KEY or UNIQUE constraint, and leaf partitions for the declaratively partitioned tables the inherited generator emits. Storage formats (heap, ao_row, ao_column, pax) are exercised through the inherited USING clause.

CloudberryCommon collects the errors Cloudberry returns for statements that are valid in PostgreSQL but violate an MPP restriction. CloudberryBugs follows the TiDBBugs pattern, gating each work-around for an unfixed target-DB bug behind a flag so it is easy to find and remove once the bug is fixed; each flag's comment records the scope that was actually verified against a live cluster.

Two inherited actions are not generated. CREATE STATISTICS can crash the coordinator, which a whitelisted error message cannot handle. CREATE VIEW builds its body with PostgresRandomQueryGenerator, which readily emits DISTINCT ON or LIMIT with no matching ORDER BY; such a view is re-evaluated per reference, and on an MPP cluster the row it yields can legitimately differ between evaluations, which breaks every oracle that compares two evaluations of the same data.

This PR covers database generation plus the three oracles that need no Cloudberry-specific oracle implementation: NoREC, TLPWhere and the fuzzer. The remaining oracles, and a CI job, follow in separate PRs.

Verified against a 3-segment cluster built from Cloudberry main (3.0.0-devel): mvn verify -DskipTests=true is clean, and NoREC, TLPWhere and the fuzzer each ran 4 threads x 140-200s across seeds 0-10 with no thread deaths and no oracle reports.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

mrigger commented Sep 20, 2026

Copy link

Copy Markdown

Contributor

@roseduan, thanks a lot for the PR. This looks great! I'm currently on vacation until September 27. I'll try to take a look asap.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

roseduan commented Sep 29, 2026

Copy link

Copy Markdown

Contributor Author

@roseduan, thanks a lot for the PR. This looks great! I'm currently on vacation until September 27. I'll try to take a look asap.

OK thanks a lot, I will give the continuous support if you have any comments about this issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mrigger reviewed

Oct 1, 2026

View reviewed changes

mrigger left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR! This looks great! Sorry also that it took so long to review it. I first did not see the PR, was on a short vacation, and then catching up on other work. I have some minor nitpicks that I added as comments; it's up to you which ones you want to address.

What would be great, though, is if you could update the GitHub actions workflow to also execute Cloudberry's tests in the CI. This makes sure that we won't break anything (e.g., during a refactoring).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

src/sqlancer/cloudberry/CloudberryProvider.java Outdated

		}

		private static boolean isHashDistributable(String dataType) {
		// Types without a default hash operator class cannot be a Cloudberry distribution key.

mrigger Oct 1, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's fine, but it would be more future-proof to encode this in an enum and make it fail when encountering an unknown (potentially new) data type.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

src/sqlancer/cloudberry/CloudberryBugs.java

		package sqlancer.cloudberry;

		// do not make the fields final to avoid warnings
		public final class CloudberryBugs {

mrigger Oct 1, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For these, would it be possible to add a URL? This would make it easy for others or coding agents to disable them after the issue has been addressed.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

src/sqlancer/cloudberry/gen/CloudberryCommon.java Outdated

		// because the partitions would be left carrying a constraint their parent no longer has.
		errors.add("cannot remove constraint from only the partitioned table when partitions exist");

		// See CloudberryBugs for details on each of these known, still-open target-DB bugs. Each is gated behind

mrigger Oct 1, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess some of these AI-generated comments could be removed, but I'm also fine with leaving them in.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

src/sqlancer/cloudberry/gen/CloudberryCommon.java

		errors.add("could not devise a query plan");
		// Live wording is "could not devise a plan (cdbpath.c:...)" -- no "query" -- so the entry above never
		// actually matches; keep both since it's unclear which build/error-path produces which variant.
		errors.add("could not devise a plan");

mrigger Oct 1, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the "could not devise a plan" hint towards a bug?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

src/sqlancer/cloudberry/gen/CloudberryCommon.java

		private CloudberryCommon() {
		}

		public static List<String> getCloudberryErrors() {

mrigger Oct 1, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Advice for future changes: I think it would be nice to make the expected errors more fine-grained and to have separate instantiations for specific statements (e.g., DDL statements), expressions, etc., so that only those expected errors that can actually occur are added for a given SQL statement.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

roseduan force-pushed the cloudberry-provider branch 3 times, most recently from 7cb8df5 to 380d3d0 Compare

October 14, 2026 08:26


          Add support for Apache Cloudberry

cde01fd

Apache Cloudberry is a PostgreSQL-derived MPP database, so the provider extends
PostgresProvider and reuses PostgreSQL's expression generators and oracles.
Cloudberry is a transparent MPP system: a single connection to the coordinator
is enough, so there is no worker-node registration and no extension to create.
createDatabase is inherited, wrapped only in a retry for an intermittent
MPP-specific failure (see the comment on the override).

The Cloudberry-specific parts are the distribution policy, assigned per table
via ALTER TABLE ... SET DISTRIBUTED with a distribution key chosen to be a
subset of any PRIMARY KEY or UNIQUE constraint, and leaf partitions for the
declaratively partitioned tables the inherited generator emits. Storage formats
(heap, ao_row, ao_column, pax) are exercised through the inherited USING clause.

CloudberryCommon collects the errors Cloudberry returns for statements that are
valid in PostgreSQL but violate an MPP restriction. CloudberryBugs follows the
TiDBBugs pattern, gating each work-around for an unfixed target-DB bug behind a
flag so it is easy to find and remove once the bug is fixed; each flag's comment
records the scope that was actually verified against a live cluster.

Two inherited actions are not generated. CREATE STATISTICS can crash the
coordinator, which a whitelisted error message cannot handle. CREATE VIEW builds
its body with PostgresRandomQueryGenerator, which readily emits DISTINCT ON or
LIMIT with no matching ORDER BY; such a view is re-evaluated per reference, and
on an MPP cluster the row it yields can legitimately differ between evaluations,
which breaks every oracle that compares two evaluations of the same data.

This PR covers database generation plus the three oracles that need no
Cloudberry-specific oracle implementation: NoREC, TLPWhere and the fuzzer. The
remaining oracles, and a CI job, follow in separate PRs.

Verified against a 3-segment cluster built from Cloudberry main (3.0.0-devel):
mvn verify -DskipTests=true is clean, and NoREC, TLPWhere and the fuzzer each
ran 4 threads x 140-200s across seeds 0-10 with no thread deaths and no oracle
reports.

roseduan force-pushed the cloudberry-provider branch from 380d3d0 to cde01fd Compare

October 14, 2026 08:28

roseduan requested a review from mrigger

October 14, 2026 08:44

mrigger approved these changes

Oct 14, 2026

View reviewed changes

mrigger left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot for adding support for Cloudberry! Very excited about it!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

mrigger merged commit ab328b7 into sqlancer:main

Oct 14, 2026

23 of 26 checks passed

Uh oh!

There was an error while loading. Please reload this page.

roseduan commented Oct 14, 2026

Copy link

Copy Markdown

Contributor Author

LGTM. Thanks a lot for adding support for Cloudberry! Very excited about it!

Thanks for your review! I'll continue improving and maintaining Cloudberry support in sqlancer.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Reviewers

mrigger

mrigger approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.