GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |
GeezerVison is showing github.com — feat: support scan planning and pushdown for Java table providers by wirybeaver · Pull Request #119 · apache/datafusion-java ConversationWhich issue does this PR close?Closes #118. Rationale for this changeJava table providers currently receive no physical scan request. A selective source cannot choose an access path from pushed predicates, read only projected columns, or distinguish exact filtering from candidate pruning. Expose that existing DataFusion contract while keeping the Java/JNI binding thin and source-agnostic. What changes are included in this PR?Add an opt-in DataFusion still owns residual filtering and limit placement. Planned readers produce the requested projected schema, including zero-column batches that preserve row counts. Source-specific access-path decisions stay in provider implementations; no storage SDK or DataFusion Rust-core change is introduced. This remains a bounded, single-partition leaf-scan API, not a binding for arbitrary child-consuming physical operators. Are these changes tested?A Java/JNI/Arrow integration harness registers an in-memory selective source and inspects its planned requests and actual reads. EXPLAIN plans without opening the source; execution requests only the matching row and projected column. Inexact and unsupported predicates retain residual evaluation before LIMIT, exact predicates permit limit pushdown, and independent scans keep separate requests and readers. The existing full-schema provider path is exercised through the same native boundary. The source is synthetic: this does not establish performance or correctness against an external storage service. Are there any user-facing changes?The new
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactions
wirybeaver
force-pushed
the
feat/java-table-scan-pushdown
branch
from
268a98f to
0b3a113
Compare
wirybeaver
marked this pull request as ready for review
wirybeaver
marked this pull request as draft
pgwhalen
reviewed
Uh oh!
There was an error while loading. Please reload this page. Uh oh!
There was an error while loading. Please reload this page. Uh oh!
There was an error while loading. Please reload this page. Uh oh!
There was an error while loading. Please reload this page. Expose scanWithArgs and filter capabilities as default TableProvider methods with immutable ScanArgs. Plan a lazy ExecutionPlan before execution while retaining existing full-schema providers; execute receives an explicit partition index. Forward DataFusion filter capabilities and preserve residual-filter/limit semantics without source-specific dependencies or DataFusion-core changes. Cover the Java/JNI/Arrow path with selective-source integration tests.
wirybeaver
force-pushed
the
feat/java-table-scan-pushdown
branch
from
0b3a113 to
c1869ed
Compare
pgwhalen
approved these changes
wirybeaver
marked this pull request as ready for review
Author
All reactions
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers1 more reviewerUh oh!
There was an error while loading. Please reload this page. AssigneesNo one assignedLabels
None yet
ProjectsMilestoneNo milestoneDevelopmentSuccessfully merging this pull request may close these issues. Support source scan planning and pushdown for Java table providersUh oh!
There was an error while loading. Please reload this page. Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later. | |
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |