GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: Add PolarisSource and PolarisProvider for Apache Polaris by haoxu0 · Pull Request #7006 · feast-dev/feast

Conversation

haoxu0 commented Oct 13, 2026

Copy link

Copy Markdown

Collaborator

What this PR does / why we need it:

Adds Apache Polaris support on top of the Iceberg REST catalog integration (iceberg_catalog contrib), mirroring the Unity Catalog layering from #6499.

PolarisSource(IcebergSource): endpoint defaults to $POLARIS_HOST/api/catalog; OAuth2 client-credentials or bearer-token auth read from env vars (only the variable names are stored in the registry); credential vending for reads/writes; PolarisSource.discover() builds sources for tables in a catalog.
PolarisClient: OAuth client-credentials token acquisition with refresh before expiry, and table property updates via the standard Iceberg REST updateTable (set-properties / remove-properties).
PolarisProvider (provider: polaris): on feast apply, records feast.project, feast.feature_view, feast.owner, feast.description, feast.tag.* and feast.is_feature_table on the backing Polaris tables. A deleted feature view keeps the table and marks it feast.is_feature_table=false + feast.deleted_at. A failed sync is logged and never fails feast apply.
Refactor: the PyIceberg REST catalog config was duplicated in three places (DuckDB read, DuckDB write, Spark). It now lives in IcebergSource.get_pyiceberg_catalog_config() / load_pyiceberg_table() so subclasses can override it. Behavior for existing IcebergSource users is unchanged (covered by a test).

Polaris has no lineage API, so there is no sync_lineage equivalent.

Not included (follow-up): column projection / predicate pushdown on the read path, which needs changes to how the offline stores call into the source. Multi-level namespaces keep the existing REST client behavior (single-level only).

Which issue(s) this PR fixes:

Refs #7005 (related: #6499)

Checks

I've made sure the tests are passing.
My commits are signed off (git commit -s)
My PR title follows conventional commits format

Testing Strategy

Unit tests
Integration tests
Manual tests
Testing is not required for this change

Unit: 33 new tests in iceberg_catalog/tests/test_polaris.py (HTTP mocked); ruff and mypy clean. In the existing iceberg_catalog tests, 7 fail identically on master and with this change: 2 DuckDB tests that patch get_catalog_client on a REST source, and 5 Spark tests that need pyspark.

Manual: ran against a local Apache Polaris (apache/polaris:latest) with MinIO: table discovery, DuckDB read with OAuth + vended credentials, feast apply with provider: polaris (properties verified by reloading the table through PyIceberg), get_historical_features, materialize + online read, feature view deletion, static-token mode, and bad credentials (sync returns False, no exception).

Misc

Docs are updated in docs/reference/data-sources/iceberg.md.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ntkathole reacted with heart emoji

All reactions

❤️ 1 reaction


          feat: Add PolarisSource and PolarisProvider for Apache Polaris

de3da24

Add Polaris support on top of the Iceberg REST catalog integration:

- PolarisSource(IcebergSource): endpoint defaults to $POLARIS_HOST/api/catalog,
  OAuth2 client-credentials or bearer-token auth read from env vars (only
  names are stored in the registry), credential vending, and a discover()
  helper that builds sources for tables in a catalog.
- PolarisClient: OAuth client-credentials token acquisition with refresh, and
  table property updates through the standard Iceberg REST updateTable
  (set-properties / remove-properties).
- PolarisProvider: on feast apply, record feast.* table properties for feature
  views backed by a PolarisSource; deleted views are marked, tables are kept.
  Registered as provider "polaris".
- Centralize the PyIceberg REST catalog config in
  IcebergSource.get_pyiceberg_catalog_config()/load_pyiceberg_table() and use
  it from the DuckDB and Spark offline stores, replacing three duplicated
  copies. Behavior for existing IcebergSource users is unchanged.

Refs feast-dev#7005

Signed-off-by: hao-xu5 <[email protected]>

haoxu0 requested a review from a team as a code owner

October 13, 2026 18:49

github-advanced-security Bot found potential problems

Oct 13, 2026

View reviewed changes

Comment thread

sdk/python/feast/infra/data_sources/contrib/iceberg_catalog/polaris_source.py Fixed

Uh oh!

There was an error while loading. Please reload this page.

github-advanced-security Bot found potential problems

Oct 13, 2026

View reviewed changes

Comment thread

sdk/python/feast/infra/data_sources/contrib/iceberg_catalog/polaris_source.py Outdated

		"No Polaris credentials found: set %s and %s (OAuth client "
		"credentials) or %s (bearer token)",
		self.client_id_env_var,
		self.client_secret_env_var,

codecov-commenter commented Oct 13, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 21.52466% with 175 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.94%. Comparing base (2dfb9e6) to head (5ea289d).

Files with missing lines	Patch %	Lines
..._sources/contrib/iceberg_catalog/polaris_source.py	27.84%	57 Missing ⚠️
..._sources/contrib/iceberg_catalog/polaris_client.py	30.55%	50 Missing ⚠️
...ources/contrib/iceberg_catalog/polaris_provider.py	0.00%	50 Missing ⚠️
..._sources/contrib/iceberg_catalog/iceberg_source.py	11.76%	15 Missing ⚠️
sdk/python/feast/infra/offline_stores/duckdb.py	0.00%	2 Missing ⚠️
...ffline_stores/contrib/spark_offline_store/spark.py	0.00%	1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7006      +/-   ##
==========================================
- Coverage   51.02%   50.94%   -0.09%     
==========================================
  Files         452      455       +3     
  Lines       56728    56915     +187     
  Branches     8265     8290      +25     
==========================================
+ Hits        28946    28994      +48     
- Misses      25801    25940     +139     
  Partials     1981     1981

Flag	Coverage Δ
go-feature-server	`30.58% <ø> (ø)`
python-unit	`52.40% <21.52%> (-0.10%)`	⬇️

Files with missing lines	Coverage Δ
...a/data_sources/contrib/iceberg_catalog/__init__.py	`100.00% <100.00%> (ø)`
sdk/python/feast/infra/provider.py	`92.30% <ø> (ø)`
...ffline_stores/contrib/spark_offline_store/spark.py	`36.55% <0.00%> (+0.56%)`	⬆️
sdk/python/feast/infra/offline_stores/duckdb.py	`43.20% <0.00%> (+2.43%)`	⬆️
..._sources/contrib/iceberg_catalog/iceberg_source.py	`23.07% <11.76%> (-1.93%)`	⬇️
..._sources/contrib/iceberg_catalog/polaris_client.py	`30.55% <30.55%> (ø)`
...ources/contrib/iceberg_catalog/polaris_provider.py	`0.00% <0.00%> (ø)`
..._sources/contrib/iceberg_catalog/polaris_source.py	`27.84% <27.84%> (ø)`

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2dfb9e6...5ea289d. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          Merge branch 'master' into feat/polaris-source

e5162c6

haoxu0 commented Oct 13, 2026

Copy link

Copy Markdown

Collaborator Author

@ntkathole PTAL, thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          fix: Address CI secret-scan and CodeQL findings in Polaris source

5ea289d

Allowlist the env var names used as test fixtures (detect-secrets false positives) and stop logging credential env var names (CodeQL py/clear-text-logging-sensitive-data).

Signed-off-by: hao-xu5 <[email protected]>

haoxu0 merged commit 386759c into feast-dev:master

Oct 15, 2026

26 checks passed

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Oct 15, 2026

feat: Push column projection and time-range filters down to Iceberg scans in the DuckDB and Spark offline stores #7019

Open

feat: Add PolarisSource and Polaris governance sync on top of IcebergSource #7005

Closed

haoxu0 deleted the feat/polaris-source branch

October 15, 2026 08:12

franciscojavierarceo pushed a commit that referenced this pull request

Nov 10, 2026


          chore(release): release 0.68.0

fb5e5d5

# [0.68.0](v0.67.0...v0.68.0) (2026-11-10)

### Bug Fixes

* add watch verb on clusterroles and clusterrolebindings for feast-operator ([#7062](#7062)) ([b6bdda4](b6bdda4))
* Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([#6993](#6993)) ([f416309](f416309))
* Break event timestamp ties by created timestamp in Milvus write batches ([#7084](#7084)) ([bbaca76](bbaca76))
* Break event timestamp ties by created timestamp in the Dask offline store ([#6952](#6952)) ([eb93b08](eb93b08))
* Build the Go feature server image in two stages ([1e5e90c](1e5e90c)), closes [#7060](#7060)
* Fall back to the serialized body when UDF source misses globals ([#6816](#6816)) ([f61079f](f61079f))
* Install patched libbrotli1 to address DSA-6582-1 ([#7096](#7096)) ([d458393](d458393))
* Keep description, tags and nested types of ODFV schema fields ([8ef6e75](8ef6e75))
* Keep entity columns out of StreamFeatureView proto features ([#7033](#7033)) ([d520584](d520584))
* Keep the metrics server running when a Gunicorn worker exits ([#7012](#7012)) ([68c5d54](68c5d54)), closes [#7011](#7011)
* Key registry proto caches on the proto, not its id() ([#7085](#7085)) ([147adad](147adad))
* Log ODFV materialization skips instead of printing them ([#7117](#7117)) ([8fdcee7](8fdcee7))
* Pin build dependencies of source builds to the CI lock ([#7104](#7104)) ([dec2f24](dec2f24))
* Populate spec.project in registry REST feature view responses (#… ([#7037](#7037)) ([e304a84](e304a84))
* Preserve colliding source timestamps in Dask retrieval ([#7071](#7071)) ([1396c88](1396c88))
* Preserve Dask historical requests without eligible features ([#7000](#7000)) ([fff09f7](fff09f7))
* Read None elements of UUID and Decimal lists as None ([#7039](#7039)) ([03f30fe](03f30fe))
* Retire the support of Python3.10 ([#7101](#7101)) ([a56766e](a56766e)), closes [#7104](#7104) [#7089](#7089) [#7106](#7106) [#7105](#7105)
* Serialize UUID and TimeUuid entity keys ([#7032](#7032)) ([0c845a1](0c845a1))
* Stop the Ray engine from hiding materialization errors ([#7026](#7026)) ([7903e2e](7903e2e)), closes [#7024](#7024)
* **ui:** stop treating the monitoring baseline as the latest window ([#7056](#7056)) ([0f01802](0f01802)), closes [#7054](#7054)
* Use date-only bounds for partition filters built from string timestamps ([#7116](#7116)) ([b1f2168](b1f2168))
* Validate entity column types in feast dbt import ([#7067](#7067)) ([abca80c](abca80c)), closes [#5876](#5876)
* Warm up the whole DynamoDB connection pool ([#6985](#6985)) ([8ce548c](8ce548c))

### Features

* Add --gc-freeze option to feast serve ([#7089](#7089)) ([aad0c15](aad0c15))
* Add native complex types and scalar index tags to the Milvus online store ([#7120](#7120)) ([f7068d6](f7068d6))
* Add num_partitions and partition_key_isolation to the Milvus online store ([#7083](#7083)) ([68de84e](68de84e)), closes [#6940](#6940)
* Add Pinecone online store integration ([#6585](#6585)) ([7a84f2a](7a84f2a))
* Add PolarisSource and PolarisProvider for Apache Polaris ([#7006](#7006)) ([386759c](386759c))
* Add row deletion to the Milvus online store ([#7082](#7082)) ([d3e2041](d3e2041))
* Add upstream FeatureView lineage support for PushSource ([#6853](#6853)) ([c9b9d72](c9b9d72))
* Optional nameless pre-computed vectors with a schema fingerprint ([#7077](#7077)) ([9420fac](9420fac))

### Performance Improvements

* Scan Lance sources lazily in the DuckDB offline store ([1299e4e](1299e4e))

patelchaitany mentioned this pull request

Nov 11, 2026

chore: Sync upstream feast-dev/feast stable (0.68.0) opendatahub-io/feast#291

Merged

shuchu pushed a commit to shuchu/feast that referenced this pull request

Nov 11, 2026


          chore(release): release 0.68.0

5b94c9e

# [0.68.0](feast-dev/feast@v0.67.0...v0.68.0) (2026-11-10)

### Bug Fixes

* add watch verb on clusterroles and clusterrolebindings for feast-operator ([feast-dev#7062](feast-dev#7062)) ([b6bdda4](feast-dev@b6bdda4))
* Apply on demand feature views in SparkRetrievalJob.to_spark_df() ([feast-dev#6993](feast-dev#6993)) ([f416309](feast-dev@f416309))
* Break event timestamp ties by created timestamp in Milvus write batches ([feast-dev#7084](feast-dev#7084)) ([bbaca76](feast-dev@bbaca76))
* Break event timestamp ties by created timestamp in the Dask offline store ([feast-dev#6952](feast-dev#6952)) ([eb93b08](feast-dev@eb93b08))
* Build the Go feature server image in two stages ([1e5e90c](feast-dev@1e5e90c)), closes [feast-dev#7060](feast-dev#7060)
* Fall back to the serialized body when UDF source misses globals ([feast-dev#6816](feast-dev#6816)) ([f61079f](feast-dev@f61079f))
* Install patched libbrotli1 to address DSA-6582-1 ([feast-dev#7096](feast-dev#7096)) ([d458393](feast-dev@d458393))
* Keep description, tags and nested types of ODFV schema fields ([8ef6e75](feast-dev@8ef6e75))
* Keep entity columns out of StreamFeatureView proto features ([feast-dev#7033](feast-dev#7033)) ([d520584](feast-dev@d520584))
* Keep the metrics server running when a Gunicorn worker exits ([feast-dev#7012](feast-dev#7012)) ([68c5d54](feast-dev@68c5d54)), closes [feast-dev#7011](feast-dev#7011)
* Key registry proto caches on the proto, not its id() ([feast-dev#7085](feast-dev#7085)) ([147adad](feast-dev@147adad))
* Log ODFV materialization skips instead of printing them ([feast-dev#7117](feast-dev#7117)) ([8fdcee7](feast-dev@8fdcee7))
* Pin build dependencies of source builds to the CI lock ([feast-dev#7104](feast-dev#7104)) ([dec2f24](feast-dev@dec2f24))
* Populate spec.project in registry REST feature view responses (#… ([feast-dev#7037](feast-dev#7037)) ([e304a84](feast-dev@e304a84))
* Preserve colliding source timestamps in Dask retrieval ([feast-dev#7071](feast-dev#7071)) ([1396c88](feast-dev@1396c88))
* Preserve Dask historical requests without eligible features ([feast-dev#7000](feast-dev#7000)) ([fff09f7](feast-dev@fff09f7))
* Read None elements of UUID and Decimal lists as None ([feast-dev#7039](feast-dev#7039)) ([03f30fe](feast-dev@03f30fe))
* Retire the support of Python3.10 ([feast-dev#7101](feast-dev#7101)) ([a56766e](feast-dev@a56766e)), closes [feast-dev#7104](feast-dev#7104) [feast-dev#7089](feast-dev#7089) [feast-dev#7106](feast-dev#7106) [feast-dev#7105](feast-dev#7105)
* Serialize UUID and TimeUuid entity keys ([feast-dev#7032](feast-dev#7032)) ([0c845a1](feast-dev@0c845a1))
* Stop the Ray engine from hiding materialization errors ([feast-dev#7026](feast-dev#7026)) ([7903e2e](feast-dev@7903e2e)), closes [feast-dev#7024](feast-dev#7024)
* **ui:** stop treating the monitoring baseline as the latest window ([feast-dev#7056](feast-dev#7056)) ([0f01802](feast-dev@0f01802)), closes [feast-dev#7054](feast-dev#7054)
* Use date-only bounds for partition filters built from string timestamps ([feast-dev#7116](feast-dev#7116)) ([b1f2168](feast-dev@b1f2168))
* Validate entity column types in feast dbt import ([feast-dev#7067](feast-dev#7067)) ([abca80c](feast-dev@abca80c)), closes [feast-dev#5876](feast-dev#5876)
* Warm up the whole DynamoDB connection pool ([feast-dev#6985](feast-dev#6985)) ([8ce548c](feast-dev@8ce548c))

### Features

* Add --gc-freeze option to feast serve ([feast-dev#7089](feast-dev#7089)) ([aad0c15](feast-dev@aad0c15))
* Add native complex types and scalar index tags to the Milvus online store ([feast-dev#7120](feast-dev#7120)) ([f7068d6](feast-dev@f7068d6))
* Add num_partitions and partition_key_isolation to the Milvus online store ([feast-dev#7083](feast-dev#7083)) ([68de84e](feast-dev@68de84e)), closes [feast-dev#6940](feast-dev#6940)
* Add Pinecone online store integration ([feast-dev#6585](feast-dev#6585)) ([7a84f2a](feast-dev@7a84f2a))
* Add PolarisSource and PolarisProvider for Apache Polaris ([feast-dev#7006](feast-dev#7006)) ([386759c](feast-dev@386759c))
* Add row deletion to the Milvus online store ([feast-dev#7082](feast-dev#7082)) ([d3e2041](feast-dev@d3e2041))
* Add upstream FeatureView lineage support for PushSource ([feast-dev#6853](feast-dev#6853)) ([c9b9d72](feast-dev@c9b9d72))
* Optional nameless pre-computed vectors with a schema fingerprint ([feast-dev#7077](feast-dev#7077)) ([9420fac](feast-dev@9420fac))

### Performance Improvements

* Scan Lance sources lazily in the DuckDB offline store ([1299e4e](feast-dev@1299e4e))

Signed-off-by: Shuchu Han <[email protected]>

rhods-devops-app Bot mentioned this pull request

Nov 12, 2026

Automated Sync from main to stable red-hat-data-services/feast#630

Open

Reviewers

1 more reviewer

github-advanced-security[bot]

github-advanced-security[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

haoxu0 commented Oct 13, 2026

What this PR does / why we need it:

Which issue(s) this PR fixes:

Checks

Testing Strategy

Misc

Uh oh!

Uh oh!

codecov-commenter commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

haoxu0 commented Oct 13, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Oct 13, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.