GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — fix: Spark Materialization Engine Cannot Infer Schema by jfw-ppi · Pull Request #5806 · feast-dev/feast

Conversation

jfw-ppi commented Jan 1, 2026

Copy link

Copy Markdown

Collaborator

What this PR does / why we need it:

Handle Arrow-to-Spark conversion without schema inference errors by deriving schema from Arrow and creating an empty Spark DataFrame when no rows are returned.

Which issue(s) this PR fixes:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

jfw-ppi requested a review from a team as a code owner

January 1, 2026 22:13

jfw-ppi changed the title ~~fix: build Spark DataFrame from Arrow with schema and empty handling(…~~ fix: Spark Materialization Engine Cannot Infer Schema

Jan 1, 2026

jfw-ppi force-pushed the 5594-spark-infer-schema branch from cf9b66e to b77a740 Compare

January 1, 2026 22:18

franciscojavierarceo approved these changes

Jan 2, 2026

View reviewed changes

franciscojavierarceo added the ok-to-test label

Jan 2, 2026

jfw-ppi added 2 commits

January 2, 2026 13:27


          fix: build Spark DataFrame from Arrow with schema and empty handling(f…

611ead0

…east-dev#5594)

Signed-off-by: Jacob Weinhold <[email protected]>


          fix: build Spark DataFrame from Arrow with schema and empty handling(f…

237a65d

…east-dev#5594)

Signed-off-by: Jacob Weinhold <[email protected]>

ntkathole force-pushed the 5594-spark-infer-schema branch from b77a740 to 237a65d Compare

January 2, 2026 07:57

ntkathole approved these changes

Jan 2, 2026

View reviewed changes

ntkathole merged commit 58d0325 into feast-dev:master

Jan 2, 2026

16 of 17 checks passed

Uh oh!

There was an error while loading. Please reload this page.

jfw-ppi deleted the 5594-spark-infer-schema branch

January 2, 2026 09:17

franciscojavierarceo pushed a commit that referenced this pull request

Jan 16, 2026


          chore(release): release 0.59.0

6214d05

# [0.59.0](v0.58.0...v0.59.0) (2026-01-16)

### Bug Fixes

* Add get_table_query_string_with_alias() for PostgreSQL subquery aliasing ([#5811](#5811)) ([11122ce](11122ce))
* Add hybrid online store to ONLINE_STORE_CLASS_FOR_TYPE mapping ([#5810](#5810)) ([678589b](678589b))
* Add possibility to overwrite send_receive_timeout for clickhouse offline store ([#5792](#5792)) ([59dbb33](59dbb33))
* Denial by default to all resources when no permissions set  ([#5663](#5663)) ([1524f1c](1524f1c))
* Make operator include full OIDC secret in repo config ([#5676](#5676)) ([#5809](#5809)) ([a536bc2](a536bc2))
* Populate Postgres `registry.path` during `feast init` ([#5785](#5785)) ([f293ae8](f293ae8))
* **redis:** Preserve millisecond timestamp precision for Redis online store ([#5807](#5807)) ([9e3f213](9e3f213))
* Search API to return all matching tags in matched_tags field ([#5843](#5843)) ([de37f66](de37f66))
* Spark Materialization Engine Cannot Infer Schema ([#5806](#5806)) ([58d0325](58d0325)), closes [#5594](#5594) [#5594](#5594)
* Support arro3 table schema with newer deltalake packages ([#5799](#5799)) ([103c5e9](103c5e9))
* Timestamp formatting and lakehouse-type connector for trino_offline_store. ([#5846](#5846)) ([c2ea7e9](c2ea7e9))
* Update model_validator to use instance method signature (Pydantic v2.12 deprecation) ([#5825](#5825)) ([3c10b6e](3c10b6e))

### Features

* Add dbt integration for importing models as FeatureViews ([#5827](#5827)) ([b997361](b997361)), closes [#3335](#3335) [#3335](#3335) [#3335](#3335)
* Add GCS registry store in Go feature server ([#5818](#5818)) ([1dc2be5](1dc2be5))
* Add progress bar to CLI from feast apply ([#5867](#5867)) ([ab3562b](ab3562b))
* Add RBAC blog post to website ([#5861](#5861)) ([b1844a3](b1844a3))
* Add skip_feature_view_validation parameter to FeatureStore.apply() and plan() ([#5859](#5859)) ([5482a0e](5482a0e))
* Added batching to feature server /push to offline store ([#5683](#5683)) ([#5729](#5729)) ([ce35ce6](ce35ce6))
* Enable static artifacts for feature server that can be used in Feature Transformations ([#5787](#5787)) ([edefc3f](edefc3f))
* Improve lambda materialization engine ([#5829](#5829)) ([f6116f9](f6116f9))
* Offline Store historical features retrieval based on datetime range in Ray ([#5738](#5738)) ([e484c12](e484c12))
* Read, Save docs and chat fixes ([#5865](#5865)) ([2081b55](2081b55))
* Resolve pyarrow >21 installation with ibis-framework ([#5847](#5847)) ([8b9bb50](8b9bb50))
* Support staging for spark materialization ([#5671](#5671)) ([#5797](#5797)) ([5b787af](5b787af))

kyledepasquale pushed a commit to kyledepasquale/feast that referenced this pull request

Jan 21, 2026


          chore(release): release 0.59.0

b52f528

# [0.59.0](feast-dev/feast@v0.58.0...v0.59.0) (2026-01-16)

### Bug Fixes

* Add get_table_query_string_with_alias() for PostgreSQL subquery aliasing ([feast-dev#5811](feast-dev#5811)) ([11122ce](feast-dev@11122ce))
* Add hybrid online store to ONLINE_STORE_CLASS_FOR_TYPE mapping ([feast-dev#5810](feast-dev#5810)) ([678589b](feast-dev@678589b))
* Add possibility to overwrite send_receive_timeout for clickhouse offline store ([feast-dev#5792](feast-dev#5792)) ([59dbb33](feast-dev@59dbb33))
* Denial by default to all resources when no permissions set  ([feast-dev#5663](feast-dev#5663)) ([1524f1c](feast-dev@1524f1c))
* Make operator include full OIDC secret in repo config ([feast-dev#5676](feast-dev#5676)) ([feast-dev#5809](feast-dev#5809)) ([a536bc2](feast-dev@a536bc2))
* Populate Postgres `registry.path` during `feast init` ([feast-dev#5785](feast-dev#5785)) ([f293ae8](feast-dev@f293ae8))
* **redis:** Preserve millisecond timestamp precision for Redis online store ([feast-dev#5807](feast-dev#5807)) ([9e3f213](feast-dev@9e3f213))
* Search API to return all matching tags in matched_tags field ([feast-dev#5843](feast-dev#5843)) ([de37f66](feast-dev@de37f66))
* Spark Materialization Engine Cannot Infer Schema ([feast-dev#5806](feast-dev#5806)) ([58d0325](feast-dev@58d0325)), closes [feast-dev#5594](feast-dev#5594) [feast-dev#5594](feast-dev#5594)
* Support arro3 table schema with newer deltalake packages ([feast-dev#5799](feast-dev#5799)) ([103c5e9](feast-dev@103c5e9))
* Timestamp formatting and lakehouse-type connector for trino_offline_store. ([feast-dev#5846](feast-dev#5846)) ([c2ea7e9](feast-dev@c2ea7e9))
* Update model_validator to use instance method signature (Pydantic v2.12 deprecation) ([feast-dev#5825](feast-dev#5825)) ([3c10b6e](feast-dev@3c10b6e))

### Features

* Add dbt integration for importing models as FeatureViews ([feast-dev#5827](feast-dev#5827)) ([b997361](feast-dev@b997361)), closes [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335)
* Add GCS registry store in Go feature server ([feast-dev#5818](feast-dev#5818)) ([1dc2be5](feast-dev@1dc2be5))
* Add progress bar to CLI from feast apply ([feast-dev#5867](feast-dev#5867)) ([ab3562b](feast-dev@ab3562b))
* Add RBAC blog post to website ([feast-dev#5861](feast-dev#5861)) ([b1844a3](feast-dev@b1844a3))
* Add skip_feature_view_validation parameter to FeatureStore.apply() and plan() ([feast-dev#5859](feast-dev#5859)) ([5482a0e](feast-dev@5482a0e))
* Added batching to feature server /push to offline store ([feast-dev#5683](feast-dev#5683)) ([feast-dev#5729](feast-dev#5729)) ([ce35ce6](feast-dev@ce35ce6))
* Enable static artifacts for feature server that can be used in Feature Transformations ([feast-dev#5787](feast-dev#5787)) ([edefc3f](feast-dev@edefc3f))
* Improve lambda materialization engine ([feast-dev#5829](feast-dev#5829)) ([f6116f9](feast-dev@f6116f9))
* Offline Store historical features retrieval based on datetime range in Ray ([feast-dev#5738](feast-dev#5738)) ([e484c12](feast-dev@e484c12))
* Read, Save docs and chat fixes ([feast-dev#5865](feast-dev#5865)) ([2081b55](feast-dev@2081b55))
* Resolve pyarrow >21 installation with ibis-framework ([feast-dev#5847](feast-dev#5847)) ([8b9bb50](feast-dev@8b9bb50))
* Support staging for spark materialization ([feast-dev#5671](feast-dev#5671)) ([feast-dev#5797](feast-dev#5797)) ([5b787af](feast-dev@5b787af))

Signed-off-by: Kyle DePasquale <[email protected]>

YassinNouh21 pushed a commit to YassinNouh21/feast that referenced this pull request

Feb 7, 2026


          chore(release): release 0.59.0

61dff40

# [0.59.0](feast-dev/feast@v0.58.0...v0.59.0) (2026-01-16)

### Bug Fixes

* Add get_table_query_string_with_alias() for PostgreSQL subquery aliasing ([feast-dev#5811](feast-dev#5811)) ([11122ce](feast-dev@11122ce))
* Add hybrid online store to ONLINE_STORE_CLASS_FOR_TYPE mapping ([feast-dev#5810](feast-dev#5810)) ([678589b](feast-dev@678589b))
* Add possibility to overwrite send_receive_timeout for clickhouse offline store ([feast-dev#5792](feast-dev#5792)) ([59dbb33](feast-dev@59dbb33))
* Denial by default to all resources when no permissions set  ([feast-dev#5663](feast-dev#5663)) ([1524f1c](feast-dev@1524f1c))
* Make operator include full OIDC secret in repo config ([feast-dev#5676](feast-dev#5676)) ([feast-dev#5809](feast-dev#5809)) ([a536bc2](feast-dev@a536bc2))
* Populate Postgres `registry.path` during `feast init` ([feast-dev#5785](feast-dev#5785)) ([f293ae8](feast-dev@f293ae8))
* **redis:** Preserve millisecond timestamp precision for Redis online store ([feast-dev#5807](feast-dev#5807)) ([9e3f213](feast-dev@9e3f213))
* Search API to return all matching tags in matched_tags field ([feast-dev#5843](feast-dev#5843)) ([de37f66](feast-dev@de37f66))
* Spark Materialization Engine Cannot Infer Schema ([feast-dev#5806](feast-dev#5806)) ([58d0325](feast-dev@58d0325)), closes [feast-dev#5594](feast-dev#5594) [feast-dev#5594](feast-dev#5594)
* Support arro3 table schema with newer deltalake packages ([feast-dev#5799](feast-dev#5799)) ([103c5e9](feast-dev@103c5e9))
* Timestamp formatting and lakehouse-type connector for trino_offline_store. ([feast-dev#5846](feast-dev#5846)) ([c2ea7e9](feast-dev@c2ea7e9))
* Update model_validator to use instance method signature (Pydantic v2.12 deprecation) ([feast-dev#5825](feast-dev#5825)) ([3c10b6e](feast-dev@3c10b6e))

### Features

* Add dbt integration for importing models as FeatureViews ([feast-dev#5827](feast-dev#5827)) ([b997361](feast-dev@b997361)), closes [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335)
* Add GCS registry store in Go feature server ([feast-dev#5818](feast-dev#5818)) ([1dc2be5](feast-dev@1dc2be5))
* Add progress bar to CLI from feast apply ([feast-dev#5867](feast-dev#5867)) ([ab3562b](feast-dev@ab3562b))
* Add RBAC blog post to website ([feast-dev#5861](feast-dev#5861)) ([b1844a3](feast-dev@b1844a3))
* Add skip_feature_view_validation parameter to FeatureStore.apply() and plan() ([feast-dev#5859](feast-dev#5859)) ([5482a0e](feast-dev@5482a0e))
* Added batching to feature server /push to offline store ([feast-dev#5683](feast-dev#5683)) ([feast-dev#5729](feast-dev#5729)) ([ce35ce6](feast-dev@ce35ce6))
* Enable static artifacts for feature server that can be used in Feature Transformations ([feast-dev#5787](feast-dev#5787)) ([edefc3f](feast-dev@edefc3f))
* Improve lambda materialization engine ([feast-dev#5829](feast-dev#5829)) ([f6116f9](feast-dev@f6116f9))
* Offline Store historical features retrieval based on datetime range in Ray ([feast-dev#5738](feast-dev#5738)) ([e484c12](feast-dev@e484c12))
* Read, Save docs and chat fixes ([feast-dev#5865](feast-dev#5865)) ([2081b55](feast-dev@2081b55))
* Resolve pyarrow >21 installation with ibis-framework ([feast-dev#5847](feast-dev#5847)) ([8b9bb50](feast-dev@8b9bb50))
* Support staging for spark materialization ([feast-dev#5671](feast-dev#5671)) ([feast-dev#5797](feast-dev#5797)) ([5b787af](feast-dev@5b787af))

Signed-off-by: yassinnouh21 <[email protected]>

YassinNouh21 pushed a commit to YassinNouh21/feast that referenced this pull request

Feb 7, 2026


          chore(release): release 0.59.0

ac546ff

# [0.59.0](feast-dev/feast@v0.58.0...v0.59.0) (2026-01-16)

### Bug Fixes

* Add get_table_query_string_with_alias() for PostgreSQL subquery aliasing ([feast-dev#5811](feast-dev#5811)) ([11122ce](feast-dev@11122ce))
* Add hybrid online store to ONLINE_STORE_CLASS_FOR_TYPE mapping ([feast-dev#5810](feast-dev#5810)) ([678589b](feast-dev@678589b))
* Add possibility to overwrite send_receive_timeout for clickhouse offline store ([feast-dev#5792](feast-dev#5792)) ([59dbb33](feast-dev@59dbb33))
* Denial by default to all resources when no permissions set  ([feast-dev#5663](feast-dev#5663)) ([1524f1c](feast-dev@1524f1c))
* Make operator include full OIDC secret in repo config ([feast-dev#5676](feast-dev#5676)) ([feast-dev#5809](feast-dev#5809)) ([a536bc2](feast-dev@a536bc2))
* Populate Postgres `registry.path` during `feast init` ([feast-dev#5785](feast-dev#5785)) ([f293ae8](feast-dev@f293ae8))
* **redis:** Preserve millisecond timestamp precision for Redis online store ([feast-dev#5807](feast-dev#5807)) ([9e3f213](feast-dev@9e3f213))
* Search API to return all matching tags in matched_tags field ([feast-dev#5843](feast-dev#5843)) ([de37f66](feast-dev@de37f66))
* Spark Materialization Engine Cannot Infer Schema ([feast-dev#5806](feast-dev#5806)) ([58d0325](feast-dev@58d0325)), closes [feast-dev#5594](feast-dev#5594) [feast-dev#5594](feast-dev#5594)
* Support arro3 table schema with newer deltalake packages ([feast-dev#5799](feast-dev#5799)) ([103c5e9](feast-dev@103c5e9))
* Timestamp formatting and lakehouse-type connector for trino_offline_store. ([feast-dev#5846](feast-dev#5846)) ([c2ea7e9](feast-dev@c2ea7e9))
* Update model_validator to use instance method signature (Pydantic v2.12 deprecation) ([feast-dev#5825](feast-dev#5825)) ([3c10b6e](feast-dev@3c10b6e))

### Features

* Add dbt integration for importing models as FeatureViews ([feast-dev#5827](feast-dev#5827)) ([b997361](feast-dev@b997361)), closes [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335)
* Add GCS registry store in Go feature server ([feast-dev#5818](feast-dev#5818)) ([1dc2be5](feast-dev@1dc2be5))
* Add progress bar to CLI from feast apply ([feast-dev#5867](feast-dev#5867)) ([ab3562b](feast-dev@ab3562b))
* Add RBAC blog post to website ([feast-dev#5861](feast-dev#5861)) ([b1844a3](feast-dev@b1844a3))
* Add skip_feature_view_validation parameter to FeatureStore.apply() and plan() ([feast-dev#5859](feast-dev#5859)) ([5482a0e](feast-dev@5482a0e))
* Added batching to feature server /push to offline store ([feast-dev#5683](feast-dev#5683)) ([feast-dev#5729](feast-dev#5729)) ([ce35ce6](feast-dev@ce35ce6))
* Enable static artifacts for feature server that can be used in Feature Transformations ([feast-dev#5787](feast-dev#5787)) ([edefc3f](feast-dev@edefc3f))
* Improve lambda materialization engine ([feast-dev#5829](feast-dev#5829)) ([f6116f9](feast-dev@f6116f9))
* Offline Store historical features retrieval based on datetime range in Ray ([feast-dev#5738](feast-dev#5738)) ([e484c12](feast-dev@e484c12))
* Read, Save docs and chat fixes ([feast-dev#5865](feast-dev#5865)) ([2081b55](feast-dev@2081b55))
* Resolve pyarrow >21 installation with ibis-framework ([feast-dev#5847](feast-dev#5847)) ([8b9bb50](feast-dev@8b9bb50))
* Support staging for spark materialization ([feast-dev#5671](feast-dev#5671)) ([feast-dev#5797](feast-dev#5797)) ([5b787af](feast-dev@5b787af))

Signed-off-by: yassinnouh21 <[email protected]>

jyejare pushed a commit to opendatahub-io/feast that referenced this pull request

Mar 9, 2026


          fix: Spark Materialization Engine Cannot Infer Schema (feast-dev#5806)

6ab1b78

* fix: build Spark DataFrame from Arrow with schema and empty handling(feast-dev#5594)

Signed-off-by: Jacob Weinhold <[email protected]>

* fix: build Spark DataFrame from Arrow with schema and empty handling(feast-dev#5594)

Signed-off-by: Jacob Weinhold <[email protected]>

---------

Signed-off-by: Jacob Weinhold <[email protected]>

jyejare pushed a commit to opendatahub-io/feast that referenced this pull request

Mar 9, 2026


          chore(release): release 0.59.0

dedd393

# [0.59.0](feast-dev/feast@v0.58.0...v0.59.0) (2026-01-16)

### Bug Fixes

* Add get_table_query_string_with_alias() for PostgreSQL subquery aliasing ([feast-dev#5811](feast-dev#5811)) ([11122ce](feast-dev@11122ce))
* Add hybrid online store to ONLINE_STORE_CLASS_FOR_TYPE mapping ([feast-dev#5810](feast-dev#5810)) ([678589b](feast-dev@678589b))
* Add possibility to overwrite send_receive_timeout for clickhouse offline store ([feast-dev#5792](feast-dev#5792)) ([59dbb33](feast-dev@59dbb33))
* Denial by default to all resources when no permissions set  ([feast-dev#5663](feast-dev#5663)) ([1524f1c](feast-dev@1524f1c))
* Make operator include full OIDC secret in repo config ([feast-dev#5676](feast-dev#5676)) ([feast-dev#5809](feast-dev#5809)) ([a536bc2](feast-dev@a536bc2))
* Populate Postgres `registry.path` during `feast init` ([feast-dev#5785](feast-dev#5785)) ([f293ae8](feast-dev@f293ae8))
* **redis:** Preserve millisecond timestamp precision for Redis online store ([feast-dev#5807](feast-dev#5807)) ([9e3f213](feast-dev@9e3f213))
* Search API to return all matching tags in matched_tags field ([feast-dev#5843](feast-dev#5843)) ([de37f66](feast-dev@de37f66))
* Spark Materialization Engine Cannot Infer Schema ([feast-dev#5806](feast-dev#5806)) ([58d0325](feast-dev@58d0325)), closes [feast-dev#5594](feast-dev#5594) [feast-dev#5594](feast-dev#5594)
* Support arro3 table schema with newer deltalake packages ([feast-dev#5799](feast-dev#5799)) ([103c5e9](feast-dev@103c5e9))
* Timestamp formatting and lakehouse-type connector for trino_offline_store. ([feast-dev#5846](feast-dev#5846)) ([c2ea7e9](feast-dev@c2ea7e9))
* Update model_validator to use instance method signature (Pydantic v2.12 deprecation) ([feast-dev#5825](feast-dev#5825)) ([3c10b6e](feast-dev@3c10b6e))

### Features

* Add dbt integration for importing models as FeatureViews ([feast-dev#5827](feast-dev#5827)) ([b997361](feast-dev@b997361)), closes [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335) [feast-dev#3335](feast-dev#3335)
* Add GCS registry store in Go feature server ([feast-dev#5818](feast-dev#5818)) ([1dc2be5](feast-dev@1dc2be5))
* Add progress bar to CLI from feast apply ([feast-dev#5867](feast-dev#5867)) ([ab3562b](feast-dev@ab3562b))
* Add RBAC blog post to website ([feast-dev#5861](feast-dev#5861)) ([b1844a3](feast-dev@b1844a3))
* Add skip_feature_view_validation parameter to FeatureStore.apply() and plan() ([feast-dev#5859](feast-dev#5859)) ([5482a0e](feast-dev@5482a0e))
* Added batching to feature server /push to offline store ([feast-dev#5683](feast-dev#5683)) ([feast-dev#5729](feast-dev#5729)) ([ce35ce6](feast-dev@ce35ce6))
* Enable static artifacts for feature server that can be used in Feature Transformations ([feast-dev#5787](feast-dev#5787)) ([edefc3f](feast-dev@edefc3f))
* Improve lambda materialization engine ([feast-dev#5829](feast-dev#5829)) ([f6116f9](feast-dev@f6116f9))
* Offline Store historical features retrieval based on datetime range in Ray ([feast-dev#5738](feast-dev#5738)) ([e484c12](feast-dev@e484c12))
* Read, Save docs and chat fixes ([feast-dev#5865](feast-dev#5865)) ([2081b55](feast-dev@2081b55))
* Resolve pyarrow >21 installation with ibis-framework ([feast-dev#5847](feast-dev#5847)) ([8b9bb50](feast-dev@8b9bb50))
* Support staging for spark materialization ([feast-dev#5671](feast-dev#5671)) ([feast-dev#5797](feast-dev#5797)) ([5b787af](feast-dev@5b787af))

alexxfan mentioned this pull request

Sep 14, 2026

Automated Sync from master to stable rhoai-rhtap/feast#52

Open

Reviewers

franciscojavierarceo

franciscojavierarceo approved these changes

ntkathole

ntkathole approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.