GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix: Handle numpy ndarray in Array(String) materialization by SIDDHESH1564 · Pull Request #6327 · feast-dev/feast

Conversation

@SIDDHESH1564

@SIDDHESH1564 SIDDHESH1564 commented Apr 24, 2026 •
edited by devin-ai-integration Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Materializing feature views that contain Array(String) columns using the Athena offline store fails with TypeError or ValueError.

Root cause:
Arrow/Athena deserializes array columns as numpy.ndarray (object dtype) instead of Python lists, which breaks assumptions in feast/type_map.py.

Issues observed:

  • _validate_collection_item_types: None elements inside ndarray fail strict type validation.
  • _convert_list_values_to_proto: Passing numpy.ndarray directly to protobuf constructors (e.g. StringList) raises TypeError. None elements are also invalid for protobuf repeated fields.
  • _convert_scalar_values_to_proto: pd.isnull(ndarray) returns an array of booleans; applying not raises ValueError ("truth value of an empty array is ambiguous").

Fix implemented:

  • Convert numpy.ndarray → Python list using .tolist() before proto conversion
  • Replace None elements with type-appropriate defaults:
    • "" (string), 0 (int), 0.0 (float), False (bool)
  • Allow None during intermediate validation in _validate_collection_item_types

Which issue(s) this PR fixes:

Fixes #6325


Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Details:
Added TestNdarrayListConversion in test_type_map.py with regression coverage for:

  • ndarray string/int/double/bool list conversions
  • Replacement of None elements with defaults
  • Empty ndarray producing null ProtoValue
  • Mixed batch scenarios (populated array + None + empty array)

Misc

Adds regression coverage to ensure stable handling of ndarray-backed array feature columns from Athena/Arrow going forward.


Open in Devin Review

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@SIDDHESH1564
SIDDHESH1564 requested a review from a team as a code owner April 24, 2026 18:25
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

SIDDHESH1564 added 2 commits April 25, 2026 00:08
@SIDDHESH1564
fix: Handle numpy ndarray in Array(String) materialization
62f947a
Signed-off-by: Siddhesh Khairnar <[email protected]>
@SIDDHESH1564
fix: Use NULL_TIMESTAMP_INT_VALUE for UNIX_TIMESTAMP_LIST default
e754d58
Signed-off-by: Siddhesh Khairnar <[email protected]>
@SIDDHESH1564
SIDDHESH1564 force-pushed the fix/ndarray-array-string-materialization branch from db3a30b to e754d58 Compare April 24, 2026 18:39
@ntkathole

ntkathole commented Apr 25, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

@SIDDHESH1564 dupe of #6324 ? Feel free to review other PR

SIDDHESH1564 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SIDDHESH1564
fix: correct test expectations for empty ndarray list conversion
c4b6398
Signed-off-by: Siddhesh Khairnar <[email protected]>
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SIDDHESH1564
fix: null semantics for empty ndarray and revert set-type validation …
c77302a
…change

Signed-off-by: Siddhesh Khairnar <[email protected]>
@SIDDHESH1564
SIDDHESH1564 force-pushed the fix/ndarray-array-string-materialization branch from a7d3fc5 to c77302a Compare April 25, 2026 18:43
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

SIDDHESH1564 and others added 3 commits April 26, 2026 00:21
@SIDDHESH1564
fix: only convert empty ndarray to null, preserve empty Python lists
e6c69d5
Signed-off-by: Siddhesh Khairnar <[email protected]>
@SIDDHESH1564
Merge branch 'master' into fix/ndarray-array-string-materialization
6434228
@SIDDHESH1564
Merge branch 'master' into fix/ndarray-array-string-materialization
99644e4
@alan-gauthier-jt alan-gauthier-jt mentioned this pull request May 7, 2026
fix: Handle array of strings columns in Athena materialization #6324
Merged
@SIDDHESH1564 SIDDHESH1564 closed this Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devin-ai-integration devin-ai-integration[bot] devin-ai-integration[bot] left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bug: TypeError / ValueError when materializing Array(String) feature views with Athena offline store

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@SIDDHESH1564 @ntkathole
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.