GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — fix: Infer vector width when vector_length is undeclared by haoxu0 · Pull Request #6935 · feast-dev/feast Conversation
Collaborator
Addresses item 3 of #6907 — but not the way the issue proposed, because that way does not work. Why "require
|
| case | before | after |
|---|---|---|
vector_length declared |
enforced | enforced (unchanged) |
| undeclared, uniform widths | skipped | passes |
| undeclared, ragged widths | skipped | rejected |
undeclared, fixed_size_list |
skipped | no check needed — uniform by construction |
| undeclared, all-null column | skipped | no-op, nothing to infer from |
vector_index=False |
skipped | skipped |
The gate also moves from vector_length to vector_index, since vector_index is the attribute that actually says "this field is a vector".
Error messages distinguish the two sources, so an inferred failure is not mistaken for a declared one:
Row 2: Vector length 7 does not match expected 4, inferred from the first row
for feature 'embedding' in feature view 'fv'.
11 new tests in test_vector_length_validation.py (32 total), covering both the Arrow and DataFrame paths: ragged rejected, uniform passes, declared still wins over inference, fixed_size_list needs no check, nulls neither infer nor compare, all-null is a no-op, non-sequence still caught, and vector_index=False untouched.
Three of them fail against master and pass here.
The important check for this PR is that inference does not break the 29 undeclared sites. I diffed failure names across the full unit suite rather than comparing counts, since the suite has flaky members:
with this change : 41 failures
master : 41 failures
only with change : (none)
That is also the evidence that none of those sites produce ragged columns in practice.
ruff check / ruff format --check clean. mypy unchanged — same 2 pre-existing errors at feature_store.py:2123.
Field.__eq__With this, #6907 can be closed.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## master #6935 +/- ## ========================================== + Coverage 48.64% 48.70% +0.05% ========================================== Files 427 427 Lines 53864 53924 +60 Branches 7849 7863 +14 ========================================== + Hits 26203 26262 +59 + Misses 25792 25789 -3 - Partials 1869 1873 +4
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Addresses item 3 of feast-dev#6907, which asked for vector_length to be required when vector_index=True. That specific change is not viable, so this takes a non-breaking route to the same goal. Requiring it would break roughly 29 call sites that declare vector_index=True without vector_length, including the bundled feast init -t rag and -t ray_rag templates, several entries under examples/, and four online stores that deliberately fall back to 512. The onboarding path would stop working. The underlying problem is still real: vector_length defaults to 0 and 0 meant skip, so the common case was unvalidated. Instead of skipping, the width of the first non-null row now becomes the contract for the rest of the table. An approximate nearest neighbour index requires a fixed width, so a ragged vector column is a defect whether or not anyone declared a length, and inferring is strictly better than skipping. A declared vector_length still wins over inference, so nothing about the declared case changes. Fixed-size list columns are uniform by construction, so an undeclared one needs no check at all. Null rows carry no width and are neither used for inference nor compared. The gate moves from vector_length to vector_index, since vector_index is what says the field is a vector. Verified no regressions by diffing failure names across the full unit suite before and after: 41 pre-existing failures on both sides, none introduced, which also confirms none of the undeclared sites produce ragged columns. Signed-off-by: hao-xu5 <[email protected]>
63b2d5b to
bb0c5a1
Compare
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.