GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: Add native_numeric_types to the Milvus online store by simonhearne · Pull Request #7025 · feast-dev/feast

Conversation

@simonhearne

@simonhearne simonhearne commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

The Milvus online store stores numeric and bool features as VARCHAR unless
enable_openai_compatible_store is set. Range filters (gt, gte, lt, lte) on those fields
compare strings, so "9" > "100" is true. The flag that fixes this is shared with the SQLite,
Postgres and Elasticsearch stores, and its name doesn't say that it controls numeric storage.

This PR adds native_numeric_types as a Milvus-specific name for the same behaviour. Either flag
stores numeric and bool features as native INT32, INT64, FLOAT, DOUBLE and BOOL fields.
The warning logged for numeric range filters on VARCHAR collections now points to the new flag.

online_store:
  type: milvus
  native_numeric_types: true

The docs describe the migration: set the flag, run feast teardown and feast apply, then
materialize again. Collections that haven't been recreated keep working, because Feast detects the
storage type of each collection.

If you would rather not add a Milvus-only alias for a cross-store flag, I can reduce this PR to
the docs and the updated warning.

Which issue(s) this PR fixes:

N/A

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests

  • Integration tests

  • Manual tests

  • Testing is not required for this change

  • Unit (Milvus Lite): field types with the default config, with native_numeric_types and with
    enable_openai_compatible_store; a gt filter compares numerically; values round-trip.

Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). The existing server tests in sdk/python/tests/integration/online_store/test_milvus_remote.py pass against a local Milvus 2.6.0, a local Milvus 3.0.2 server and Zilliz Cloud. The existing Milvus unit and universal integration tests pass unchanged.

Misc

Part of a series of Milvus online store improvements for Zilliz Cloud and production Milvus, following #6882, #6895, #6917, #6921 and #6940.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@simonhearne @claude
feat: Add native_numeric_types to the Milvus online store
fadb832
Milvus stores numeric features as VARCHAR unless
enable_openai_compatible_store is set, so range filters compare
lexicographically. That flag is shared by several online stores, and
its name doesn't say that for Milvus it only switches numeric storage.

Adds native_numeric_types as a Milvus-specific, clearly named way to
store numeric and bool features as native Milvus types. Either flag
enables it, so existing configs are unaffected. The warning for
numeric filters on VARCHAR collections now points to the new flag, and
the docs describe how to migrate existing collections.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Simon Hearne <[email protected]>
@simonhearne
simonhearne requested a review from a team as a code owner October 16, 2026 09:05
@codecov-commenter

codecov-commenter commented Oct 16, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.02%. Comparing base (1e5e90c) to head (19ba48c).
⚠️ Report is 13 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7025      +/-   ##
==========================================
+ Coverage   50.97%   51.02%   +0.04%     
==========================================
  Files         455      455              
  Lines       56994    56997       +3     
  Branches     8321     8321              
==========================================
+ Hits        29055    29082      +27     
+ Misses      25954    25934      -20     
+ Partials     1985     1981       -4     
Flag Coverage Δ
go-feature-server 30.58% <ø> (+0.20%) ⬆️
python-unit 52.48% <100.00%> (+0.03%) ⬆️
Files with missing lines Coverage Δ
.../infra/online_stores/milvus_online_store/milvus.py 79.41% <100.00%> (+3.24%) ⬆️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e5e90c...19ba48c. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@simonhearne
Merge branch 'master' into feat/milvus-native-numeric-types
675a536
@simonhearne

simonhearne commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

@ntkathole LMK if you have any thoughts on this PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@haoxu0 ChatGPT Codex Connector

haoxu0 commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

The numeric filter correctness fix and the backward-compatible alias look reasonable. One migration concern: the documented feast teardown → feast apply → rematerialize sequence drops the existing collection before it is repopulated, so this can create a serving outage and potentially lose online-only state. Could we add an explicit destructive / downtime warning and recommend a backup or staged migration (e.g. a new collection with cutover) for production deployments? Also, the current tests establish the newly created native-typed collection, but could we add a regression that creates a VARCHAR collection first, flips native_numeric_types=True without teardown, then verifies reads/writes still work and numeric range filters continue to warn/use the existing storage semantics? This would pin down the promised mixed old/new collection behavior.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@simonhearne @claude
docs: Add migration warning and correct VARCHAR filter behaviour
206309f
Teardown drops every collection in the project and loses online-only
rows, so the docs now warn about it and describe a staged cutover that
avoids it.

Numeric range filters on VARCHAR collections don't compare
lexicographically as the warning and docs said: Milvus rejects numeric
filter values on VARCHAR fields, and only string values compare as
text. Correct both, and add a regression test that turns on
native_numeric_types over an existing VARCHAR collection.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Simon Hearne <[email protected]>
@simonhearne

simonhearne commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@haoxu0 thank you for the review, both points are addressed in 206309f.
Let me know if there's anything else - could a maintainer rerun the failed unit-test-python check?

Migration warning. The Numeric types section now has a warning hint: feast teardown drops the collections of every feature view in the project, online reads fail until materialization finishes, and rows that only exist in the online store (push sources, write_to_online_store) are lost. The section also documents a staged cutover that needs no teardown: add a copy of the feature view under a new name (which creates a new native-typed collection), materialize it, move readers to it, then remove the old view. It points to Milvus Backup for anyone who still wants to tear down.

Mixed-collection regression. test_native_numeric_types_keeps_existing_varchar_collection creates a VARCHAR collection, turns on native_numeric_types without a teardown, and checks that the collection stays VARCHAR and that writes and online_read still round-trip.

Writing that test showed that the existing warning (and my docs) described the wrong behaviour on VARCHAR collections. A numeric filter value doesn't compare lexicographically: Milvus rejects it (comparisons between VarChar and Int64 are not supported on Milvus 3.0.2, with a similar error on Lite). Only a string value like "50" compares as text. The test now pins both cases and the warning, and I've corrected the warning text and docs to match. I left the filter behaviour itself unchanged: an error is safer than silently wrong results.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@simonhearne
Merge branch 'master' into feat/milvus-native-numeric-types
19ba48c
@simonhearne

simonhearne commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

@haoxu0 just following up. Both points from your review are addressed in 206309f: there's a warning before teardown plus a staged cutover in the docs, and a regression test for mixed VARCHAR collections. CI is now fully green.

If you're happy with it, could you approve and merge, or add ok-to-test if you'd like the integration suites to run first? I have a BM25 / hybrid search PR ready that builds on this one.

Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jiwidi jiwidi mentioned this pull request Nov 9, 2026
feat: Add native complex types and scalar index tags to the Milvus online store #7120
Merged
5 of 7 tasks
@simonhearne

simonhearne commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Closing this as superseded by #7120, which extends native storage to arrays and JSON behind enable_openai_compatible_store and includes the migration warning and existing-collection detection from the review here. A separate native_numeric_types flag would now only be an alias for it. Thanks @haoxu0 for the review.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@simonhearne simonhearne closed this Nov 10, 2026
@jiwidi jiwidi mentioned this pull request Nov 10, 2026
feat: Store arrays and Json natively in every new Milvus collection #7123
Merged
5 of 7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@simonhearne @codecov-commenter @haoxu0
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.