GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: Add a schema check and mutation timeout to the Milvus online store by jiwidi · Pull Request #7118 · feast-dev/feast

Conversation

@jiwidi

@jiwidi jiwidi commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Two small additions to how the Milvus online store treats a collection that already exists, both
behind settings that are off by default.

Schema check. A collection's schema only applies when Feast creates it. Until now Feast used an
existing collection as it was, so a feature view that gained a field, lost a field or changed its
vector dimension failed later with a Milvus error at write or read time, or kept writing into a
stale schema. Feast now compares the fields of an existing collection with the schema the feature
view declares, the first time a process touches the collection and on every feast apply. By
default it logs a warning that names each difference, as it already does for a missing partition key. With
fail_on_schema_mismatch: true it raises a ValueError instead, so the mismatch is caught at
feast apply. Field types are not compared, because reads and writes adapt to the types a
collection has. The Redis, Postgres and SQLite online stores do not have this problem: a feature is
a hash field or a row there, so a changed feature view needs no schema change.

Mutation timeout. PyMilvus retries a failed upsert or delete and, without a timeout, a call can
wait for a long time, for example while a cluster refuses writes at its memory quota. Streaming
writers need a bound so a stuck call fails and the batch can be retried. mutation_timeout is
passed as the PyMilvus timeout of every upsert and delete. PyMilvus applies it to each attempt
and stops retrying once the elapsed time exceeds it, so a call ends within the timeout plus one
attempt. When unset, nothing is sent and PyMilvus keeps its default retry policy. The Redis online store takes the same control from socket_timeout and
socket_connect_timeout in its connection_string.

online_store:
  type: milvus
  fail_on_schema_mismatch: true
  mutation_timeout: 60

Which issue(s) this PR fixes:

N/A

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests

  • Integration tests

  • Manual tests

  • Testing is not required for this change

  • Unit (mocked client): a missing field, an extra field and a changed vector dimension are logged,
    or raised with fail_on_schema_mismatch, and the placeholder vector of older collections is
    ignored; an invalid mutation_timeout is rejected by the config.

  • Unit (Milvus Lite): a changed feature view applied again by the same process warns, then
    raises with the setting; timeout is sent to upserts and deletes only when mutation_timeout is set.

  • Server: the schema check raises on a real collection; writes and deletes with a timeout succeed.

Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). The server tests in
sdk/python/tests/integration/online_store/test_milvus_remote.py pass against a local Milvus
3.0.2 server. The existing Milvus unit and server tests pass unchanged.

Misc

Follows #7082, #7083 and #7084, which brought the Milvus online store the row deletion, partition
options and write-batch ordering that the Redis online store has, and Simon Hearne's series #6882,
#6895, #6917, #6921 and #6940.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@jiwidi
feat: Add a schema check and mutation timeout to the Milvus online store
b59e834
A collection's schema only applies when Feast creates it. Until now Feast
used an existing collection as it was, so a feature view that gained a
field, lost a field or changed its vector dimension failed later with a
Milvus error at write or read time, or kept writing into a stale schema.

Feast now compares the fields of an existing collection with the schema
the feature view declares, when a process first touches the collection
and on every apply. By default it logs a warning that names each
difference, as it already does for a missing partition key. With
fail_on_schema_mismatch: true it raises a ValueError instead, so the
mismatch is caught at feast apply. Field types are not compared, because
reads and writes adapt to the types a collection has.

PyMilvus retries a failed upsert or delete and, without a timeout, a call
can wait for a long time, for example while a cluster refuses writes at
its memory quota. mutation_timeout is passed as the PyMilvus timeout of
every upsert and delete. PyMilvus applies it to each attempt and stops
retrying once the elapsed time exceeds it, so a call ends within the
timeout plus one attempt. When unset, nothing is sent and PyMilvus keeps
its default retry policy.

Signed-off-by: jiwidi <[email protected]>
@jiwidi
jiwidi requested a review from a team as a code owner November 9, 2026 13:14
@codecov-commenter

codecov-commenter commented Nov 9, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 97.67442% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 52.28%. Comparing base (ff8abe0) to head (a455bad).

Files with missing lines Patch % Lines
.../infra/online_stores/milvus_online_store/milvus.py 97.67% 0 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7118      +/-   ##
==========================================
+ Coverage   52.25%   52.28%   +0.02%     
==========================================
  Files         459      459              
  Lines       58045    58084      +39     
  Branches     8550     8559       +9     
==========================================
+ Hits        30331    30368      +37     
  Misses      25640    25640              
- Partials     2074     2076       +2     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 53.80% <97.67%> (+0.02%) ⬆️
Files with missing lines Coverage Δ
.../infra/online_stores/milvus_online_store/milvus.py 88.67% <97.67%> (+0.53%) ⬆️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff8abe0...a455bad. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jiwidi
Merge remote-tracking branch 'upstream/master' into feat/milvus-schem…
4176b42
…a-check-and-timeout

Signed-off-by: jiwidi <[email protected]>

# Conflicts:
#	docs/reference/online-stores/milvus.md
#	sdk/python/feast/infra/online_stores/milvus_online_store/milvus.py
#	sdk/python/tests/integration/online_store/test_milvus_remote.py
#	sdk/python/tests/unit/infra/online_store/test_milvus_online_store.py
@jiwidi jiwidi mentioned this pull request Nov 10, 2026
feat: Store arrays and Json natively in every new Milvus collection #7123
Merged
5 of 7 tasks
@ntkathole

ntkathole commented Nov 11, 2026

Copy link
Copy Markdown
Member

@jiwidi Please rebase and resolve conflicts

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jiwidi added 2 commits November 11, 2026 10:29
@jiwidi
Merge remote-tracking branch 'upstream/master' into feat/milvus-schem…
59236f5
…a-check-and-timeout

Signed-off-by: jiwidi <[email protected]>
@jiwidi
Merge remote-tracking branch 'upstream/master' into feat/milvus-schem…
a455bad
…a-check-and-timeout

Signed-off-by: jiwidi <[email protected]>
@jiwidi

jiwidi commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

@ntkathole

Hi!

Done !

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ntkathole
ntkathole approved these changes Nov 12, 2026
View reviewed changes
@ntkathole ntkathole added the ok-to-test label Nov 12, 2026
@ntkathole
ntkathole merged commit f540542 into feast-dev:master Nov 12, 2026
35 of 37 checks passed

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ntkathole ntkathole ntkathole approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@jiwidi @codecov-commenter @ntkathole
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.