GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||
GeezerVison is showing github.com — feat: Add a schema check and mutation timeout to the Milvus online store by jiwidi · Pull Request #7118 · feast-dev/feast Conversation
Contributor
What this PR does / why we need it:Two small additions to how the Milvus online store treats a collection that already exists, both Schema check. A collection's schema only applies when Feast creates it. Until now Feast used an Mutation timeout. PyMilvus retries a failed upsert or delete and, without a timeout, a call can online_store: type: milvus fail_on_schema_mismatch: true mutation_timeout: 60 Which issue(s) this PR fixes:N/A Checks
Testing Strategy
Unit tests run on Milvus Lite 3.2.1 (pymilvus 3.0.2). The server tests in MiscFollows #7082, #7083 and #7084, which brought the Milvus online store the row deletion, partition
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactionsA collection's schema only applies when Feast creates it. Until now Feast used an existing collection as it was, so a feature view that gained a field, lost a field or changed its vector dimension failed later with a Milvus error at write or read time, or kept writing into a stale schema. Feast now compares the fields of an existing collection with the schema the feature view declares, when a process first touches the collection and on every apply. By default it logs a warning that names each difference, as it already does for a missing partition key. With fail_on_schema_mismatch: true it raises a ValueError instead, so the mismatch is caught at feast apply. Field types are not compared, because reads and writes adapt to the types a collection has. PyMilvus retries a failed upsert or delete and, without a timeout, a call can wait for a long time, for example while a cluster refuses writes at its memory quota. mutation_timeout is passed as the PyMilvus timeout of every upsert and delete. PyMilvus applies it to each attempt and stops retrying once the elapsed time exceeds it, so a call ends within the timeout plus one attempt. When unset, nothing is sent and PyMilvus keeps its default retry policy. Signed-off-by: jiwidi <[email protected]>
|
There was an error while loading. Please reload this page.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## master #7118 +/- ## ========================================== + Coverage 52.25% 52.28% +0.02% ========================================== Files 459 459 Lines 58045 58084 +39 Branches 8550 8559 +9 ========================================== + Hits 30331 30368 +37 Misses 25640 25640 - Partials 2074 2076 +2
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…a-check-and-timeout Signed-off-by: jiwidi <[email protected]> # Conflicts: # docs/reference/online-stores/milvus.md # sdk/python/feast/infra/online_stores/milvus_online_store/milvus.py # sdk/python/tests/integration/online_store/test_milvus_remote.py # sdk/python/tests/unit/infra/online_store/test_milvus_online_store.py
|
@jiwidi Please rebase and resolve conflicts |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…a-check-and-timeout Signed-off-by: jiwidi <[email protected]>
…a-check-and-timeout Signed-off-by: jiwidi <[email protected]>
|
Hi! Done ! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.