GeezerVison is showing github.com — feat: Add text analysis tags to the Milvus online store by jiwidi · Pull Request #7143 · feast-dev/feast
Conversation
What this PR does / why we need it:
Adds two field tags so the Milvus online store can set up text analysis on a String feature when
it creates a collection, in the same way milvus.index declares scalar indexes (#7120):
Field(
name="title",
dtype=String,
tags={
"milvus.analyzer_params": '{"tokenizer": "whitespace"}',
"milvus.enable_match": "true",
},
),
milvus.analyzer_params: JSON object with the field's analyzer. Turns on enable_analyzer.
milvus.enable_match: "true" builds the token index that Milvus TEXT_MATCH filters need.
- Both apply only to
String features and need Milvus 2.5 or later.
- Feast does not query the tokens: Feast filters have no
TEXT_MATCH operator and
retrieve_online_documents_v2 still filters a query_string with LIKE. Clients that query
the collection directly can use them. A follow-up will add BM25 field tags on top of the
analyzer.
- Like the other creation-time tags, the settings apply only when Feast creates a collection.
Unlike them, Feast does not compare them with an existing collection or warn when they differ.
_nullable_tag becomes _boolean_tag(field, tag) so both boolean tags share one parser and
error message.
Which issue(s) this PR fixes:
N/A
Checks
- I've made sure the tests are passing.
- My commits are signed off (
git commit -s)
- My PR title follows conventional commits format
Testing Strategy
-
Unit tests
-
Integration tests
-
Manual tests
-
Testing is not required for this change
-
Unit: the created schema per tag combination, and validation (non-String feature, invalid
boolean, analyzer JSON that is not an object).
-
Integration (test_milvus_remote.py, against Milvus 2.5.27, 2.6.27 and 3.0.2): a whitespace analyzer
keeps wool-blend as one token, so TEXT_MATCH finds wool-blend and not wool.
-
Milvus Lite accepts both tags but does not apply them (its TEXT_MATCH always uses the standard
analyzer), so only the server tests cover the behaviour.
Misc
Milvus 2.6 makes new rows visible to TEXT_MATCH about 2 seconds after the write, even with
Strong consistency, so the integration test retries that query.
Written with @jonasfagerlund, who is a co-author of the commit. A BM25 PR builds on this one.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
The milvus.analyzer_params and milvus.enable_match field tags set the
analyzer and TEXT_MATCH on a String feature when Feast creates the
collection. Feast does not query the tokens; clients that query Milvus
directly can.
Co-authored-by: jonasfagerlund <[email protected]>
Signed-off-by: jonasfagerlund <[email protected]>
Signed-off-by: jiwidi <[email protected]>
|
⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.
✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.31%. Comparing base (e81e69d) to head (69b54e2).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

@@ Coverage Diff @@
## master #7143 +/- ##
==========================================
+ Coverage 52.29% 52.31% +0.01%
==========================================
Files 459 459
Lines 58090 58104 +14
Branches 8560 8564 +4
==========================================
+ Hits 30381 30395 +14
Misses 25629 25629
Partials 2080 2080
Continue to review full report in Codecov by Harness.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e81e69d...69b54e2. Read the comment docs.
🚀 New features to boost your workflow:
- ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This branch has not been deployed
No deployments
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|