GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: Adding Milvus demo to examples by franciscojavierarceo · Pull Request #4910 · feast-dev/feast

Conversation

@franciscojavierarceo

@franciscojavierarceo franciscojavierarceo commented Jan 10, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

What this PR does / why we need it:

This pull request introduces several updates and new features to the Feast repository, particularly around the integration of Milvus as an online store. Below is a summary of the key changes:

Documentation Updates

  • Updated Overview of Online Stores: Added Milvus to the matrix of supported functionalities in the overview.md file under docs/reference/online-stores/.

New Example Files

  • RAG Example: Introduced a new example for RAG (Retrieval-Augmented Generation) under examples/rag/.
    • Added __init__.py.
    • Added README.md.
    • Added feature_repo/ directory containing:
      • __init__.py
      • example_repo.py
      • feature_store.yaml
      • test_workflow.py
    • Added milvus-quickstart.ipynb notebook demo containing documentation for data scientists to easily get started

Code Updates

  • Milvus Online Store Integration:
    • Renamed _get_collection method to _get_or_create_collection in milvus.py.
    • Updated method calls to use _get_or_create_collection instead of _get_collection.
    • Added logic to handle vector fields and search metrics.
    • Improved handling of Proto values and serialization in retrieve_online_documents_v2 method.

Detailed Changes

  1. Documentation:

    • Added Milvus to the list of supported online stores in the feature matrix.
  2. Examples:

    • Added new example files under examples/rag/.
  3. Milvus Integration:

    • Method Renaming: Renamed _get_collection to _get_or_create_collection.
    • Vector Field Handling: Added handling for vector fields and search metrics within the _get_or_create_collection method.
    • Method Updates: Updated the following methods to use _get_or_create_collection:
      • online_write_batch
      • update
      • retrieve_online_documents_v2
    • Proto Value Handling: Improved the handling of Proto values and serialization in retrieve_online_documents_v2.

Which issue(s) this PR fixes:

#4364

Misc

N/A

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@franciscojavierarceo
franciscojavierarceo force-pushed the milvus-demo branch 2 times, most recently from 3731e17 to b7c3426 Compare January 10, 2025 21:59
@franciscojavierarceo
franciscojavierarceo force-pushed the milvus-demo branch 2 times, most recently from ce41480 to 8360bbb Compare January 29, 2025 01:02
@franciscojavierarceo franciscojavierarceo added the ok-to-test label Jan 29, 2025
@franciscojavierarceo
franciscojavierarceo marked this pull request as ready for review January 29, 2025 02:32
@franciscojavierarceo
franciscojavierarceo requested a review from a team as a code owner January 29, 2025 02:32
@franciscojavierarceo
franciscojavierarceo requested review from dmartinol and lokeshrangineni January 29, 2025 03:17
dmartinol
dmartinol reviewed Jan 29, 2025
View reviewed changes
Comment thread examples/rag/README.md Outdated
@@ -0,0 +1,29 @@
# Feast Quickstart
If you haven't already, check out the quickstart guide on Feast's website (http://docs.feast.dev/quickstart), which

@dmartinol dmartinol Jan 29, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This README seems a generic quickstart guide with no specific connections with the RAG function or the included example.
I assume that it will be more clear once we run it and see what happens, but I believe than a dev experience driven by a notebook may help, WDYT?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@franciscojavierarceo franciscojavierarceo Jan 29, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! I'll update to include a notebook! Will update this README as well.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@franciscojavierarceo franciscojavierarceo Jan 30, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know what you think!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
franciscojavierarceo added 16 commits January 29, 2025 14:38
@franciscojavierarceo
checking in progress but this Pr still is not ready yet
468368a
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
feat: Adding new method to FeatureStore to allow more flexible retrie…
8f26494
…val of features from vector similarity search

Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
Adding requested_features back into online_store
ddc963b
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
feat: Adding RAG demo displaying Milvus usage for RAG
006e897
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
uploading sample data and updated yaml
a1a9408
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
updating workflow
4fba66c
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
updated example
c4736b7
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
removing modified files
f7393a7
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
reverting postgres change
d0a1e30
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
updating test_workflow
0d4c351
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
updated and fixed bug
7c006b1
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
fixing a bad merge/rebase
63aa43b
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
updated linter because of latest nistall
aa81350
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
reverting feature store change
e2a1949
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
adding logging of local milvus back
017ff54
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
Updating readme and adding notebook
ac96e36
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
franciscojavierarceo force-pushed the milvus-demo branch from 1599ac4 to ac96e36 Compare January 29, 2025 22:24
@franciscojavierarceo
updated readme
c629a42
Signed-off-by: Francisco Javier Arceo <[email protected]>
@franciscojavierarceo
franciscojavierarceo requested review from a team January 30, 2025 14:54
@franciscojavierarceo
franciscojavierarceo requested review from haoxu0 and shuchu and removed request for a team January 30, 2025 14:54
shuchu
shuchu approved these changes Jan 30, 2025
View reviewed changes

@shuchu shuchu left a comment

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@franciscojavierarceo
franciscojavierarceo merged commit 2daf852 into master Jan 30, 2025
franciscojavierarceo pushed a commit that referenced this pull request Feb 4, 2025
@feast-ci-bot
chore(release): release 0.44.0
aa7224a
# [0.44.0](v0.43.0...v0.44.0) (2025-02-04)

### Bug Fixes

* Adding periodic check to fix the sporadic failures of the operator e2e tests.  ([#4952](#4952)) ([1d086be](1d086be))
* Adding the feast-operator/bin to the .gitignore directory. Somehow it… ([#5005](#5005)) ([1a027ee](1a027ee))
* Changed Env Vars for e2e tests ([#4975](#4975)) ([fa0084f](fa0084f))
* Fix GitHub Actions to pass authentication ([#4963](#4963)) ([22b9138](22b9138)), closes [#4937](#4937) [#4939](#4939) [#4941](#4941) [#4940](#4940) [#4943](#4943) [#4944](#4944) [#4945](#4945) [#4946](#4946) [#4947](#4947) [#4948](#4948) [#4951](#4951) [#4954](#4954) [#4957](#4957) [#4958](#4958) [#4959](#4959) [#4960](#4960) [#4962](#4962)
* Fix showing selected navigation item in UI sidebar ([#4969](#4969)) ([8ac6a85](8ac6a85))
* Invalid column names in get_historical_features when there are field mappings on join keys ([#4886](#4886)) ([c9aca2d](c9aca2d))
* Read project data from the 'projects' key while loading the registry state in the Feast UI ([#4772](#4772)) ([cb81939](cb81939))
* Remove grpcurl dependency from Operator ([#4972](#4972)) ([439e0b9](439e0b9))
* Removed the dry-run flag to test and we will add it back later. ([#5007](#5007)) ([d112b52](d112b52))
* Render UI navigation items as links instead of buttons ([#4970](#4970)) ([1267703](1267703))
* Resolve Operator CRD bloat due to long field descriptions ([#4985](#4985)) ([7593bb3](7593bb3))
* Update manifest to add feature server image for odh ([#4973](#4973)) ([6a1c102](6a1c102))
* Updating release workflows to refer to yml instead of yaml ([#4935](#4935)) ([02b0a68](02b0a68))
* Use locally built feast-ui package in dev feature-server image ([#4998](#4998)) ([0145e55](0145e55))

### Features

* Added OWNERS file for OpenshiftCI ([#4991](#4991)) ([86a2ee8](86a2ee8))
* Adding Milvus demo to examples ([#4910](#4910)) ([2daf852](2daf852))
* Adding retrieve_online_documents endpoint ([#5002](#5002)) ([6607d3d](6607d3d))
* Adding support to return additional features from vector retrieval for Milvus db ([#4971](#4971)) ([6ce08d3](6ce08d3))
* Creating/updating the stable branch after the release. ([#5003](#5003)) ([e9b53cc](e9b53cc))
* Implementing online_read for MilvusOnlineStore ([#4996](#4996)) ([92dde13](92dde13))
* Improve exception message for unsupported Snowflake data types ([#4779](#4779)) ([5992364](5992364))
* Operator add feast ui deployment ([#4930](#4930)) ([b026d0c](b026d0c))
* Updating documents to highlight v2 api for Vector Similarity Se… ([#5000](#5000)) ([32b82a4](32b82a4))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dmartinol dmartinol dmartinol left review comments
@shuchu shuchu shuchu approved these changes
@lokeshrangineni lokeshrangineni Awaiting requested review from lokeshrangineni
@haoxu0 haoxu0 Awaiting requested review from haoxu0 haoxu0 was automatically assigned from feast-dev/maintainers

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@franciscojavierarceo @shuchu @dmartinol
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.