GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — feat: Add Vector Search support to MongoDBOnlineStore by caseyclements · Pull Request #6344 · feast-dev/feast Conversation
|
|
Note: This will be rebased onto master. I mistakenly built from #6138 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
4f99bd7 to
dcf08d1
Compare
dcf08d1 to
ebe08c5
Compare
| "pytest-asyncio<=0.24.0", | ||
| "py>=1.11.0", | ||
| "testcontainers==4.9.0", | ||
| "testcontainers==4.15.0rc2", |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
need to update requirements using make lock-python-dependencies-all, doesn't using stable release help ?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
beaa3e9 to
d1e5348
Compare
- Extend MongoDBOnlineStoreConfig with VectorStoreConfig (vector_enabled, similarity, vector_index_wait_timeout) - Auto-create/drop Atlas vector search indexes in update() for feature views with vector_index=True fields - Implement retrieve_online_documents_v2 using \ aggregation - Add MongoDBAtlasOnlineStoreCreator using MongoDBAtlasLocalContainer from testcontainers-python fork - Add integration tests for index lifecycle, write+retrieve round-trip, top_k limiting, and teardown cleanup Signed-off-by: Casey Clements <[email protected]>
…_WAIT - Explicitly create collection in _ensure_vector_indexes before calling create_search_index (Atlas requires it to exist) - Restructure tests to share store instance and setup via module-scoped fixture - Add INDEX_WAIT (default 5s) for Atlas Search eventual consistency after writes Signed-off-by: Casey Clements <[email protected]>
- Test write_data now uses np.float32 to match Array(Float32) schema - retrieve_online_documents_v2 coerces embedding to native Python floats before passing to \, avoiding BSON encoding errors with numpy float types Signed-off-by: Casey Clements <[email protected]>
…tion test - Add unit tests for retrieve_online_documents_v2 error paths: vector_enabled=False, embedding=None, no vector_index fields, missing vector_length - Add idempotency integration test: calling update() twice does not duplicate indexes - Add vector_index_wait_poll_interval config option - Change _wait_for_index_ready to raise TimeoutError instead of silently continuing on timeout Signed-off-by: Casey Clements <[email protected]>
…ntainer Replace MongoDbContainer with MongoDBAtlasLocalContainer in MongoDB offline store unit tests. This uses the mongodb/mongodb-atlas-local image which includes Atlas Search services, enabling future vector search testing. - Bump testcontainers from 4.9.0 to 4.15.0rc2 - Switch test fixture to MongoDBAtlasLocalContainer - Simplify connection string fixture to use get_connection_url() Signed-off-by: Casey Clements <[email protected]>
bool is a subclass of int in Python, so isinstance(True, int) returns True. Move the bool check before int so boolean values are correctly converted to ValueProto(bool_val=...) instead of ValueProto(int64_val=...). Signed-off-by: Casey Clements <[email protected]>
Add docstring to _ensure_vector_indexes explaining the current one-index-per-field design and noting that a single composite index with multiple field definitions would reduce cluster-wide index count and memory overhead. Signed-off-by: Casey Clements <[email protected]>
Run 'make lock-python-dependencies-all' to update all lockfiles to reflect testcontainers>=4.15.0rc2 and mongodb in ci extras. Signed-off-by: Casey Clements <[email protected]>
d1e5348 to
81aea6e
Compare
|
@ntkathole - This is ready for your review/merge. TIA! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Looks good
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@caseyclements Let's update the pixi lock file to fix failures and will merge |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
make lock-python-dependencies-all and pixi lock Signed-off-by: Casey Clements <[email protected]>
I've updated in my environment - MacOS. CI is still failing. What needs to be done? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@caseyclements seems there are conflicts with master |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Oh. Of course. Thanks. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Signed-off-by: Casey Clements <[email protected]>
|
@ntkathole Are we good to merge? Failures appear to be unrelated. ● Here's the summary of all 4 failures:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@caseyclements snowflake is expected but mysql failures are due to testcontainer version upgrade (testcontainers/testcontainers-python#739) Change conftest.py to include dialect: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
The testcontainers upgrade changed the default MySQL dialect, causing 'No module named MySQLdb' errors. Explicitly set dialect='pymysql' in all MySqlContainer instantiations. Signed-off-by: Casey Clements <[email protected]>
There was an error while loading. Please reload this page.
# [0.63.0](v0.62.0...v0.63.0) (2026-05-04) ### Bug Fixes * Add project filter to apply_data_source and delete_data_source (closes [#6206](#6206)) ([#6322](#6322)) ([96562c4](96562c4)) * Add project_id filter to SnowflakeRegistry UPDATE path ([#6243](#6243)) ([6658b71](6658b71)), closes [#6208](#6208) [#6208](#6208) * Add subprocess timeouts to prevent test_e2e_local hanging on Dask atexit handler ([3de6556](3de6556)) * Ambiguous truth value of array during materialization ([#6259](#6259)) ([d0c8984](d0c8984)) * Auto-detect GCS/S3 registry store when registry is passed as string ([#6260](#6260)) ([7ebcf03](7ebcf03)) * **bigquery:** Prefer query over table in get_table_query_string ([#6360](#6360)) ([77ed779](77ed779)), closes [#6200](#6200) * correct project_id scoping in get_user_metadata and delete_project ([0c469a7](0c469a7)) * disable Redis RDB persistence in test deployments ([44cd682](44cd682)) * Disable snowflake tests temporarily in CI ([#6356](#6356)) ([31d5a98](31d5a98)) * Filter empty SQL commands at execute_snowflake_statement call sites ([#6249](#6249)) ([92ffbb9](92ffbb9)) * Fix five bugs in milvus online store ([#6275](#6275)) ([212504b](212504b)) * Fix issue with apply feature view ([835cda8](835cda8)) * Fix streaming materialization for exotic sources with lazy UDF pipelines ([c07972d](c07972d)) * Handle missing features gracefully instead of panicking ([7d00b3a](7d00b3a)) * Harden informer cache with label selectors and memory optimizations ([#6242](#6242)) ([3f11356](3f11356)) * **helm:** Avoid nil pointer for metrics.enabled inside podAnnotations ([#6251](#6251)) ([c833f1a](c833f1a)) * Include git in feast server image ([fb03c46](fb03c46)) * Include StreamFeatureView in freshness metric ([#6269](#6269)) ([463f16c](463f16c)) * Pre-create S3A event log dir before SparkContext init ([#6317](#6317)) ([9feca77](9feca77)) * Remote Online Store Type Inference Error with All-NULL Columns ([#6063](#6063)) ([de67bdd](de67bdd)) * Remove selector with kustomize overlay using a JSON 6902 patch ([9107a43](9107a43)) * Resolve multiple bugs in SnowflakeRegistry and Snowflake connection handling ([#6315](#6315)) ([7e66a2e](7e66a2e)) * **spark:** BatchFeatureView with TransformationMode.PYTHON now reads all source columns ([a310eaf](a310eaf)) * **spark:** Use SELECT * when feature_name_columns is empty in pull_all_from_table_or_query ([e1b1d2d](e1b1d2d)) * Support pandas mode in feature builder and fix dask column extraction ([863315e](863315e)) * support SQL string as entity_df in RemoteOfflineStore.get_historical_features ([c559889](c559889)) * Wrap LocalOutputNode return value in ArrowTableValue for consist… ([#6286](#6286)) ([a16cd55](a16cd55)) ### Features * Add agent skills and Cursor/Claude rules for Feast development ([312eea3](312eea3)) * Add feature view versioning support to FAISS online store ([b36acb7](b36acb7)) * Add feature view versioning support to Redis and DynamoDB online stores ([#6257](#6257)) ([edf25af](edf25af)), closes [#6164](#6164) [#6163](#6163) * Add optional 'org' in feature view ([#6288](#6288)) ([#6301](#6301)) ([608b105](608b105)) * Add RaySource, to_ray_dataset first-class method, docs, and tests ([1c98157](1c98157)) * Add TLS support for Go Feature Server ([#6229](#6229)) ([28a58d0](28a58d0)) * Add Vector Search support to MongoDBOnlineStore ([#6344](#6344)) ([c102738](c102738)) * Add versioning support to Milvus online store ([#6330](#6330)) ([3268ced](3268ced)) * Addresses performance issues in the Redis online store ([2e50da0](2e50da0)) * Allow to set gpu for ray ([5580ab4](5580ab4)) * Bump redis-py version cap from <5 to <8 ([#6339](#6339)) ([9538180](9538180)) * Expose feature_server, materialization, and openlineage configuration via FeatureStore CRD ([ec6ecfd](ec6ecfd)) * Make online_write_batch_size configurable in MaterializationConfig ([#6268](#6268)) ([d41becf](d41becf)) * Make udf optional if agg defined ([#5689](#5689)) ([#6328](#6328)) ([f630056](f630056)) * MongoDB offline store ([#6138](#6138)) ([8eebad7](8eebad7)) * Optional input_schema for ODFV ([#6308](#6308)) ([#6312](#6312)) ([f08b4e8](f08b4e8)) * Provision minimal TokenReview RBAC for OIDC auth and add SSL error logging in token parser ([#6240](#6240)) ([dca57e8](dca57e8)) * **spark:** Add compute-on-read support for BatchFeatureView in get_… ([#6357](#6357)) ([630d9f8](630d9f8))
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.