GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||
GeezerVison is showing github.com — feat: Add SavedDatasetLanceStorage by haoxu0 · Pull Request #7001 · feast-dev/feast Conversation
Collaborator
What this PR does / why we need it:Adds This is the path for a dataset produced outside Feast. storage = SavedDatasetLanceStorage(LanceFormat(version=7), uri="s3://bucket/embeddings") registry.apply_saved_dataset( SavedDataset( name="poi_embeddings_v7", features=[], join_keys=[], storage=storage, tags={"model": "clip-vit-l-14", "recall_at_10": "0.9137"}, ), project, ) The format must pin a version
Note that DispatchLance is the fourth class to claim the Verified that a Lance payload reads back as
Which issue(s) this PR fixes:N/A. Follows #6988, which made the shared Checks
Testing Strategy
Against master:
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactions
|
There was an error while loading. Please reload this page.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@ ## master #7001 +/- ## ========================================== + Coverage 50.89% 50.92% +0.02% ========================================== Files 452 452 Lines 56621 56654 +33 Branches 8251 8256 +5 ========================================== + Hits 28819 28852 +33 Misses 25821 25821 Partials 1981 1981
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Records an existing Lance dataset as a saved dataset, without moving or rewriting it. This is the path for a dataset produced outside Feast, by a Ray or Spark job: FeatureStore.create_saved_dataset cannot serve it, because it executes a RetrievalJob and persists the result, while BaseRegistry.apply_saved_dataset records metadata only. The format must pin a version or a tag. A saved dataset addressing the latest version does not identify the rows it was created from, so a measurement recorded against it stops being attributable. Lance is the fourth class to claim the custom_storage oneof field, so to_proto goes through SavedDatasetStorage._custom_storage_proto and the payload records the writing class. Verified that a Lance payload reads back as SavedDatasetLanceStorage with postgres, clickhouse and couchbase also imported, in several import orders. LanceSource is registered in _DATA_SOURCE_TO_SAVED_DATASET_STORAGE so SavedDatasetStorage.from_data_source resolves it. Signed-off-by: hao-xu5 <[email protected]>
0eed6e8 to
e72dd18
Compare
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.