GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat: Adding write capabability to online store to on demand feature … by franciscojavierarceo · Pull Request #4418 · feast-dev/feast

feat: Adding write capabability to online store to on demand feature … #4418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
franciscojavierarceo wants to merge 45 commits into master from persistent-odfvs

Uh oh!

There was an error while loading. Please reload this page.
Closed

feat: Adding write capabability to online store to on demand feature … #4418

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
d03a895
merged changes
franciscojavierarceo Sep 24, 2024
e0f42c6
saving progress
franciscojavierarceo Aug 17, 2024
856cea2
merged changes to odfv
franciscojavierarceo Sep 24, 2024
f6a7133
linted
franciscojavierarceo Aug 18, 2024
fe387fc
adding the test needed to show the expected behavior
franciscojavierarceo Aug 18, 2024
646a124
updated test case
franciscojavierarceo Aug 21, 2024
b345e7e
saving progress
franciscojavierarceo Aug 21, 2024
16e68f0
merging
franciscojavierarceo Sep 24, 2024
24c4ae2
merged
franciscojavierarceo Sep 24, 2024
3d21881
merged
franciscojavierarceo Sep 24, 2024
7afaa84
merging
franciscojavierarceo Sep 24, 2024
436478a
adding the entity keys for now to do retrieval
franciscojavierarceo Aug 29, 2024
c78aead
adding entity to odfv
franciscojavierarceo Aug 29, 2024
fbebc63
checking in progress...getting closer
franciscojavierarceo Aug 29, 2024
5b3e2f5
may have to revert some of this...looks like the challenge is getting…
franciscojavierarceo Aug 31, 2024
7879e21
moving things around to make it easier to debug
franciscojavierarceo Sep 1, 2024
72caa15
debugging
franciscojavierarceo Sep 1, 2024
6d23889
merged
franciscojavierarceo Sep 24, 2024
b76bf4e
merging
franciscojavierarceo Sep 24, 2024
3cf0369
Rebasing and merging changes from other PR
franciscojavierarceo Sep 6, 2024
3a6dfc4
Merging changes continued
franciscojavierarceo Sep 7, 2024
1c37e54
update the _make_inference to include odfvs with writes in the update…
franciscojavierarceo Sep 7, 2024
2f9546f
have the table being written now...the create table happens in the Sq…
franciscojavierarceo Sep 8, 2024
c46e157
checking in progress
franciscojavierarceo Sep 9, 2024
580b77f
adding logs
franciscojavierarceo Sep 10, 2024
815a352
updating permissions
franciscojavierarceo Sep 10, 2024
2eda92c
going to error out on purpose
franciscojavierarceo Sep 10, 2024
e326e9b
adding unit test and merging changes
franciscojavierarceo Sep 18, 2024
a54b5f8
almost got everything working and type validation behaving
franciscojavierarceo Sep 18, 2024
029c9cf
cleaned up and have tests behaving
franciscojavierarceo Sep 18, 2024
e2c6b35
adding print
franciscojavierarceo Sep 21, 2024
adf147f
removing print
franciscojavierarceo Sep 21, 2024
d34db1d
checking in progress
franciscojavierarceo Sep 23, 2024
731bacb
updating test
franciscojavierarceo Sep 25, 2024
8479296
adding test
franciscojavierarceo Sep 25, 2024
3068c3b
linted and updated
franciscojavierarceo Sep 25, 2024
238dc29
removed print
franciscojavierarceo Sep 25, 2024
06315b9
updated tests to test actual behavior
franciscojavierarceo Sep 25, 2024
05efa37
checking in progress
franciscojavierarceo Sep 28, 2024
8c559c8
changing typo
franciscojavierarceo Sep 28, 2024
8b8aa16
updating test
franciscojavierarceo Sep 28, 2024
4562a23
testing changes
franciscojavierarceo Sep 28, 2024
14f837d
checking to see if thing still working
franciscojavierarceo Sep 29, 2024
082a860
removed print
franciscojavierarceo Sep 29, 2024
1d52d29
undo change for odfv file
franciscojavierarceo Sep 29, 2024
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view
Prev Previous commit
Next Next commit
saving progress 
Signed-off-by: Francisco Javier Arceo <[email protected]>
  • Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

@franciscojavierarceo
franciscojavierarceo committed Sep 29, 2024
commit b345e7e22e98949da0f92ff9f70a1e50d0ee7a83

There are no files selected for viewing

22 changes: 14 additions & 8 deletions 22 sdk/python/feast/feature_store.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1444,19 +1444,18 @@ def write_to_online_store(
inputs: Optional the dictionary object to be written
allow_registry_cache (optional): Whether to allow retrieving feature views from a cached registry.
"""
# TODO: restrict this to work with online StreamFeatureViews and validate the FeatureView type
feature_view_dict = {
fv_proto.name: fv_proto
for fv_proto in self.list_all_feature_views(allow_registry_cache)
}
try:
feature_view: FeatureView = self.get_stream_feature_view(
feature_view_name, allow_registry_cache=allow_registry_cache
)
feature_view = feature_view_dict[feature_view_name]
except FeatureViewNotFoundException:
feature_view = self.get_feature_view(
feature_view_name, allow_registry_cache=allow_registry_cache
)
raise FeatureViewNotFoundException(feature_view_name, self.project)
if df is not None and inputs is not None:
raise ValueError("Both df and inputs cannot be provided at the same time.")
if df is None and inputs is not None:
if isinstance(inputs, dict):
if isinstance(inputs, dict) or isinstance(inputs, List):
try:
df = pd.DataFrame(inputs)
except Exception as _:
Expand All @@ -1465,6 +1464,13 @@ def write_to_online_store(
pass
else:
raise ValueError("inputs must be a dictionary or a pandas DataFrame.")
if df is not None and inputs is None:
if isinstance(df, dict) or isinstance(df, List):
try:
df = pd.DataFrame(df)
except Exception as _:
raise DataFrameSerializationError

provider = self._get_provider()
provider.ingest_df(feature_view, df)

Expand Down
4 changes: 4 additions & 0 deletions 4 sdk/python/feast/infra/passthrough_provider.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,10 @@ def ingest_df(
):
table = pa.Table.from_pandas(df)

# TODO: Update this to support On Demand Feature Views.
# Note: A dictionary mapping of column names in this data
# source to feature names in a feature table or view. Only used for feature
# columns, not entity or timestamp columns.
if feature_view.batch_source.field_mapping is not None:
table = _run_pyarrow_field_mapping(
table, feature_view.batch_source.field_mapping
Expand Down
8 changes: 8 additions & 0 deletions 8 sdk/python/tests/unit/test_on_demand_python_transformation.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import os
import re
import tempfile
import time
import unittest
from datetime import datetime, timedelta
from typing import Any
Expand Down Expand Up @@ -285,9 +286,16 @@ def test_stored_writes(self):
entity_rows = [
{
"driver_id": 1001,
"conv_rate": 0.25,
"acc_rate": 0.25,
}
]

self.store.write_to_online_store(
feature_view_name="python_stored_writes_feature_view",
df=entity_rows,
)
time.sleep(1)
online_python_response = self.store.get_online_features(
entity_rows=entity_rows,
features=[
Expand Down
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.