GeezerVison is showing github.com — feat: Make transform on writes configurable for ingestion by franciscojavierarceo · Pull Request #5283 · feast-dev/feast
Conversation
What this PR does / why we need it:
This PR introduces a new transform_on_write flag to various functions and classes in the Feast SDK. The flag allows users to enable or disable transformations of data when writing to the online or offline store. This feature improves flexibility in managing data transformations during feature ingestion.
This feature is particularly useful if you want to batch transform a set of data (e.g., using Spark) and upload that transformed data to the online store and then want to also offer transformation on writes with the Feast Feature Server using an On Demand transformation.
Changes Summary:
sdk/python/feast/feature_server.py
- Added
transform_on_write attribute to WriteToFeatureStoreRequest and PushFeaturesRequest classes.
- Updated
push and write_to_online_store functions to pass the transform_on_write parameter to downstream calls.
sdk/python/feast/feature_store.py
- Added
transform_on_write parameter to the push method and its docstring.
- Updated the
push method to pass the transform_on_write parameter to the write_to_online_store and write_to_offline_store methods.
- Modified
_get_feature_view_and_df_for_online_write and write_to_online_store functions to support the transform_on_write parameter.
sdk/python/tests/unit/test_on_demand_python_transformation.py
- Fixed a typo in the constant
ODFV_OTHER_STRING_CONSTANT.
- Introduced new test cases for
transform_on_write functionality to verify behavior when the flag is disabled (transform_on_write=False).
- Added constants and test data for validating untransformed writes to the online store.
This enhancement provides better control over data transformations when pushing features, which can be particularly useful for scenarios where pre-transformed data is being ingested.
Which issue(s) this PR fixes:
#5196
Misc
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
jfw-ppi
pushed a commit
to jfw-ppi/feast
that referenced
this pull request
Jun 7, 2025
…5283)
* feat: Make transform on writes configurable for batch ingestion
Signed-off-by: Francisco Javier Arceo <[email protected]>
* update test and fix bug to support skipping transformation on writes for ODFV
Signed-off-by: Francisco Javier Arceo <[email protected]>
---------
Signed-off-by: Francisco Javier Arceo <[email protected]>
Signed-off-by: Jacob Weinhold <[email protected]>
Reviewers
shuchu
shuchu approved these changes
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.