GeezerVison is showing github.com — feat: Serialization version 2 and below removed by jyejare · Pull Request #5435 · feast-dev/feast
feat: Serialization version 2 and below removed
#5435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement . We’ll occasionally send you account related emails.
Already on GitHub?
Sign in
to your account
Uh oh!
There was an error while loading. Please reload this page.
Show file tree
Hide file tree
Changes from all commits
Commits
File filter
Filter by extension
.go
(2)
.gotmpl
(1)
.ipynb
(3)
.java
(2)
.md
(13)
.py
(37)
.template
(1)
.yaml
(31)
All 8 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Hide whitespace
Diff view
Hide whitespace
There are no files selected for viewing
Original file line number
Diff line number
Diff line change
Expand Up
@@ -113,7 +113,7 @@ provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
{% endtab %}
Expand Down
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,16 @@
# Entity Key Re- Serialization from Version 2 to 3
# Entity Key Serialization from Version 2 to Version 3
Entity Key Serialization version 2 will soon be deprecated, hence we need to shift the serialization and deserilization to version 3.
Entity Key Serialization version 2 is now deprecated. All new and existing Feast deployments should shift to using serialization version 3.
But here comes the challegnge where existing FeatuteViews on stores has written features with version 2. A version 2 serialized entity key cant be retrived using version 3 deserilization algorithm.
However, a challenge arises when existing FeatureViews in online or offline stores have written features with version 2. A version 2 serialized entity key cannot be retrieved using the version 3 deserialization algorithm.
## Reserialize the Feature Views entity Keys to version 3
The solution is to reserialize the entity keys from version 2 to version 3.
Follow the following procedures to reserialize the entity key to version 3 in feature View in an offline / online store.
In hosrt , you need to iterate through all the feature views in your Feast repository, retrieve their serialized entity keys (if stored in version 2), reserialize them to version 3, and then update the online/offline store or wherever the serialized keys are stored.
In short , you need to iterate through all the feature views in your Feast repository, retrieve their serialized entity keys (if stored in version 2), reserialize them to version 3, and then update the online/offline store or wherever the serialized keys are stored.
### 1. Initialize the Feature Store
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -88,7 +88,7 @@ def materialize(data_interval_start=None, data_interval_end=None):
provider="aws",
offline_store="file",
online_store=DynamoDBOnlineStoreConfig(region="us-west-2"),
entity_key_serialization_version=2
entity_key_serialization_version=3
)
store = FeatureStore(config=repo_config)
# Option 1: materialize just one feature view
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -83,7 +83,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down
Expand Up
@@ -121,7 +121,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down
Expand Up
@@ -161,7 +161,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
offline_store:
type: remote
host: localhost
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -47,7 +47,7 @@ repo_config = RepoConfig(
"partitions": 10
},
online_store=DynamoDBOnlineStoreConfig(region="us-west-1"),
entity_key_serialization_version=2
entity_key_serialization_version=3
)
store = FeatureStore(config=repo_config)
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -79,7 +79,7 @@ online_store:
path: data/online_store.db
offline_store:
type: dask
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -44,7 +44,7 @@ online_store:
password: password # Couchbase password from access credentials
bucket_name: feast # Couchbase bucket name, defaults to feast
kv_port: 11210 # Couchbase key-value port, defaults to 11210. Required if custom ports are used.
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
{% endcode %}
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -17,7 +17,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -62,7 +62,7 @@
},
{
"cell_type": "code",
"execution_count": 1 ,
"execution_count": 2 ,
"id": "8bd21689-4a8e-4b0c-937d-0911df9db1d3",
"metadata": {},
"outputs": [],
Expand Down
Expand Up
@@ -102,15 +102,15 @@
},
{
"cell_type": "code",
"execution_count": 2 ,
"execution_count": 6 ,
"id": "b3757221-2037-49eb-867f-b9529fec06e2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing Feature_Store/feature_store.yaml\n"
"Overwriting Feature_Store/feature_store.yaml\n"
]
}
],
Expand All
@@ -125,7 +125,7 @@
"online_store:\n",
" type: sqlite\n",
" path: data/online_store.db\n",
"entity_key_serialization_version: 2 "
"entity_key_serialization_version: 3 "
]
},
{
Expand Down
Expand Up
@@ -383,10 +383,18 @@
"id": "c764a60a-b911-41a8-ba8f-7ef0a0bc7257",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/jyejare/gitrepos/feast/.venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\n",
" warnings.warn(\n"
]
},
{
"data": {
"text/plain": [
"RepoConfig(project='loan_applications', provider='local', registry_config='data/registry.db', online_config={'type': 'sqlite', 'path': 'data/online_store.db'}, offline_config={'type': 'dask'}, batch_engine_config='local', feature_server=None, flags=None, repo_path=PosixPath('Feature_Store'), entity_key_serialization_version=2 , coerce_tz_aware=True)"
"RepoConfig(project='loan_applications', provider='local', registry_config='data/registry.db', online_config={'type': 'sqlite', 'path': 'data/online_store.db'}, auth={'type': 'no_auth'}, offline_config={'type': 'dask'}, batch_engine_config='local', feature_server=None, flags=None, repo_path=PosixPath('Feature_Store'), entity_key_serialization_version=3 , coerce_tz_aware=True)"
]
},
"execution_count": 7,
Expand Down
Expand Up
@@ -779,7 +787,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel) ",
"display_name": ".venv ",
"language": "python",
"name": "python3"
},
Expand All
@@ -793,7 +801,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9 "
"version": "3.11.13 "
}
},
"nbformat": 4,
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -256,7 +256,7 @@
" \"host\": \"localhost\",\n",
" \"port\": 8815\n",
" },\n",
" entity_key_serialization_version=2 \n",
" entity_key_serialization_version=3 \n",
"))"
]
},
Expand Down
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,3 @@
# Running Feast Java Server with Redis & calling with python (with registry in GCP)
For this tutorial, we setup Feast with Redis, using the Feast CLI to register and materialize features, and then retrieving via a Feast Java server deployed in Kubernetes via a gRPC call.
Expand Down
Expand Up
@@ -40,7 +39,7 @@ For this tutorial, we setup Feast with Redis, using the Feast CLI to register an
connection_string: localhost:6379,password=[YOUR PASSWORD]
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
4. Run `feast apply` to apply your local features to the remote registry
- Note: you may need to authenticate to gcloud first with `gcloud auth login`
Expand Down
Expand Up
@@ -86,7 +85,7 @@ For this tutorial, we setup Feast with Redis, using the Feast CLI to register an
>2. `make build-java-docker-dev`
>3. In the `application-override.yaml`, uncomment the two `image: tag: dev` blocks
>4. `helm install feast-release ../../../infra/charts/feast --values application-override.yaml`
5. (Optional): check logs of the server to make sure it’ s working
5. (Optional): check logs of the server to make sure it' s working
```bash
kubectl logs svc/feast-release-feature-server
```
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -10,7 +10,7 @@ feature-server:
host: my-redis-master
port: 6379
password: [YOUR PASSWORD]
entityKeySerializationVersion: 2
entityKeySerializationVersion: 3
# Uncomment below for dev
# image:
# tag: dev
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -7,4 +7,4 @@ online_store:
connection_string: localhost:6379,password=[YOUR PASSWORD]
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -9,6 +9,6 @@ offline_store:
online_store:
path: http://localhost:8003
type: remote
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
Original file line number
Diff line number
Diff line change
Expand Up
@@ -6,4 +6,4 @@ provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,3 @@
# Running Feast Python / Go Feature Server with Redis on Kubernetes
For this tutorial, we set up Feast with Redis.
Expand Down
Expand Up
@@ -64,7 +63,7 @@ Manifests have been taken from [Deploy Minio in your project](https://ai-on-open
connection_string: localhost:6379,password=****
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
1. To run `feast apply` from the current machine we need to define the AWS credentials to connect the MinIO S3 store, which
are defined in [minio.env](./minio.env):
Expand Down
Expand Up
@@ -142,7 +141,7 @@ are defined in [minio.env](./minio.env):
]'
kubectl wait --for=condition=available deployment/online-server --timeout=2m
```
1. (Optional): check logs of the server to make sure it’ s working
1. (Optional): check logs of the server to make sure it' s working
```bash
kubectl logs svc/online-server
```
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -6,4 +6,4 @@ online_store:
connection_string: localhost:6379,password=_REDIS_PASSWORD_
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -4,4 +4,4 @@ provider: local
online_store:
path: http://localhost:6566
type: remote
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,3 @@
entity_key_serialization_version: 2
offline_store:
host: localhost
port: 8815
Expand All
@@ -10,3 +9,4 @@ project: rbac
registry:
path: localhost:6570
registry_type: remote
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -16,4 +16,4 @@ auth:
client_secret: update-this-value
username: ${FEAST_USERNAME}
password: ${FEAST_PASSWORD}
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -23,4 +23,4 @@ offline_store:
db_schema: public
user: feast
password: feast
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -13,4 +13,4 @@ offline_store:
password: feast
auth:
type: kubernetes
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -17,4 +17,4 @@ offline_store:
port: 80
auth:
type: kubernetes
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -9,4 +9,4 @@ registry:
pool_pre_ping: true
auth:
type: kubernetes
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -15,4 +15,4 @@ auth:
type: oidc
auth_discovery_url: https://keycloak-feast-dev.apps.com/realms/feast-rbac/.well-known/openid-configuration
client_id: feast-client
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -19,4 +19,4 @@ auth:
type: oidc
auth_discovery_url: https://keycloak-feast-dev.apps.com/realms/feast-rbac/.well-known/openid-configuration
client_id: feast-client
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -11,4 +11,4 @@ auth:
type: oidc
auth_discovery_url: https://keycloak-feast-dev.apps.com/realms/feast-rbac/.well-known/openid-configuration
client_id: feast-client
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -7,4 +7,4 @@ offline_store:
type: remote
host: localhost
port: 8815
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Original file line number
Diff line number
Diff line change
Expand Up
@@ -6,4 +6,4 @@ provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Loading
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.