GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — (Deprecated) feat: Support DynamoDB as online store in Go feature server by iamcodingcat · Pull Request #5462 · feast-dev/feast

Conversation

iamcodingcat commented Jun 21, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

What this PR does / why we need it:

For ultimate low latency feature serving, I am using Golang api server for production environment. Moreover, a few months ago, I developed the feature that integrates s3 registry into Golang api server.(#5352 , #5336)

I am using Golang api server with redis as online store. But redis is usually for cache server and have lower SLA than DynamoDB. So I tried to integrate dynamodb as online store in Golang api server and succeed it. I refer the source code from integrating dynamodb in python sdk.

Of course, I am not writing test code but I locally finish testing about this feature. After reviewing my PR, I will soon add test code for this. please review about this for me! Thanks.

For reference, I carefully see compatibility between python and go of mmh3 hash algorithm used for converting entity key to entity id(In feast python sdk, it use this). I use this Golang lib and I check the complete compatibility in this lib with python sdk. I found this issue while investigating this. In my local environment(64-bit little endian), the test was completed. I will test it on my production linux environment on k8s cluster, asap.

Which issue(s) this PR fixes:

Golang api server can support DynamoDB as online store. About this, the issue(#3971) is made before. I will support this.

Misc

@shuchu @jyejare cc. @franciscojavierarceo

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

iamcodingcat requested a review from a team as a code owner

June 21, 2025 10:14

iamcodingcat changed the title ~~Feat: support online store as DynamoDB in Go feature server~~ Feat: support DynamoDB as online store in Go feature server

Jun 21, 2025

iamcodingcat changed the title ~~Feat: support DynamoDB as online store in Go feature server~~ feat: Support DynamoDB as online store in Go feature server

Jun 23, 2025

franciscojavierarceo added the ok-to-test label

Jun 23, 2025

shuchu approved these changes

Jun 24, 2025

View reviewed changes

shuchu left a comment

Copy link

Copy Markdown

Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat reacted with thumbs up emoji

All reactions

👍 1 reaction

jyejare reviewed

Jun 24, 2025

View reviewed changes

jyejare left a comment

Copy link

Copy Markdown

Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, have some questions/comments.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat reacted with thumbs up emoji

All reactions

👍 1 reaction

Comment thread

go.mod

		github.com/mattn/go-sqlite3 v1.14.23
		github.com/pkg/errors v0.9.1
		github.com/redis/go-redis/v9 v9.6.1
		github.com/roberson-io/mmh3 v0.0.0-20190729202758-fdfce3ba6225

jyejare Jun 24, 2025

Copy link

Copy Markdown

Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@franciscojavierarceo @tchughesiv How are we treating these external/personal repos as a safe repositories to use ?

Something like govulncheck Github Action should really help.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat reacted with thumbs up emoji

All reactions

👍 1 reaction

Comment thread

go/internal/feast/onlinestore/dynamodbonlinestore.go Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

go/internal/feast/onlinestore/dynamodbonlinestore.go Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

go/internal/feast/onlinestore/dynamodbonlinestore.go Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

go/internal/feast/onlinestore/dynamodbonlinestore.go Resolved

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat commented

Jun 24, 2025

View reviewed changes

iamcodingcat left a comment

Copy link

Copy Markdown

Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just add this only one test code for dynamodb as online store. Should I write more test code?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

iamcodingcat requested review from jyejare and shuchu

June 24, 2025 13:19

iamcodingcat commented

Jun 24, 2025

View reviewed changes

iamcodingcat left a comment

Copy link

Copy Markdown

Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use this same serializeEntityKey method that is used in redis online store. So I think that this method is common. So, I move this method to onlinestore.go.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

jyejare approved these changes

Jun 24, 2025

View reviewed changes

jyejare commented Jun 24, 2025

Copy link

Copy Markdown

Collaborator

@iamcodingcat Please sign the commit, one of the checks are failing.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat and others added 13 commits

June 25, 2025 09:19


          feat: integrate dynamodb as online store in Golang api server

e3aa274

Signed-off-by: iamcodingcat <[email protected]>


          chore: update mod and sum files

72e03c2

Signed-off-by: iamcodingcat <[email protected]>


          refact: regard serializing entity-key as common method

7d15f83

Signed-off-by: iamcodingcat <[email protected]>


          test: add dynamodb online store

b1abd41

Signed-off-by: iamcodingcat <[email protected]>


          refact: change to false in unprocessed entity set

9d62354

Signed-off-by: iamcodingcat <[email protected]>


          feat: Remote Write to Online Store completes client / server architec…

e8ed80a

…ture (feast-dev#5422)

* Remote Write to Online Store

Signed-off-by: jyejare <[email protected]>

* Comments Resolved and code shortened

Signed-off-by: jyejare <[email protected]>

---------

Signed-off-by: jyejare <[email protected]>
Signed-off-by: iamcodingcat <[email protected]>


          chore: Update development-guide.md

341cd59

Signed-off-by: iamcodingcat <[email protected]>


          feat: Make batch_source optional in PushSource (feast-dev#5440) (feas…

8d7c4c3

…t-dev#5454)

* Make batch_source optional in PushSource (feast-dev#5440)

Signed-off-by: snehsuresh <[email protected]>

* docs: make batch_source optional in PushSource
Signed-off-by: snehsuresh <[email protected]>

* refactor: remove comment
Signed-off-by: snehsuresh <[email protected]>

* fix: ensure batch_source is valid DataSource after null check

Signed-off-by: snehsuresh <[email protected]>

---------

Signed-off-by: snehsuresh <[email protected]>
Signed-off-by: iamcodingcat <[email protected]>


          test: Add Feast Milvus Jupyter Notebook Execution for downstream test…

9e7096b

…ing (feast-dev#5446)

Signed-off-by: Srihari <[email protected]>
Signed-off-by: iamcodingcat <[email protected]>


          feat: resolve conflicts

cc0a0ed


          test: Add RHOAI upgrate test to validate feast apply and materialize …

d4ca14d

…functionality

Signed-off-by: Srihari <[email protected]>
Signed-off-by: iamcodingcat <[email protected]>


          feat: Enable materialization for ODFV Transform on Write (feast-dev#5459

efd5d35

)

* feat: Enable materialization for ODFV Transform on Write

Signed-off-by: Francisco Javier Arceo <[email protected]>
Signed-off-by: iamcodingcat <[email protected]>


          feat: Add HybridOnlineStore for multi-backend online store routing (f…

e123a5d

…east-dev#5423)

* feat: Added HybridOnlineStore for multi-backend online store routing

- Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView.
- Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection.
- Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment.
- added documentation
- fixed linter raised issues

Signed-off-by: r0b0fyi <[email protected]>

* feat: Added HybridOnlineStore for multi-backend online store routing

- Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView.
- Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection.
- Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment.
- added documentation
- fixed linter raised issues

Signed-off-by: r0b0fyi <[email protected]>

---------

Signed-off-by: r0b0fyi <[email protected]>
Co-authored-by: r0b0fyi <[email protected]>
Signed-off-by: iamcodingcat <[email protected]>

iamcodingcat force-pushed the feat/go/online-store/dynamodb branch from 3c7ac9a to e123a5d Compare

June 25, 2025 00:21


          Merge branch 'master' into feat/go/online-store/dynamodb

5e7f393

iamcodingcat commented Jun 25, 2025

Copy link

Copy Markdown

Contributor Author

@jyejare I am twisted at git commit history for signing my commits. 😢 I made duplicate PR sync with latest master branch.
Can you approve about this PR, please? @jyejare @shuchu cc. @franciscojavierarceo

feat: Support DynamoDB as online store in Go feature server #5464

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat commented Jun 25, 2025

Copy link

Copy Markdown

Contributor Author

I close this PR and delete branch

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iamcodingcat closed this

Jun 25, 2025

iamcodingcat deleted the feat/go/online-store/dynamodb branch

June 25, 2025 00:57

iamcodingcat mentioned this pull request

Jun 25, 2025

feat: Support DynamoDB as online store in Go feature server #5464

Merged

iamcodingcat changed the title ~~feat: Support DynamoDB as online store in Go feature server~~ (Deprecated) feat: Support DynamoDB as online store in Go feature server

Jun 25, 2025

Reviewers

jyejare

jyejare approved these changes

shuchu

Awaiting requested review from shuchu

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

iamcodingcat commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Which issue(s) this PR fixes:

Misc

Uh oh!

shuchu left a comment

Choose a reason for hiding this comment

Uh oh!

jyejare left a comment

Choose a reason for hiding this comment

Uh oh!

jyejare Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iamcodingcat left a comment

Choose a reason for hiding this comment

Uh oh!

iamcodingcat left a comment

Choose a reason for hiding this comment

Uh oh!

jyejare commented Jun 24, 2025

Uh oh!

iamcodingcat commented Jun 25, 2025

Uh oh!

iamcodingcat commented Jun 25, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

iamcodingcat commented Jun 21, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.