GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add support for Redis and Redis Cluster by qooba · Pull Request #1511 · feast-dev/feast

Conversation

@qooba

@qooba qooba commented Apr 27, 2021

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:
PR adds Redis and RedisCluster support as online store. As described in #1497 it is backward compatible with Feast 0.9 thus it can be used for migration to 0.10

Which issue(s) this PR fixes:
Fixes #1497

Does this PR introduce a user-facing change?:

Redis and RedisCluster online stores support added. Format is compatible with Feast 0.9.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tedhtchang and woop reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
@qooba
qooba requested review from jklegar, tsotnet and woop as code owners April 27, 2021 20:53
@feast-ci-bot feast-ci-bot added release-note needs-kind labels Apr 27, 2021
@feast-ci-bot

feast-ci-bot commented Apr 27, 2021

Copy link
Copy Markdown
Collaborator

Hi @qooba. Thanks for your PR.

I'm waiting for a feast-dev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@feast-ci-bot feast-ci-bot added needs-ok-to-test size/L labels Apr 27, 2021
@woop

woop commented Apr 27, 2021

Copy link
Copy Markdown
Member

/ok-to-test

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@feast-ci-bot feast-ci-bot added ok-to-test and removed needs-ok-to-test labels Apr 27, 2021
@woop

woop commented Apr 27, 2021

Copy link
Copy Markdown
Member

This is great work @qooba! Really surprised to see the implementation so quickly. We're still in the process of collecting feedback on the #1497, given that we may need to support a more scalable ingestion API for real time events. We may need a little bit of time before we can review and merge this code in, but we're super thankful for the contribution!!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@woop woop assigned jklegar Apr 27, 2021
@woop

woop commented Apr 28, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

One thing that stands out is that we don't have any automated tests for Redis as part of this PR. We'll definitely need to extend the PR to also include an integration test. @qooba do you want to take a stab at that?

One option is https://github.com/testcontainers/testcontainers-python, another is to just mark the tests as integration tests using @pytest.mark.integration and then only run tests on GitHub Actions with a Redis service provisioned

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@qooba

qooba commented Apr 28, 2021

Copy link
Copy Markdown
Contributor Author

Sure @woop I will extend PR with integration tests.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@qooba
qooba requested a review from a team as a code owner April 28, 2021 23:58
@qooba

qooba commented Apr 29, 2021

Copy link
Copy Markdown
Contributor Author

/retest

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@woop

woop commented Apr 30, 2021

Copy link
Copy Markdown
Member

For some reason it's not allowing you to run tests until you've actually merged some code into Feast. I'll need to approve every time.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jklegar
jklegar reviewed May 5, 2021
View reviewed changes

@jklegar jklegar left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @qooba! Yep we'll want integration tests to make sure these methods work in the same way as their GCP equivalents and don't rot in the future. For integration tests, adding a sdk/python/tests/test_cli_redis.py that is the equivalent of sdk/python/tests/test_cli_gcp.py should cover the main parts, plus adding redis into the GitHub actions workflows like Willem said above

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@qooba

qooba commented May 6, 2021

Copy link
Copy Markdown
Contributor Author

@jklegar I have added sdk/python/tests/test_cli_redis.py and also sdk/python/tests/test_offline_online_store_consistency.py::test_redis_offline_online_store_consistency to check offline consistency with redis. Redis service also added to GitHub action workflow.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@qooba

qooba commented May 6, 2021

Copy link
Copy Markdown
Contributor Author

The test sdk/python/tests/test_cli_redis.py::test_basic fails because added configuration is not used:

env:
  REDIS_TYPE: REDIS
  REDIS_CONNECTION_STRING: redis:6379,db=0

@jklegar can you help with this ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jklegar

jklegar commented May 10, 2021

Copy link
Copy Markdown
Collaborator

@qooba I've added the redis service to pr_integration_tests.yml in master, can you rebase this diff? Once you rebase it'll run the updated action here

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@qooba
qooba force-pushed the feature/online_redis branch from 6b33786 to b976599 Compare May 12, 2021 00:53
@qooba

qooba commented May 12, 2021

Copy link
Copy Markdown
Contributor Author

@jklegar there will be small change in github action configuration:
b8b0d9f

Additionally macOS-latest doesn't have docker thus tests will never pass can you remove it :)

And commit once again to master then I will rebase.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@woop woop changed the title Feature/online redis Add support for Redis and Redis Cluster May 13, 2021
@jklegar

jklegar commented May 18, 2021

Copy link
Copy Markdown
Collaborator

Thanks @qooba and apologies for the delay here, it should be good to rebase now

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@qooba
qooba force-pushed the feature/online_redis branch from 8f17664 to 55463af Compare May 18, 2021 23:04
@qooba

qooba commented May 18, 2021

Copy link
Copy Markdown
Contributor Author

Thanks @jklegar rebased and tests are passing now :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@woop

woop commented May 24, 2021

Copy link
Copy Markdown
Member

/kind feature

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@feast-ci-bot feast-ci-bot added the kind/feature New feature or request label May 24, 2021
@feast-ci-bot feast-ci-bot removed the lgtm label Jun 9, 2021
@woop
woop force-pushed the feature/online_redis branch from 771c4d2 to bb18897 Compare June 9, 2021 18:26
qooba and others added 22 commits June 9, 2021 12:52
@qooba @woop
Add support for Redis as online store
d97aaea
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add support for Redis as online store
ae9ed8e
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis provider
e165f00
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis online provider
df8f3cb
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis online provider
6bea80d
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis online provider
5745ff7
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis online provider
a3c4f3b
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis online provider - integration tests
e434e59
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis online provider - integration tests
8ccfc6b
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis provider
be9a17f
Signed-off-by: qooba <[email protected]>
@qooba @woop
Add redis provider
7606fe7
Signed-off-by: qooba <[email protected]>
@qooba @woop
correct redis provider
2a95e3e
Signed-off-by: qooba <[email protected]>
@qooba @woop
correct redis provider
435c856
Signed-off-by: qooba <[email protected]>
@qooba @woop
correct redis provider test
d329439
Signed-off-by: qooba <[email protected]>
@qooba @woop
correct redis provider
b5269e7
Signed-off-by: qooba <[email protected]>
@qooba @woop
correct redis provider
f602b9a
Signed-off-by: qooba <[email protected]>
@qooba @woop
change redis connection string
bbde0b0
Signed-off-by: qooba <[email protected]>
@qooba @woop
change redis connection string
d86f23c
Signed-off-by: qooba <[email protected]>
@qooba @woop
fix redis materialize
b0c8101
Signed-off-by: qooba <[email protected]>
@woop
Rename connection string and set defaults
52232cc
Signed-off-by: Willem Pienaar <[email protected]>
@woop
Fix argument in redis test
53f8f72
Signed-off-by: Willem Pienaar <[email protected]>
@woop
Fix broken connection string
e12ecff
Signed-off-by: Willem Pienaar <[email protected]>
@woop
woop force-pushed the feature/online_redis branch from bb18897 to e12ecff Compare June 9, 2021 19:52
@woop

woop commented Jun 9, 2021

Copy link
Copy Markdown
Member

/lgtm

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@feast-ci-bot feast-ci-bot added the lgtm label Jun 9, 2021
@feast-ci-bot
feast-ci-bot merged commit 0d7e858 into feast-dev:master Jun 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jklegar jklegar jklegar left review comments
@woop woop woop approved these changes
@tsotnet tsotnet Awaiting requested review from tsotnet
@achals achals Awaiting requested review from achals
+1 more reviewer
@abrookins abrookins abrookins approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@woop woop

@jklegar jklegar

Labels

approved kind/feature New feature or request lgtm ok-to-test release-note size/L

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add support for Redis to new Feast

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@qooba @feast-ci-bot @woop @jklegar @abrookins @codecov-commenter
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.