GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: Add services functionality to Operator by tchughesiv · Pull Request #4723 · feast-dev/feast

Conversation

tchughesiv commented Oct 30, 2024 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

What this PR does / why we need it:

This PR provides Operator users the ability to deploy the following ephemeral feast services:

Online Store (optional)
Offline Store (optional)
Registry

Any k8s services related to these feast service types, that are created/owned by the Operator CR, will be cleaned up should a user change their mind and remove said service from the CR config.

A user can affect several k8s container configs for each service type:

image
imagePullPolicy
resources

Additionally, we allow the user to point to a remote registry if they so choose. This would be instead of deploying a local registry with the FeatureStore CR.

Which issue(s) this PR fixes:

Relates to #4561

Misc

Deployed CR examples

CR w/ defaults set in status... this deploys an ephemeral registry server -

apiVersion: feast.dev/v1alpha1
kind: FeatureStore
metadata:
 name: sample
  namespace: feast-operator-system
spec:
  feastProject: my_project
status:
  applied:
    feastProject: my_project
    services:
      registry:
        local:
          image: 'feastdev/feature-server:0.41.0'
  clientConfigMap: feast-sample-client
  conditions:
    - lastTransitionTime: '2024-10-31T18:02:46Z'
      message: Registry installation complete
      reason: Ready
      status: 'True'
      type: Registry
    - lastTransitionTime: '2024-10-31T18:02:46Z'
      message: Client installation complete
      reason: Ready
      status: 'True'
      type: Client
    - lastTransitionTime: '2024-10-31T18:02:46Z'
      message: FeatureStore installation complete
      reason: Ready
      status: 'True'
      type: FeatureStore
  feastVersion: 0.41.0
  phase: Ready
  serviceHostnames:
    registry: 'feast-sample-registry.feast-operator-system.svc.cluster.local:80'

CR w/ onlineStore, offlineStore, and a remote registry set in status... this deploys ephemeral online & offline stores, both pointed to a remote registry running in a separate namespace -

apiVersion: feast.dev/v1alpha1
kind: FeatureStore
metadata:
  name: sample
  namespace: feast-operator-system
spec:
  feastProject: my_project
  services:
    offlineStore: {}
    onlineStore:
      imagePullPolicy: Always
    registry:
      remote:
        feastRef:
          name: example
          namespace: default
status:
  applied:
    feastProject: my_project
    services:
      offlineStore:
        image: 'feastdev/feature-server:0.41.0'
      onlineStore:
        image: 'feastdev/feature-server:0.41.0'
        imagePullPolicy: Always
      registry:
        remote:
          feastRef:
            name: example
            namespace: default
  clientConfigMap: feast-sample-client
  conditions:
    - lastTransitionTime: '2024-10-31T18:02:46Z'
      message: Client installation complete
      reason: Ready
      status: 'True'
      type: Client
    - lastTransitionTime: '2024-10-31T18:02:46Z'
      message: FeatureStore installation complete
      reason: Ready
      status: 'True'
      type: FeatureStore
    - lastTransitionTime: '2024-10-31T18:45:49Z'
      message: Offline Store installation complete
      reason: Ready
      status: 'True'
      type: OfflineStore
    - lastTransitionTime: '2024-10-31T18:45:49Z'
      message: Online Store installation complete
      reason: Ready
      status: 'True'
      type: OnlineStore
  feastVersion: 0.41.0
  phase: Ready
  serviceHostnames:
    offlineStore: 'feast-sample-offline.feast-operator-system.svc.cluster.local:80'
    onlineStore: 'feast-sample-online.feast-operator-system.svc.cluster.local:80'
    registry: 'feast-example-registry.default.svc.cluster.local:80'

The above CR, produces the following feature_store.yaml config for the running onlineStore service -

$ kubectl get deploy feast-sample-online -o jsonpath='{.spec.template.spec.containers[*].env[?(@.name=="FEATURE_STORE_YAML_BASE64")].value}' | base64 -d

project: my_project
provider: local
offline_store:
    host: feast-sample-offline.feast-operator-system.svc.cluster.local
    type: remote
    port: 80
online_store:
    path: /tmp/online_store.db
    type: sqlite
registry:
    path: feast-example-registry.default.svc.cluster.local:80
    registry_type: remote
entity_key_serialization_version: 3

It also produces the following ConfigMap w/ client feature_store.yaml -

$ kubectl get cm feast-sample-client -o jsonpath='{.data.feature\_store\.yaml}'

project: my_project
provider: local
offline_store:
    host: feast-sample-offline.feast-operator-system.svc.cluster.local
    type: remote
    port: 80
online_store:
    path: http://feast-sample-online.feast-operator-system.svc.cluster.local:80
    type: remote
registry:
    path: feast-example-registry.default.svc.cluster.local:80
    registry_type: remote
entity_key_serialization_version: 3

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

tchughesiv changed the title ~~feat: Add services functionality to operator~~ feat: Add services functionality to Operator

Oct 31, 2024

tchughesiv force-pushed the services branch 2 times, most recently from b4ed717 to 5aac3a8 Compare

October 31, 2024 00:14

dmartinol reviewed

Oct 31, 2024

View reviewed changes

dmartinol left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

tchughesiv force-pushed the services branch 2 times, most recently from a0684e8 to 1b32e21 Compare

October 31, 2024 18:06

tchughesiv marked this pull request as ready for review

October 31, 2024 18:52

tchughesiv requested a review from a team as a code owner

October 31, 2024 18:52

tmihalac commented Oct 31, 2024

Copy link

Copy Markdown

Contributor

lgtm

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tchughesiv force-pushed the services branch 2 times, most recently from 7a5f06a to aa55a1a Compare

November 1, 2024 13:21


          Add services functionality to operator

7816c1e

Signed-off-by: Tommy Hughes <[email protected]>

tchughesiv force-pushed the services branch from aa55a1a to 7816c1e Compare

November 1, 2024 13:30

dmartinol approved these changes

Nov 4, 2024

View reviewed changes

dmartinol enabled auto-merge (squash)

November 4, 2024 07:16

haoxu0 added the ok-to-test label

Nov 4, 2024

dmartinol merged commit d1d80c0 into feast-dev:master

Nov 4, 2024

tchughesiv mentioned this pull request

Nov 11, 2024

Feast Go Operator #4561

Closed

shuchu pushed a commit to shuchu/feast that referenced this pull request

Nov 21, 2024


          feat: Add services functionality to Operator (feast-dev#4723)

746156d

Add services functionality to operator

Signed-off-by: Tommy Hughes <[email protected]>

franciscojavierarceo pushed a commit that referenced this pull request

Dec 5, 2024


          chore(release): release 0.42.0

aac334a

# [0.42.0](v0.41.0...v0.42.0) (2024-12-05)

### Bug Fixes

* Add adapters for sqlite datetime conversion ([#4797](#4797)) ([e198b17](e198b17))
* Added grpcio extras to default feature-server image ([#4737](#4737)) ([e9cd373](e9cd373))
* Changing node version in release ([7089918](7089918))
* Feast create empty online table when FeatureView attribute online=False ([#4666](#4666)) ([237c453](237c453))
* Fix db store types in Operator CRD ([#4798](#4798)) ([f09339e](f09339e))
* Fix the config issue for postgres ([#4776](#4776)) ([a36f7e5](a36f7e5))
* Fixed example materialize-incremental and improved explanation ([#4734](#4734)) ([ca8a7ab](ca8a7ab))
* Fixed SparkSource docstrings so it wouldn't used inhereted class docstrings ([#4722](#4722)) ([32e6aa1](32e6aa1))
* Fixing PGVector integration tests ([#4778](#4778)) ([88a0320](88a0320))
* Incorrect type passed to assert_permissions in materialize endpoints ([#4727](#4727)) ([b72c2da](b72c2da))
* Issue of DataSource subclasses using parent abstract class docstrings ([#4730](#4730)) ([b24acd5](b24acd5))
* Operator envVar positioning & tls.SecretRef.Name ([#4806](#4806)) ([1115d96](1115d96))
* Populates project created_time correctly according to created ti… ([#4686](#4686)) ([a61b93c](a61b93c))
* Reduce feast-server container image size & fix dev image build ([#4781](#4781)) ([ccc9aea](ccc9aea))
* Removed version func from feature_store.py ([#4748](#4748)) ([f902bb9](f902bb9))
* Support registry instantiation for read-only users ([#4719](#4719)) ([ca3d3c8](ca3d3c8))
* Syntax Error in BigQuery While Retrieving Columns that Start wit… ([#4713](#4713)) ([60fbc62](60fbc62))
* Update release version in a pertinent Operator file ([#4708](#4708)) ([764a8a6](764a8a6))

### Features

* Add api contract to fastapi docs ([#4721](#4721)) ([1a165c7](1a165c7))
* Add Couchbase as an online store ([#4637](#4637)) ([824859b](824859b))
* Add Operator support for spec.feastProject & status.applied fields ([#4656](#4656)) ([430ac53](430ac53))
* Add services functionality to Operator ([#4723](#4723)) ([d1d80c0](d1d80c0))
* Add TLS support to the Operator ([#4796](#4796)) ([a617a6c](a617a6c))
* Added feast Go operator db stores support ([#4771](#4771)) ([3302363](3302363))
* Added support for setting env vars in feast services in feast controller  ([#4739](#4739)) ([84b24b5](84b24b5))
* Adding docs outlining native Python transformations on singletons ([#4741](#4741)) ([0150278](0150278))
* Adding first feast operator e2e test. ([#4791](#4791)) ([8339f8d](8339f8d))
* Adding github action to run the operator end-to-end tests. ([#4762](#4762)) ([d8ccb00](d8ccb00))
* Adding ssl support for registry server. ([#4718](#4718)) ([ccf7a55](ccf7a55))
* Adding SSL support for the React UI server and feast UI command. ([#4736](#4736)) ([4a89252](4a89252))
* Adding support for native Python transformations on a single dictionary  ([#4724](#4724)) ([9bbc1c6](9bbc1c6))
* Adding TLS support for offline server. ([#4744](#4744)) ([5d8d03f](5d8d03f))
* Building the feast image ([#4775](#4775)) ([6635dde](6635dde))
* File persistence definition and implementation ([#4742](#4742)) ([3bad4a1](3bad4a1))
* Object store persistence in operator ([#4758](#4758)) ([0ae86da](0ae86da))
* OIDC authorization in Feast Operator ([#4801](#4801)) ([eb111d6](eb111d6))
* Operator will create k8s serviceaccount for each feast service ([#4767](#4767)) ([cde5760](cde5760))
* Printing more verbose logs when we start the offline server  ([#4660](#4660)) ([9d8d3d8](9d8d3d8))
* PVC configuration and impl ([#4750](#4750)) ([785a190](785a190))
* Qdrant vectorstore support ([#4689](#4689)) ([86573d2](86573d2))
* RBAC Authorization in Feast Operator ([#4786](#4786)) ([0ef5acc](0ef5acc))
* Support for nested timestamp fields in Spark Offline store ([#4740](#4740)) ([d4d94f8](d4d94f8))
* Update the go feature server from Expedia code repo. ([#4665](#4665)) ([6406625](6406625))
* Updated feast Go operator db stores ([#4809](#4809)) ([2c5a6b5](2c5a6b5))
* Updated sample secret following review ([#4811](#4811)) ([dc9f825](dc9f825))

dharmisha pushed a commit to nishantgaurav-dev/feast that referenced this pull request

Jan 15, 2025


          chore(release): release 0.42.0

6c64b8f

* Add adapters for sqlite datetime conversion ([feast-dev#4797](feast-dev#4797)) ([e198b17](feast-dev@e198b17))
* Added grpcio extras to default feature-server image ([feast-dev#4737](feast-dev#4737)) ([e9cd373](feast-dev@e9cd373))
* Changing node version in release ([7089918](feast-dev@7089918))
* Feast create empty online table when FeatureView attribute online=False ([feast-dev#4666](feast-dev#4666)) ([237c453](feast-dev@237c453))
* Fix db store types in Operator CRD ([feast-dev#4798](feast-dev#4798)) ([f09339e](feast-dev@f09339e))
* Fix the config issue for postgres ([feast-dev#4776](feast-dev#4776)) ([a36f7e5](feast-dev@a36f7e5))
* Fixed example materialize-incremental and improved explanation ([feast-dev#4734](feast-dev#4734)) ([ca8a7ab](feast-dev@ca8a7ab))
* Fixed SparkSource docstrings so it wouldn't used inhereted class docstrings ([feast-dev#4722](feast-dev#4722)) ([32e6aa1](feast-dev@32e6aa1))
* Fixing PGVector integration tests ([feast-dev#4778](feast-dev#4778)) ([88a0320](feast-dev@88a0320))
* Incorrect type passed to assert_permissions in materialize endpoints ([feast-dev#4727](feast-dev#4727)) ([b72c2da](feast-dev@b72c2da))
* Issue of DataSource subclasses using parent abstract class docstrings ([feast-dev#4730](feast-dev#4730)) ([b24acd5](feast-dev@b24acd5))
* Operator envVar positioning & tls.SecretRef.Name ([feast-dev#4806](feast-dev#4806)) ([1115d96](feast-dev@1115d96))
* Populates project created_time correctly according to created ti… ([feast-dev#4686](feast-dev#4686)) ([a61b93c](feast-dev@a61b93c))
* Reduce feast-server container image size & fix dev image build ([feast-dev#4781](feast-dev#4781)) ([ccc9aea](feast-dev@ccc9aea))
* Removed version func from feature_store.py ([feast-dev#4748](feast-dev#4748)) ([f902bb9](feast-dev@f902bb9))
* Support registry instantiation for read-only users ([feast-dev#4719](feast-dev#4719)) ([ca3d3c8](feast-dev@ca3d3c8))
* Syntax Error in BigQuery While Retrieving Columns that Start wit… ([feast-dev#4713](feast-dev#4713)) ([60fbc62](feast-dev@60fbc62))
* Update release version in a pertinent Operator file ([feast-dev#4708](feast-dev#4708)) ([764a8a6](feast-dev@764a8a6))

* Add api contract to fastapi docs ([feast-dev#4721](feast-dev#4721)) ([1a165c7](feast-dev@1a165c7))
* Add Couchbase as an online store ([feast-dev#4637](feast-dev#4637)) ([824859b](feast-dev@824859b))
* Add Operator support for spec.feastProject & status.applied fields ([feast-dev#4656](feast-dev#4656)) ([430ac53](feast-dev@430ac53))
* Add services functionality to Operator ([feast-dev#4723](feast-dev#4723)) ([d1d80c0](feast-dev@d1d80c0))
* Add TLS support to the Operator ([feast-dev#4796](feast-dev#4796)) ([a617a6c](feast-dev@a617a6c))
* Added feast Go operator db stores support ([feast-dev#4771](feast-dev#4771)) ([3302363](feast-dev@3302363))
* Added support for setting env vars in feast services in feast controller  ([feast-dev#4739](feast-dev#4739)) ([84b24b5](feast-dev@84b24b5))
* Adding docs outlining native Python transformations on singletons ([feast-dev#4741](feast-dev#4741)) ([0150278](feast-dev@0150278))
* Adding first feast operator e2e test. ([feast-dev#4791](feast-dev#4791)) ([8339f8d](feast-dev@8339f8d))
* Adding github action to run the operator end-to-end tests. ([feast-dev#4762](feast-dev#4762)) ([d8ccb00](feast-dev@d8ccb00))
* Adding ssl support for registry server. ([feast-dev#4718](feast-dev#4718)) ([ccf7a55](feast-dev@ccf7a55))
* Adding SSL support for the React UI server and feast UI command. ([feast-dev#4736](feast-dev#4736)) ([4a89252](feast-dev@4a89252))
* Adding support for native Python transformations on a single dictionary  ([feast-dev#4724](feast-dev#4724)) ([9bbc1c6](feast-dev@9bbc1c6))
* Adding TLS support for offline server. ([feast-dev#4744](feast-dev#4744)) ([5d8d03f](feast-dev@5d8d03f))
* Building the feast image ([feast-dev#4775](feast-dev#4775)) ([6635dde](feast-dev@6635dde))
* File persistence definition and implementation ([feast-dev#4742](feast-dev#4742)) ([3bad4a1](feast-dev@3bad4a1))
* Object store persistence in operator ([feast-dev#4758](feast-dev#4758)) ([0ae86da](feast-dev@0ae86da))
* OIDC authorization in Feast Operator ([feast-dev#4801](feast-dev#4801)) ([eb111d6](feast-dev@eb111d6))
* Operator will create k8s serviceaccount for each feast service ([feast-dev#4767](feast-dev#4767)) ([cde5760](feast-dev@cde5760))
* Printing more verbose logs when we start the offline server  ([feast-dev#4660](feast-dev#4660)) ([9d8d3d8](feast-dev@9d8d3d8))
* PVC configuration and impl ([feast-dev#4750](feast-dev#4750)) ([785a190](feast-dev@785a190))
* Qdrant vectorstore support ([feast-dev#4689](feast-dev#4689)) ([86573d2](feast-dev@86573d2))
* RBAC Authorization in Feast Operator ([feast-dev#4786](feast-dev#4786)) ([0ef5acc](feast-dev@0ef5acc))
* Support for nested timestamp fields in Spark Offline store ([feast-dev#4740](feast-dev#4740)) ([d4d94f8](feast-dev@d4d94f8))
* Update the go feature server from Expedia code repo. ([feast-dev#4665](feast-dev#4665)) ([6406625](feast-dev@6406625))
* Updated feast Go operator db stores ([feast-dev#4809](feast-dev#4809)) ([2c5a6b5](feast-dev@2c5a6b5))
* Updated sample secret following review ([feast-dev#4811](feast-dev#4811)) ([dc9f825](feast-dev@dc9f825))

Reviewers

dmartinol

dmartinol approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

tchughesiv commented Oct 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Which issue(s) this PR fixes:

Misc

Deployed CR examples

Uh oh!

dmartinol left a comment

Choose a reason for hiding this comment

Uh oh!

tmihalac commented Oct 31, 2024

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tchughesiv commented Oct 30, 2024 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.