Description

zhilingc

opened on Dec 21, 2018

Issue body actions

Expected Behavior

Starting a job with invalid stores (e.g. using redis as a warehouse store) should not be allowed, and should fail quickly - ideally at the graph building step of ingestion.

Current Behavior

Starting a job with invalid stores will successfully send the job to the runner, which will run to completion (or indefinitely, in the case of streaming jobs). The errors will be logged, but the pipeline will run with no problems.

Steps to reproduce

• Register a feature with its warehouse sink pointing to a serving store (e.g. redis)
• Run a job (direct runner is the best way to view errors)
• Pipeline runs successfully, a successful response is returned to the caller

Possible Solution

PR #11 is a band-aid solution to this problem: it checks the store types at registration, ensuring that a feature is unable to use a serving store for warehousing, but ideally ingestion error out properly during graph building.

Activity

1. zhilingc

   assigned

   zhilingc

   and

   tims

   and unassigned

   zhilingc

   on Dec 21, 2018
2. tims commented on Dec 24, 2018

   

   tims

   on Dec 24, 2018

   Contributor

   More actions

   @zhilingc I think you misunderstood the code it's not a "bandaid" :)

   It does fail at graph build time if the feature references a store that is not found in the specs service.
   See Specs.validate(). Am I missing something?
3. tims commented on Dec 24, 2018

   

   tims

   on Dec 24, 2018

   Contributor

   More actions

   Sorry I misread this issue.

   The fix is in PR #15.
4. zhilingc commented on Dec 24, 2018

   

   zhilingc

   on Dec 24, 2018

   CollaboratorAuthor

   More actions

   It's a band-aid because if someone alters the DB the job will fail since the checks are only at feature registration time.

   And nice! If it works on remote flink we're golden :)
5. zhilingc

   added 2 commits that reference this issue on Dec 31, 2018

   Merge #12: prefetch specs and validate on job expansion

   b604eb0

   Merge #12: prefetch specs and validate on job expansion

   7f0328c
6. tims commented on Dec 31, 2018

   

   tims

   on Dec 31, 2018

   Contributor

   More actions

   This should stop those self imposed DoS events every 10 minutes too.
7. tims commented on Dec 31, 2018

   

   tims

   on Dec 31, 2018

   Contributor

   More actions

   We can close this now?
8. zhilingc commented on Jan 3, 2019

   

   zhilingc

   on Jan 3, 2019

   CollaboratorAuthor

   More actions

   Yep. Thanks for the hard work :)
9. zhilingc

   closed this as completedon Jan 3, 2019
10. Yanson

    added a commit that references this issue on Jul 29, 2020

    Merge pull request feast-dev#12 from farfetch-external/algattik/merge…

    1ae84df
11. franciscojavierarceo

    added a commit that references this issue on Apr 15, 2025

    fix: Java CI #12

    d7e44ac
12. franciscojavierarceo

    added a commit that references this issue on Apr 29, 2025

    chore(release): release 0.49.0

    ad501cd
13. jfw-ppi

    added 2 commits that reference this issue on Jun 7, 2025

    fix: Java CI feast-dev#12

    542d64a

    chore(release): release 0.49.0

    0394e64

Issue actions