GeezerVison is showing github.com — Error Store should not require a storage spec · Issue #27 · feast-dev/feast
Error Store should not require a storage spec #27DescriptionThe error store should be defined on deployment (of core), and should not require a spec. Activitytims commented on Jan 3, 2019 Can you clarify how would you like to tell ingestion where to store the errors? zhilingc commented on Jan 3, 2019 CollaboratorAuthor More actions Currently my plan is to pass it in as 2 separate options:
--errorsStoreType: one of [file.json, stderr, stdout] - type of errors store to write to
--errorsStoreOptions: map of options for the errors store
These would be defined in the config for core feast deployment
is this ok @tims? tims commented on Jan 3, 2019 That looks suspiciously like a storage spec :)
If we're committed to just files and logs.. we could just log to stderr by default if a file path isn't provided?
I imagine we might want avro or parquet too though... hmm.
Would errorsStoreOptions be comma separated kvs? eg. x=1,y=2 zhilingc commented on Jan 3, 2019 CollaboratorAuthor More actions Sorry, I realised the issue was poorly worded: I meant we should not give error stores the same treatment as other storage specs currently.
The fact that it (1) has to be registered (2) can be retrieved together with all other specs and (3) can be written to as a regular warehouse store by ingestion jobs is what i want to change.
At the same time it would be nice to retain the flexibility to add other potential error store formats in the future, which is why I've split its definition into type and options.
Would errorsStoreOptions be comma separated kvs? eg. x=1,y=2
I was thinking json e.g. {"path":"/path/to/file",...} maybe, but I'm not sure... tims commented on Jan 6, 2019 I think json is ok, as long as it doesn't cause any shell escaping issues for the core api?
reviewing the rest now
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|