GeezerVison is showing github.com — feat: Contrib azure provider with synapse/mssql offline store and Azure registry store by kevjumba · Pull Request #3072 · feast-dev/feast
feat: Contrib azure provider with synapse/mssql offline store and Azure registry store
#3072
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
#To use Athena as an offline store, you need to create an Athena database and an S3 bucket on AWS. https://docs.aws.amazon.com/athena/latest/ug/getting-started.html
#Modify environment variables ATHENA_DATA_SOURCE, ATHENA_DATABASE, ATHENA_S3_BUCKET_NAME if you want to change the data source, database, and bucket name of S3 to use.
#If tests fail with the pytest -n 8 option, change the number to 1.
Expand All
@@ -161,7 +181,8 @@ test-python-universal-athena:
not test_historical_features_persisting and \
not test_historical_retrieval_fails_on_validation and \
Feast uses a registry to store all applied Feast objects (e.g. Feature views, entities, etc). The registry exposes
Feast uses a registry to store all applied Feast objects (e.g. Feature views, entities, etc). The registry exposes
methods to apply, list, retrieve and delete these objects, and is an abstraction with multiple implementations.
### Options for registry implementations
#### File-based registry
By default, Feast uses a file-based registry implementation, which stores the protobuf representation of the registry as
a serialized file. This registry file can be stored in a local file system, or in cloud storage (in, say, S3 or GCS).
By default, Feast uses a file-based registry implementation, which stores the protobuf representation of the registry as
a serialized file. This registry file can be stored in a local file system, or in cloud storage (in, say, S3 or GCS, or Azure).
The quickstart guides that use `feast init` will use a registry on a local file system. To allow Feast to configure
The quickstart guides that use `feast init` will use a registry on a local file system. To allow Feast to configure
a remote file registry, you need to create a GCS / S3 bucket that Feast can understand:
{% tabs %}
{% tab title="Example S3 file registry" %}
Expand All
@@ -35,9 +35,9 @@ offline_store:
{% endtab %}
{% endtabs %}
However, there are inherent limitations with a file-based registry, since changing a single field in the registry
requires re-writing the whole registry file. With multiple concurrent writers, this presents a risk of data loss, or
bottlenecks writes to the registry since all changes have to be serialized (e.g. when running materialization for
However, there are inherent limitations with a file-based registry, since changing a single field in the registry
requires re-writing the whole registry file. With multiple concurrent writers, this presents a risk of data loss, or
bottlenecks writes to the registry since all changes have to be serialized (e.g. when running materialization for
multiple feature views or time ranges concurrently).
#### SQL Registry
Expand All
@@ -47,14 +47,14 @@ This supports any SQLAlchemy compatible database as a backend. The exact schema
### Updating the registry
We recommend users store their Feast feature definitions in a version controlled repository, which then via CI/CD
automatically stays synced with the registry. Users will often also want multiple registries to correspond to
different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write
We recommend users store their Feast feature definitions in a version controlled repository, which then via CI/CD
automatically stays synced with the registry. Users will often also want multiple registries to correspond to
different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write
access since they can impact real user traffic. See [Running Feast in Production](../../how-to-guides/running-feast-in-production.md#1.-automatically-deploying-changes-to-your-feature-definitions) for details on how to set this up.
### Accessing the registry from clients
Users can specify the registry through a `feature_store.yaml` config file, or programmatically. We often see teams
Users can specify the registry through a `feature_store.yaml` config file, or programmatically. We often see teams
preferring the programmatic approach because it makes notebook driven development very easy:
#### Option 1: programmatically specifying the registry
The MsSQL offline store provides support for reading [MsSQL Sources](../data-sources/mssql.md). Specifically, it is developed to read from [Synapse SQL](https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/overview-features) on Microsoft Azure
* Entity dataframes can be provided as a SQL query or can be provided as a Pandas dataframe.
## Disclaimer
The MsSQL offline store does not achieve full test coverage.
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.