GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: Add async DynamoDB timeout and retry configuration by sebastjaeger · Pull Request #5178 · feast-dev/feast

Conversation

sebastjaeger commented Mar 21, 2025

Copy link

Copy Markdown

Contributor

What this PR does / why we need it:

In some cases it can be useful to be able to control the timeout and retry behavior of (async) DynamoDB connections. This PR exposes the respective functionality of (aio)botocore in the current project. By default, connect_timeout and read_timeout are set to 60 seconds, which is the default in (aio)botocore (https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html). Therefore this value is currently also implicitly used.

The DynamoDBOnlineStoreRetryConfig model was introduced to stay close to what the configuration would look like in botocore. If preferred, the respective fields could also be moved into the top level config model, i.e., DynamoDBOnlineStoreConfig.

Which issue(s) this PR fixes:

Misc

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

sebastjaeger requested a review from a team as a code owner

March 21, 2025 15:50

franciscojavierarceo added the ok-to-test label

Mar 21, 2025

franciscojavierarceo reviewed

Mar 21, 2025

View reviewed changes

Comment thread

sdk/python/feast/infra/online_stores/dynamodb.py Outdated

		logger = logging.getLogger(__name__)


		class DynamoDBOnlineStoreRetryConfig(FeastConfigBaseModel):

franciscojavierarceo Mar 21, 2025

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i would move this into DynamoDBOnlineStoreConfig.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

franciscojavierarceo requested changes

Mar 21, 2025

View reviewed changes

franciscojavierarceo left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move the new fields to the single config

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sebastjaeger reacted with thumbs up emoji

All reactions

👍 1 reaction

sebastjaeger commented Mar 21, 2025

Copy link

Copy Markdown

Contributor Author

Moved the retry configuration into DynamoDBOnlineStoreConfig.

While doing so, I also did some digging into botocore, which led me to remove the max_attempts configuration field. The reason is that max_attempts is either ignored when total_max_attempts is set or is only used to set total_max_attempts if the latter is not provided: https://github.com/boto/botocore/blob/dd8406d5fa1df18037d1dd2977aec47334f7e3ce/botocore/args.py#L558

The retry configuration is provided to AioConfig as a dict (retries), where keys are only set if they are not None. This is to ensure that the retry default configuration remains unaffected if the respective values are not set.

Note the naming:

total_max_retry_attempts maps to total_max_attempts in the retries dict of botocore.config.Config
retry_mode maps to mode in the aforementioned dict

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sebastjaeger added 6 commits

March 21, 2025 23:27


          Add async DynamoDB timeout and retry configuration

61bea08

Signed-off-by: Sebastian Jäger <[email protected]>


          Use union sytnax

fc42cc7

Signed-off-by: Sebastian Jäger <[email protected]>


          Move retry configutation into top level, remove max_attempts

86be5fb

Signed-off-by: Sebastian Jäger <[email protected]>


          Add explicity type hint to retries dict

3787fef

Signed-off-by: Sebastian Jäger <[email protected]>


          Update docstrings

88d9619

Signed-off-by: Sebastian Jäger <[email protected]>


          Fix docstrings

05f9eb2

Signed-off-by: Sebastian Jäger <[email protected]>

sebastjaeger force-pushed the feature/add_botocore_configuration_options branch from 36ab012 to 05f9eb2 Compare

March 21, 2025 22:27

sebastjaeger commented Mar 21, 2025

Copy link

Copy Markdown

Contributor Author

Force push to fix missing signoffs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sebastjaeger commented Mar 22, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor Author

Not clear if test failures are due to the changes. Potentially networking issues.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          Update doc string, use None as retries default

5c844b7

Signed-off-by: Sebastian Jäger <[email protected]>

sebastjaeger requested a review from franciscojavierarceo

March 24, 2025 11:50

franciscojavierarceo approved these changes

Mar 24, 2025

View reviewed changes

franciscojavierarceo left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for this!!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sebastjaeger reacted with rocket emoji

All reactions

🚀 1 reaction

franciscojavierarceo merged commit 2f3bcf5 into feast-dev:master

Mar 24, 2025

sebastjaeger commented Mar 24, 2025

Copy link

Copy Markdown

Contributor Author

My pleasure :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo pushed a commit that referenced this pull request

Apr 7, 2025


          chore(release): release 0.48.0

# [0.48.0](v0.47.0...v0.48.0) (2025-04-07)

### Bug Fixes

* Enhance integration logos display and styling in the UI ([#5221](#5221)) ([5799257](5799257))
* Fix space typo in push.md docs ([#5184](#5184)) ([81677b2](81677b2))
* Fixed integration tests for qdrant and milvus ([#5224](#5224)) ([d6b080d](d6b080d))
* Formatting trino ([760ec0e](760ec0e))
* Multiple fixes in retrieval of online documents ([#5168](#5168)) ([66ddd3e](66ddd3e))
* Operator route creation for Feast UI in OpenShift ([e3946b4](e3946b4))
* Remove entity_rows parameter from retrieve_online_documents_v2 call ([#5225](#5225)) ([2a2e304](2a2e304))
* Styling ([#5222](#5222)) ([34c393c](34c393c))
* typo in the chart ([bd3448b](bd3448b))
* Update milvus-quickstart and feature_store.yaml with correct Milvus Config ([#5200](#5200)) ([306acca](306acca))
* Update Qdrant online store paths in repo_config.py ([#5207](#5207)) ([ab35b0b](ab35b0b)), closes [#5206](#5206)
* Update the doc ([#5194](#5194)) ([726464e](726464e))
* Updated the operator-rabc example to test RBAC from a Kubernete pod ([#5147](#5147)) ([d23a1a5](d23a1a5))

### Features

* add `real`(float32) type for trino offline store ([#4749](#4749)) ([0947f96](0947f96))
* Add async DynamoDB timeout and retry configuration ([#5178](#5178)) ([2f3bcf5](2f3bcf5))
* Add CronJob capability to the Operator (feast apply & materialize-incremental) ([#5217](#5217)) ([285c0dc](285c0dc))
* Add RAG tutorial and Use Cases documentation ([#5226](#5226)) ([99f4004](99f4004))
* Added CLI for features, get historical and online features ([#5197](#5197)) ([4ab9f74](4ab9f74))
* Added export support in feast UI ([#5198](#5198)) ([b079553](b079553))
* Added global registry search support in Feast UI ([#5195](#5195)) ([f09ea49](f09ea49))
* Added UI for Features list ([#5192](#5192)) ([cc7fd47](cc7fd47))
* Adding blog on RAG with Milvus ([#5161](#5161)) ([b9e2e6c](b9e2e6c))
* Adding Docling RAG demo ([#5109](#5109)) ([569404b](569404b))
* Allow transformations on writes to output list of entities ([#5209](#5209)) ([955521a](955521a))
* Cache get_any_feature_view results ([#5175](#5175)) ([924b8a3](924b8a3))
* Clickhouse offline store ([#4725](#4725)) ([86794c2](86794c2))
* Enable keyword search for Milvus ([#5199](#5199)) ([ac44967](ac44967))
* Enable transformations on PDFs ([#5172](#5172)) ([3674971](3674971))
* Enable users to use Entity Query as CTE during historical retrieval ([#5202](#5202)) ([fe69eaf](fe69eaf))
* helm support more deployment config ([d575372](d575372))
* Improved CLI file structuring ([#5201](#5201)) ([972ed34](972ed34))
* Kickoff Transformation implementationtransformation code base ([#5181](#5181)) ([0083303](0083303))
* Make keep-alive timeout configurable for async DynamoDB connections ([#5167](#5167)) ([7f3e528](7f3e528))
* Operator mounts the odh-trusted-ca-bundle configmap when deployed on RHOAI or ODH ([d4d7b0d](d4d7b0d))
* Spark Transformation ([#5185](#5185)) ([be3d85c](be3d85c))

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request

Jun 7, 2025


          feat: Add async DynamoDB timeout and retry configuration (feast-dev#5178

c8a542c

)

* Add async DynamoDB timeout and retry configuration

Signed-off-by: Sebastian Jäger <[email protected]>

* Use union sytnax

Signed-off-by: Sebastian Jäger <[email protected]>

* Move retry configutation into top level, remove max_attempts

Signed-off-by: Sebastian Jäger <[email protected]>

* Add explicity type hint to retries dict

Signed-off-by: Sebastian Jäger <[email protected]>

* Update docstrings

Signed-off-by: Sebastian Jäger <[email protected]>

* Fix docstrings

Signed-off-by: Sebastian Jäger <[email protected]>

* Update doc string, use None as retries default

Signed-off-by: Sebastian Jäger <[email protected]>

---------

Signed-off-by: Sebastian Jäger <[email protected]>
Signed-off-by: Jacob Weinhold <[email protected]>

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request

Jun 7, 2025


          chore(release): release 0.48.0

cc36081

# [0.48.0](feast-dev/feast@v0.47.0...v0.48.0) (2025-04-07)

### Bug Fixes

* Enhance integration logos display and styling in the UI ([feast-dev#5221](feast-dev#5221)) ([5799257](feast-dev@5799257))
* Fix space typo in push.md docs ([feast-dev#5184](feast-dev#5184)) ([81677b2](feast-dev@81677b2))
* Fixed integration tests for qdrant and milvus ([feast-dev#5224](feast-dev#5224)) ([d6b080d](feast-dev@d6b080d))
* Formatting trino ([760ec0e](feast-dev@760ec0e))
* Multiple fixes in retrieval of online documents ([feast-dev#5168](feast-dev#5168)) ([66ddd3e](feast-dev@66ddd3e))
* Operator route creation for Feast UI in OpenShift ([e3946b4](feast-dev@e3946b4))
* Remove entity_rows parameter from retrieve_online_documents_v2 call ([feast-dev#5225](feast-dev#5225)) ([2a2e304](feast-dev@2a2e304))
* Styling ([feast-dev#5222](feast-dev#5222)) ([34c393c](feast-dev@34c393c))
* typo in the chart ([bd3448b](feast-dev@bd3448b))
* Update milvus-quickstart and feature_store.yaml with correct Milvus Config ([feast-dev#5200](feast-dev#5200)) ([306acca](feast-dev@306acca))
* Update Qdrant online store paths in repo_config.py ([feast-dev#5207](feast-dev#5207)) ([ab35b0b](feast-dev@ab35b0b)), closes [feast-dev#5206](feast-dev#5206)
* Update the doc ([feast-dev#5194](feast-dev#5194)) ([726464e](feast-dev@726464e))
* Updated the operator-rabc example to test RBAC from a Kubernete pod ([feast-dev#5147](feast-dev#5147)) ([d23a1a5](feast-dev@d23a1a5))

### Features

* add `real`(float32) type for trino offline store ([feast-dev#4749](feast-dev#4749)) ([0947f96](feast-dev@0947f96))
* Add async DynamoDB timeout and retry configuration ([feast-dev#5178](feast-dev#5178)) ([2f3bcf5](feast-dev@2f3bcf5))
* Add CronJob capability to the Operator (feast apply & materialize-incremental) ([feast-dev#5217](feast-dev#5217)) ([285c0dc](feast-dev@285c0dc))
* Add RAG tutorial and Use Cases documentation ([feast-dev#5226](feast-dev#5226)) ([99f4004](feast-dev@99f4004))
* Added CLI for features, get historical and online features ([feast-dev#5197](feast-dev#5197)) ([4ab9f74](feast-dev@4ab9f74))
* Added export support in feast UI ([feast-dev#5198](feast-dev#5198)) ([b079553](feast-dev@b079553))
* Added global registry search support in Feast UI ([feast-dev#5195](feast-dev#5195)) ([f09ea49](feast-dev@f09ea49))
* Added UI for Features list ([feast-dev#5192](feast-dev#5192)) ([cc7fd47](feast-dev@cc7fd47))
* Adding blog on RAG with Milvus ([feast-dev#5161](feast-dev#5161)) ([b9e2e6c](feast-dev@b9e2e6c))
* Adding Docling RAG demo ([feast-dev#5109](feast-dev#5109)) ([569404b](feast-dev@569404b))
* Allow transformations on writes to output list of entities ([feast-dev#5209](feast-dev#5209)) ([955521a](feast-dev@955521a))
* Cache get_any_feature_view results ([feast-dev#5175](feast-dev#5175)) ([924b8a3](feast-dev@924b8a3))
* Clickhouse offline store ([feast-dev#4725](feast-dev#4725)) ([86794c2](feast-dev@86794c2))
* Enable keyword search for Milvus ([feast-dev#5199](feast-dev#5199)) ([ac44967](feast-dev@ac44967))
* Enable transformations on PDFs ([feast-dev#5172](feast-dev#5172)) ([3674971](feast-dev@3674971))
* Enable users to use Entity Query as CTE during historical retrieval ([feast-dev#5202](feast-dev#5202)) ([fe69eaf](feast-dev@fe69eaf))
* helm support more deployment config ([d575372](feast-dev@d575372))
* Improved CLI file structuring ([feast-dev#5201](feast-dev#5201)) ([972ed34](feast-dev@972ed34))
* Kickoff Transformation implementationtransformation code base ([feast-dev#5181](feast-dev#5181)) ([0083303](feast-dev@0083303))
* Make keep-alive timeout configurable for async DynamoDB connections ([feast-dev#5167](feast-dev#5167)) ([7f3e528](feast-dev@7f3e528))
* Operator mounts the odh-trusted-ca-bundle configmap when deployed on RHOAI or ODH ([d4d7b0d](feast-dev@d4d7b0d))
* Spark Transformation ([feast-dev#5185](feast-dev#5185)) ([be3d85c](feast-dev@be3d85c))

Signed-off-by: Jacob Weinhold <[email protected]>

Reviewers

franciscojavierarceo

franciscojavierarceo approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

sebastjaeger commented Mar 21, 2025

What this PR does / why we need it:

Which issue(s) this PR fixes:

Misc

Uh oh!

franciscojavierarceo Mar 21, 2025

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

sebastjaeger commented Mar 21, 2025

Uh oh!

sebastjaeger commented Mar 21, 2025

Uh oh!

sebastjaeger commented Mar 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

sebastjaeger commented Mar 24, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sebastjaeger commented Mar 22, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.