GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix(server): add request timeouts to language model and reranker clients by deshmukh-viraj · Pull Request #1931 · MemMachine/MemMachine

Conversation

@deshmukh-viraj

@deshmukh-viraj deshmukh-viraj commented Nov 5, 2026

Copy link
Copy Markdown

What does this PR do?

OpenAI and Cohere clients were built without a timeout, so the SDK defaults applied: ten minutes per language-model call (with two SDK retries) and up to five minutes per rerank. One slow or hung provider held searches and stalled semantic extraction for every project, as described in the issue with traces showing a reranker endpoint answering after 75 s and a hung language-model call pausing all extraction for about ten minutes.

This adds a configurable request_timeout_seconds field to the three affected configs and passes it to the client constructors:

  • OpenAIResponsesLanguageModelConf and OpenAIChatCompletionsLanguageModelConf: default 120 s, since large prompts against local OpenAI-compatible endpoints (Ollama, vLLM) can legitimately take minutes
  • CohereRerankerConf: default 30 s, since rerankers sit in the search path and individual rerank calls are lightweight

Related Issue

Fixes #1920

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Changes Made

  • packages/server/src/memmachine_server/common/configuration/language_model_conf.py: request_timeout_seconds field on both OpenAI configs (default 120.0, gt=0)
  • packages/server/src/memmachine_server/common/configuration/reranker_conf.py: request_timeout_seconds field on CohereRerankerConf (default 30.0, gt=0)
  • packages/server/src/memmachine_server/common/resource_manager/language_model_manager.py: pass the timeout to both AsyncOpenAI constructors
  • packages/server/src/memmachine_server/common/resource_manager/reranker_manager.py: pass the timeout to both AsyncClientV2 constructors
  • packages/server/server_tests/memmachine_server/common/resource_manager/test_provider_client_timeouts.py: new tests pinning that configured and default timeouts reach the clients
  • packages/server/server_tests/memmachine_server/common/resource_manager/test_reranker_manager.py: two existing exact-kwargs assertions updated for the new timeout kwarg

How to Test

  1. uv run pytest packages/server/server_tests/memmachine_server/common/resource_manager/test_provider_client_timeouts.py -v - 2 passed (configured values 45 s / 60 s and the 120 s default reach the clients)
  2. uv run pytest packages/server/server_tests -q (excluding the pre-existing hnswlib collection gap) - 2063 passed, 16 skipped
  3. uv run ruff check and uv run ruff format --check on touched files - clean

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@deshmukh-viraj
fix(server): add request timeouts to language model and reranker clients
800d1f5 
OpenAI and Cohere clients were built without a timeout, so the SDK
defaults applied (ten minutes for OpenAI). One slow or hung provider
held searches and stalled semantic extraction for every project.

Add a request_timeout_seconds field to the OpenAI responses, OpenAI
chat completions, and Cohere reranker configs (defaults 120s and 30s)
and pass it to the client constructors.
@deshmukh-viraj deshmukh-viraj mentioned this pull request Nov 5, 2026
[Bug]: Language-model and reranker calls have no timeout, so one slow call holds searches and stalls semantic extraction for every project #1920
Open

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: Language-model and reranker calls have no timeout, so one slow call holds searches and stalls semantic extraction for every project

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@deshmukh-viraj
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.