GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — v0.6.46: mothership streaming fixes, brightdata integration by icecrasher321 · Pull Request #4190 · simstudioai/sim

Conversation

@icecrasher321

@icecrasher321 icecrasher321 commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

fix(brightdata): fix async Discover API (polling), echo-back fields via params, registry ordering, improved error handling, and platform-aligned timeout (#4188)
fix(mothership): chat stream structuring + logs resource post fix (#4189)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

greptile-apps[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
waleedlatif1 and others added 2 commits April 15, 2026 16:20
@waleedlatif1 @claude
fix(brightdata): fix async Discover API, echo-back fields, and regist…
6dddc3f
…ry ordering (#4188)

* fix(brightdata): use params for echo-back fields in transformResponse

transformResponse receives params as its second argument. Use it to
return the original url, query, snapshotId, and searchEngine values
instead of hardcoding null or extracting from response data that may
not contain them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(brightdata): handle async Discover API with polling

The Bright Data Discover API is asynchronous — POST /discover returns
a task_id, and results must be polled via GET /discover?task_id=...
The previous implementation incorrectly treated it as synchronous,
always returning empty results.

Uses postProcess (matching Firecrawl crawl pattern) to poll every 3s
with a 120s timeout until status is "done".

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(brightdata): alphabetize block registry entry

Move box before brandfetch/brightdata to maintain alphabetical ordering.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* lint

* fix(brightdata): return error objects instead of throwing in postProcess

The executor wraps postProcess in try-catch and falls back to the
intermediate transformResponse result on error, which has success: true
with empty results. Throwing errors would silently return empty results.

Match Firecrawl's pattern: return { ...result, success: false, error }
instead of throwing. Also add taskId to BrightDataDiscoverResponse type
to eliminate unsafe casts.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(brightdata): use platform execution timeout for Discover polling

Replace hardcoded 120s timeout with DEFAULT_EXECUTION_TIMEOUT_MS to
match Firecrawl and other async polling tools. Respects platform-
configured limits (300s free, 3000s paid).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
@icecrasher321
fix(mothership): chat stream structuring + logs resource post fix (#4189
377712c
)

* fix(mothership): chat streaming structure

* fix logs resource thinking bug"

* address comments

* address comments
@vercel

vercel Bot commented Apr 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Apr 15, 2026 11:45pm

Request Review

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@cursor

cursor Bot commented Apr 15, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches core chat streaming and cancellation paths plus client-side caching, which can affect message ordering/duplication and queued follow-up delivery. Bright Data changes add polling and output-shaping that may impact runtime behavior and timeouts.

Overview
Improves mothership/copilot chat streaming by introducing buildEffectiveChatTranscript to merge Redis stream snapshots into the returned chat messages, and updating both copilot and mothership chat GET APIs to normalize persisted messages and append a consistent “live assistant” tail while a stream is active.

Refactors the home useChat hook to drive streaming updates through the React Query TaskChatHistory cache (stable live-assistant:<streamId> IDs), improve stop/cancel handling (mark executing tools as cancelled), and add a sessionStorage-backed queued-send handoff to safely resume queued follow-ups across navigation/stop boundaries. The /api/copilot/chat/stop route now always persists a cancelled completion block (and uses generateId), with a new route test.

Fixes Bright Data tool behavior by polling the async Discover API until completion (platform timeout-aligned), echoing key output fields from request params across several tools, and adding taskId to Discover outputs; also expands resource attachment types, tweaks handleTaskStatusEvent to invalidate only the workspace task list, and includes minor doc/registry/lockfile updates.

Reviewed by Cursor Bugbot for commit 377712c. Configure here.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@greptile-apps

greptile-apps Bot commented Apr 15, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR delivers two independent fixes: (1) BrightData Discover now correctly polls for async task results via postProcess and echoes input params back through output fields across all BrightData tools; (2) Mothership streaming is refactored so the live assistant transcript is materialized from Redis-backed stream events (effective-transcript.ts) and pushed directly into the React Query cache, replacing the previous local useState approach and eliminating the need for server refetches during an active stream.

Confidence Score: 5/5

Safe to merge; all findings are P2 quality-of-life improvements that do not affect the primary streaming or tool-execution paths.

Three P2 findings: polling elapsed-time precision, no retry on transient HTTP errors in BrightData Discover, and reduced cache invalidation for passive observers. None break core functionality — the streaming experience is correctly handled, and the BrightData issues only affect edge-case reliability in the polling loop.

apps/sim/tools/brightdata/discover.ts — timing and error-handling in the polling loop

Important Files Changed

Filename Overview
apps/sim/tools/brightdata/discover.ts Rewrites Discover tool to use async polling via postProcess; elapsed-time counter only counts sleep intervals not fetch latency, and any transient HTTP error immediately terminates the poll.
apps/sim/lib/copilot/chat/effective-transcript.ts New module that materializes a live assistant message from Redis-backed stream events, replacing the previous per-component state approach; logic is thorough and well-tested.
apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.ts Large refactor moving live stream state from local useState to React Query cache; adds queued-send handoff via sessionStorage; looks architecturally sound.
apps/sim/hooks/use-task-events.ts Scopes list invalidation to the current workspace (improvement), but removes detail invalidation for completed events — passive observers in other tabs may see stale chat transcripts.
apps/sim/app/api/copilot/chat/stop/route.ts Now always appends a stopped assistant message (even with empty content), fixes UUID generation to use generateId(), and synthesizes correct contentBlocks with a stopped marker.

Sequence Diagram

sequenceDiagram
    participant Client as useChat browser
    participant RQ as React Query Cache
    participant Server as Next.js API
    participant Redis as Redis stream events

    Client->>Server: POST copilot/chat send message
    Server-->>Client: SSE stream text/tool/complete events
    Client->>RQ: upsertTaskChatHistory on each flush

    alt Stream completes normally
        Client->>Client: finalize
        Client->>RQ: invalidateChatQueries includeDetail true
        RQ->>Server: GET copilot/chat
        Server->>Redis: fetch streamSnapshot
        Server->>Server: buildEffectiveChatTranscript
        Server-->>RQ: persisted messages plus live snapshot
    end

    alt SSE task_status event any tab
        Server-->>Client: task_status event
        Client->>RQ: invalidateQueries taskKeys.list workspaceId only
    end
Loading

Comments Outside Diff (1)

  1. apps/sim/hooks/use-task-events.ts, line 41-54 (link)

    P2 Stale detail cache for passive observers after stream completes

    The previous implementation invalidated taskKeys.detail(chatId) when a completed event arrived. That guaranteed any tab watching a chat (but not actively streaming) would re-fetch the final persisted messages.

    With that invalidation removed, a passive tab (e.g. the same chat open in a second browser tab, or a tab that navigated away mid-stream) will not see the final transcript until its own staleTime expires or it navigates. The active streaming tab is fine because finalize() still calls invalidateChatQueries. If passive-observer freshness matters, consider re-adding a targeted detail invalidation here.

Reviews (1): Last reviewed commit: "fix(mothership): chat stream structuring..." | Re-trigger Greptile

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

greptile-apps[bot]
greptile-apps Bot reviewed Apr 15, 2026
View reviewed changes
Comment thread apps/sim/tools/brightdata/discover.ts
Comment on lines +127 to +137
let elapsedTime = 0

while (elapsedTime < MAX_POLL_TIME_MS) {
try {
const pollResponse = await fetch(
`https://api.brightdata.com/discover?task_id=${encodeURIComponent(taskId)}`,
{
method: 'GET',
headers: {
Authorization: `Bearer ${params.apiKey}`,
},

@greptile-apps greptile-apps Bot Apr 15, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Elapsed-time counter under-counts wall-clock duration

elapsedTime only accumulates the sleep interval and ignores the time spent in each fetch call. On a slow network, where each poll round-trip takes seconds, the loop can run well past MAX_POLL_TIME_MS before the guard fires.

Replace with a wall-clock deadline:

Suggested change
let elapsedTime = 0
while (elapsedTime < MAX_POLL_TIME_MS) {
try {
const pollResponse = await fetch(
`https://api.brightdata.com/discover?task_id=${encodeURIComponent(taskId)}`,
{
method: 'GET',
headers: {
Authorization: `Bearer ${params.apiKey}`,
},
const deadline = Date.now() + MAX_POLL_TIME_MS
while (Date.now() < deadline) {

And remove elapsedTime += POLL_INTERVAL_MS at the end of the loop body.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread apps/sim/tools/brightdata/discover.ts
Comment on lines +141 to +147
if (!pollResponse.ok) {
return {
...result,
success: false,
error: `Failed to poll discover results: ${pollResponse.statusText}`,
}
}

@greptile-apps greptile-apps Bot Apr 15, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No retry on transient HTTP errors during polling

Any non-2xx response (including 429 Too Many Requests or 503 Service Unavailable) immediately terminates the poll and returns a failure. A single transient error aborts the entire discovery task, even though the Bright Data job itself may still be running.

Consider retrying on retriable status codes (e.g. 429, 5xx) up to a small limit before giving up, or at minimum logging the status code in the error message so callers can distinguish transient from permanent failures.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
cursor[bot]
cursor Bot reviewed Apr 15, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 377712c. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread apps/sim/app/api/copilot/chat/stop/route.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread apps/sim/lib/copilot/chat/effective-transcript.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

@icecrasher321
icecrasher321 merged commit c0bc62c into main Apr 16, 2026
31 checks passed

Uh oh!

There was an error while loading. Please reload this page.

This branch was successfully deployed

1 active deployment
Preview — 377712c9 Deployed Apr 15, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments
@cursor cursor[bot] cursor[bot] left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@icecrasher321 @waleedlatif1
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.