GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix(gemini): support structured output with tools on Gemini 3 models by waleedlatif1 · Pull Request #4184 · simstudioai/sim

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Apr 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Gemini 3 models support responseJsonSchema alongside tools/function declarations in the same request — Gemini 2 does not
  • When an Agent block uses a Gemini 3 model with both tools and a response format, structured output was silently skipped, returning plain text instead of JSON
  • Added isGemini3Model() helper and branching logic to use responseJsonSchema (not responseSchema) for Gemini 3 + tools, preserving existing Gemini 2 behavior
  • Verified all scenarios against the live Gemini API: Gemini 3 + tools + schema, Gemini 3 no tools, Gemini 2 + tools (400 rejection), Gemini 2 no tools

Type of Change

  • Bug fix

Testing

Tested manually against the live Gemini API with comprehensive matrix covering all model/tools/schema combinations.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

greptile-apps[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
TheodoreSpeaks and others added 2 commits April 15, 2026 12:06
@TheodoreSpeaks @waleedlatif1
v0.6.29: login improvements, posthog telemetry (#4026)
ec8a2bf
* feat(posthog): Add tracking on mothership abort (#4023)

Co-authored-by: Theodore Li <[email protected]>

* fix(login): fix captcha headers for manual login  (#4025)

* fix(signup): fix turnstile key loading

* fix(login): fix captcha header passing

* Catch user already exists, remove login form captcha
@waleedlatif1
fix(gemini): support structured output with tools on Gemini 3 models
fc62dc5
@vercel

vercel Bot commented Apr 15, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Apr 15, 2026 7:12pm

Request Review

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@cursor

cursor Bot commented Apr 15, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes request/iteration config for Gemini structured outputs and tool loops; incorrect branching could alter response formatting or cause API rejections for some model/tool combinations.

Overview
Fixes structured JSON output when using Gemini 3 models with both tools and responseFormat by sending the schema via responseJsonSchema (while keeping Gemini 2’s behavior of applying responseSchema only when tools are absent or after tool execution).

Adds isGemini3Model() and threads model into the tool-iteration config builder to avoid reapplying/overwriting schema settings when stripping tools for the final response, and updates warnings/logging to distinguish Gemini 2 vs 3 behavior.

Reviewed by Cursor Bugbot for commit 8e93c05. Configure here.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@waleedlatif1
fix(home): remove duplicate handleStopGeneration declaration
13eaaee
@vercel
vercel Bot temporarily deployed to Preview April 15, 2026 19:09 Inactive
@greptile-apps

greptile-apps Bot commented Apr 15, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes structured output being silently skipped when using Gemini 3 models with tools by branching on isGemini3Model() to use responseJsonSchema (which Gemini 3 accepts alongside function declarations) instead of the responseSchema path that only works without tools. The prefix-based isGemini3Model helper is future-proof and already accounts for vertex/ prefixed variants.

Confidence Score: 5/5

Safe to merge — targeted bug fix with correct branching logic and no regressions to Gemini 2 behavior.

All findings are P2 or lower. The implementation correctly uses responseJsonSchema for Gemini 3 + tools, preserves it through the tool execution loop via baseConfig inheritance, and leaves Gemini 2 paths entirely unchanged. No P0/P1 issues found.

No files require special attention.

Important Files Changed

Filename Overview
apps/sim/providers/utils.ts Adds isGemini3Model() using a prefix check that handles vertex/ variants; clean and future-proof implementation.
apps/sim/providers/gemini/core.ts Branches Gemini 3 to use responseJsonSchema alongside tools in initial config and preserves it through buildNextConfig and streaming paths; Gemini 2 behavior unchanged.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Request: responseFormat + tools?] --> B{tools present?}
    B -- No --> C[responseSchema via cleanSchemaForGemini\nresponseMimeType = application/json]
    B -- Yes --> D{isGemini3Model?}
    D -- Yes --> E[responseJsonSchema = raw schema\nresponseMimeType = application/json\ntools kept in config]
    D -- No --> F[Warn: structured output deferred\ntools kept in config]
    E --> G[Tool execution loop]
    F --> G
    G --> H{More tool calls?}
    H -- Yes --> G
    H -- No --> I{isGemini3Model?}
    I -- Yes --> J[Strip tools\nresponseJsonSchema inherited from baseConfig]
    I -- No --> K[Strip tools\nApply responseSchema + responseMimeType]
    J --> L[Final structured response]
    K --> L
Loading

Reviews (2): Last reviewed commit: "refactor(gemini): use prefix-based Gemin..." | Re-trigger Greptile

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

greptile-apps[bot]
greptile-apps Bot reviewed Apr 15, 2026
View reviewed changes
Comment thread apps/sim/providers/utils.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

@waleedlatif1
refactor(gemini): use prefix-based Gemini 3 model detection
8e93c05
@vercel
vercel Bot temporarily deployed to Preview April 15, 2026 19:12 Inactive
@waleedlatif1

waleedlatif1 commented Apr 15, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

greptile-apps[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@waleedlatif1

waleedlatif1 commented Apr 15, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review

cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cursor[bot]
cursor Bot reviewed Apr 15, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8e93c05. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@waleedlatif1
waleedlatif1 merged commit 05c1c5b into staging Apr 15, 2026
14 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@waleedlatif1
waleedlatif1 deleted the waleedlatif1/debug-agent-response branch April 15, 2026 19:21
@waleedlatif1 waleedlatif1 mentioned this pull request Apr 15, 2026
Agent Response format not working #4180
Closed
@icecrasher321 icecrasher321 mentioned this pull request Apr 15, 2026
v0.6.45: superagent, csp, brightdata integration, gemini response format, logs performance improvements #4187
Merged
Sg312 added a commit that referenced this pull request Apr 15, 2026
@Sg312
v0.6.45: superagent, csp, brightdata integration, gemini response for…
010435c
…mat, logs performance improvements

fix(csp): add missing analytics domains, remove unsafe-eval, fix workspace CSP gap (#4179)
fix(landing): return 404 for invalid dynamic route slugs (#4182)
improvement(seo): optimize sitemaps, robots.txt, and core web vitals across sim and docs (#4170)
fix(gemini): support structured output with tools on Gemini 3 models (#4184)
feat(brightdata): add Bright Data integration with 8 tools (#4183)
fix(mothership): fix superagent credentials (#4185)
fix(logs): close sidebar when selected log disappears from filtered list; cleanup (#4186)

This branch was previously deployed

1 inactive deployment
Preview — 8e93c05c Deployed Apr 15, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments
@cursor cursor[bot] cursor[bot] left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@waleedlatif1 @TheodoreSpeaks
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.