GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix(providers): correct max-tokens param and add schema guidance for NVIDIA/Z.ai by waleedlatif1 · Pull Request #5569 · simstudioai/sim

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #5560 addressing review findings that surfaced on #5567 (staging→main) after merge:

  • Real bug (both providers): NVIDIA NIM and Z.ai both document max_tokens for output-length control, not OpenAI's newer max_completion_tokens — verified against each provider's own API reference. The latter was silently ignored, so user-configured output caps were never applied. Cursor caught this for NVIDIA; I found the identical bug in Z.ai while fixing it and corrected both.
  • Real gap, best-effort mitigation (Z.ai): Z.ai's response_format has no json_schema mode (only text/json_object, re-verified against docs.z.ai), so schema/name/strict fields were silently dropped for structured-output requests. Now also injects the expected schema into the system prompt as guidance, in addition to the json_object hint — can't be enforced at the API level for this provider, but gives the model a real shot at matching the expected shape instead of nothing.
  • Not a bug (Z.ai): "forced tools become optional" is the documented, unavoidable consequence of Z.ai's API only supporting tool_choice: 'auto' (already logged as a warning) — no code change, reasoned pushback on the review thread.

Type of Change

  • Bug fix

Testing

  • bunx vitest run providers/models.test.ts — 18/18 passing
  • tsc --noEmit — zero new type errors
  • biome check — clean
  • No live API key exercised

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
greptile-apps[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@waleedlatif1
fix(providers): correct max-tokens param and add schema guidance for …
bb4869d 
…NVIDIA/Z.ai

- NVIDIA NIM and Z.ai both document max_tokens for output-length control,
  not OpenAI's newer max_completion_tokens - the latter was silently
  ignored by both vLLM-served NIM models and Z.ai's GLM models
- Z.ai has no json_schema response_format mode (only text/json_object),
  so structured-output requests now also inject the expected schema into
  the system prompt as best-effort guidance, since the request param
  alone can't enforce field names/types
@vercel

vercel Bot commented Jul 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 10, 2026 8:32pm

Request Review
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@cursor

cursor Bot commented Jul 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches live LLM request payloads for NVIDIA and Z.ai (max tokens and Z.ai structured-output prompts); behavior fixes are targeted but affect agent runs using those providers.

Overview
Fixes NVIDIA NIM and Z.ai so user-configured output limits are sent as max_tokens instead of max_completion_tokens, which those APIs ignore.

For Z.ai structured output (no native json_schema), the PR adds system-prompt schema guidance via buildSchemaGuidance / withSchemaGuidance on initial, deferred, and post-tool streaming/final calls, alongside the existing json_object hint.

models.ts drops only the long comment on Z.ai modelPatterns (behavior unchanged). NVIDIA/Z.ai files also trim redundant inline comments and document API quirks in block comments.

Reviewed by Cursor Bugbot for commit 159a48a. Configure here.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
cursor[bot]
cursor Bot reviewed Jul 10, 2026
View reviewed changes
Comment thread apps/sim/providers/zai/index.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.
@greptile-apps

greptile-apps Bot commented Jul 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates the NVIDIA and Z.ai provider adapters. The main changes are:

  • Sends max_tokens for NVIDIA NIM and Z.ai output limits.
  • Adds Z.ai schema guidance to structured-output requests.
  • Keeps Z.ai structured output on its supported json_object mode.
  • Documents provider-specific request parameter behavior.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
apps/sim/providers/models.ts Removes obsolete comments around Z.ai routing without changing provider behavior.
apps/sim/providers/nvidia/index.ts Maps configured output limits to NVIDIA NIM's max_tokens request field.
apps/sim/providers/zai/index.ts Maps configured output limits to Z.ai's max_tokens field and adds schema guidance for structured-output calls.

Reviews (3): Last reviewed commit: "fix(providers): scope Z.ai schema guidan..." | Re-trigger Greptile
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@waleedlatif1
style(providers): replace inline comments with TSDoc, per CLAUDE.md
1f1a45a 
Consolidated the scattered narrative // comments in nvidia/index.ts and
zai/index.ts into a single TSDoc block per provider documenting the
provider-specific API quirks; removed the rest where they only restated
what the code already shows. Also dropped an inline comment on zai's
modelPatterns field in models.ts.
@vercel
vercel Bot temporarily deployed to Preview July 10, 2026 20:24 Inactive
@waleedlatif1

waleedlatif1 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile
greptile-apps[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@waleedlatif1

waleedlatif1 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
cursor[bot]
cursor Bot reviewed Jul 10, 2026
View reviewed changes
Comment thread apps/sim/providers/zai/index.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.
@waleedlatif1
fix(providers): scope Z.ai schema guidance to the response_format cal…
159a48a 
…l only

- schemaGuidance now falls back to the bare responseFormat object when
  .schema is absent, matching the schema-or-format fallback used
  elsewhere in the codebase (was silently injecting nothing for callers
  that pass a bare JSON schema)
- guidance is now only appended to the messages sent alongside an
  actual response_format (the immediate call, or whichever pass
  deferResponseFormat applies it to) instead of every turn of an
  active tool loop, where it wrongly told the model to return final
  JSON instead of continuing to call tools
@vercel
vercel Bot temporarily deployed to Preview July 10, 2026 20:32 Inactive
@waleedlatif1

waleedlatif1 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile
greptile-apps[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@waleedlatif1

waleedlatif1 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review
cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
cursor[bot]
cursor Bot reviewed Jul 10, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 159a48a. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@waleedlatif1
waleedlatif1 merged commit ea4f269 into staging Jul 10, 2026
18 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@waleedlatif1
waleedlatif1 deleted the fix-nvidia-zai-followup branch July 10, 2026 21:21
@waleedlatif1 waleedlatif1 mentioned this pull request Jul 10, 2026
v0.7.29: GLM, ui and ux improvements #5567
Merged

This branch was previously deployed

1 inactive deployment
Preview — 159a48ae Deployed Jul 10, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@waleedlatif1
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.