fix(models): correct model catalog data and Gemini thinking-config wire format by waleedlatif1 · Pull Request #5559 · simstudioai/sim

waleedlatif1 · 2026-07-10T07:37:15Z

Summary

Noticed the models comparison page was showing mostly OpenAI models and asked for a /validate-model sweep across OpenAI, Anthropic, and Gemini — found and fixed real data bugs, not just cosmetic ones
OpenAI: removed a fabricated 'max' reasoning-effort value from the gpt-5.6 family (not a documented value, would 400 on selection)
Anthropic: fixed claude-sonnet-4-6 maxOutputTokens (was 64k, docs say 128k); removed 3 models that are fully retired per Anthropic's own deprecations page (claude-opus-4-0, claude-sonnet-4-0, claude-3-haiku-20240307 — all past their retirement dates, requests would fail); fixed a budget_tokens >= max_tokens bug that would 400 on claude-opus-4-1 at its default thinking level
Google/Vertex: un-deprecated gemini-3-flash-preview (no shutdown date actually announced); added the thinking capability to gemini-2.5-pro/flash/flash-lite (google + vertex), which were missing it entirely despite Google supporting thinking control for them
Fixed gemini/core.ts to send thinkingBudget (not thinkingLevel) for Gemini 2.5-series models — confirmed via multiple independent sources that 2.5-series rejects thinkingLevel outright and only Gemini 3.x supports it, so the new capability above would have 400'd without this
Bedrock: marked claude-opus-4-1 deprecated per AWS's own Bedrock lifecycle page (Legacy since Jul 8 2026 — a separate schedule from Anthropic's direct API)
Audited Azure OpenAI/Anthropic and Bedrock for missing recent models; deliberately did not add anything there — Azure's pricing feed doesn't have gpt-5.6 yet and the rest required guessing at abbreviated meter names, Bedrock's newer Claude models need a different API surface than what's currently wired up

Type of Change

Bug fix

Testing

Every changed field independently verified against live provider docs, cross-referenced against a second source where available (OpenRouter, AWS pricing/lifecycle pages, Anthropic's own pricing page)
Ran a dedicated /validate-model pass on every single modified model, one at a time, across all affected providers
Added regression tests for the new mapToThinkingBudget gemini/core.ts fix
Full providers/ test suite: 753/753 passing
bun run type-check and bun run lint clean on all touched files

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…re format - OpenAI: remove fabricated 'max' reasoning-effort value from gpt-5.6 family - Anthropic: fix claude-sonnet-4-6 maxOutputTokens (64k -> 128k); remove 3 fully retired models (claude-opus-4-0, claude-sonnet-4-0, claude-3-haiku-20240307); fix budget_tokens/max_tokens clamp that could send budget_tokens >= max_tokens for claude-opus-4-1 at its default thinking level - Google/Vertex: un-deprecate gemini-3-flash-preview (no shutdown date announced); add thinking capability to gemini-2.5-pro/flash/flash-lite (google + vertex) - Fix gemini/core.ts to send thinkingBudget (not thinkingLevel) for Gemini 2.5-series models, which reject thinkingLevel entirely - only Gemini 3.x supports it - Bedrock: mark claude-opus-4-1 deprecated per AWS's own Bedrock lifecycle schedule (Legacy since Jul 8 2026, separate from Anthropic's direct-API date)

vercel · 2026-07-10T07:37:20Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 10, 2026 4:04pm

cursor · 2026-07-10T07:37:25Z

PR Summary

Medium Risk
Changes live LLM request shaping (thinking budgets and max_tokens) and model capability metadata that drives UI and billing helpers; wrong values cause API errors or unexpected thinking behavior, but scope is provider-layer with added unit tests.

Overview
Aligns the model catalog with provider docs and fixes request payloads that could 400 when users pick thinking or output limits.

Anthropic extended-thinking requests now shrink budget_tokens (respecting the 1024 floor and output headroom) when the model’s max output cap is too small—not only bump max_tokens—so budget_tokens < max_tokens stays valid on tighter models.

Gemini sends thinkingLevel only on Gemini 3.x; 2.5-series uses per-model thinkingBudget mappings, and thinkingBudget: 0 when thinking is set to none on flash variants that support disable (omitting config would leave thinking on by default).

Catalog / UI metadata: claude-sonnet-4-6 maxOutputTokens → 128k; thinking levels added for gemini-2.5-pro/flash/flash-lite (Google + Vertex); removed fabricated 'max' reasoning effort on gpt-5.6 models; deprecation flags adjusted (gemini-3-flash-preview un-deprecated, bedrock claude-opus-4-1 deprecated). Provider utils tests updated for current Claude IDs and the new Sonnet 4.6 output cap.

^{Reviewed by Cursor Bugbot for commit 1103ca5. Configure here.}

greptile-apps · 2026-07-10T07:39:42Z

Greptile Summary

This PR updates model catalog data and provider thinking configuration. The main changes are:

Corrected OpenAI, Anthropic, Gemini, Vertex, and Bedrock model metadata.
Restored retired Anthropic model IDs as deprecated entries for saved workflow resolution.
Switched Gemini 2.5 thinking control to thinkingBudget while keeping Gemini 3.x on thinkingLevel.
Added tests for Gemini thinking-budget helpers and updated provider capability expectations.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
apps/sim/providers/models.ts	Updates provider model metadata and keeps retired Anthropic IDs available as deprecated catalog entries.
apps/sim/providers/gemini/core.ts	Sends Gemini 2.5 thinking settings as budgets and Gemini 3.x settings as levels.
apps/sim/providers/google/utils.ts	Adds Gemini 2.5 thinking-budget mapping and explicit-disable detection helpers.
apps/sim/providers/anthropic/core.ts	Adjusts Anthropic thinking budget handling so payloads stay within model token limits.
apps/sim/providers/google/utils.test.ts	Adds coverage for Gemini thinking-budget mapping and disable-support detection.
apps/sim/providers/utils.test.ts	Updates provider utility expectations for the revised model catalog.

_{Reviews (4): Last reviewed commit: "chore(models): tighten inline comments i..." | Re-trigger Greptile}

…removing Greptile caught a real backward-compat regression: fully removing claude-opus-4-0/claude-sonnet-4-0/claude-3-haiku-20240307 dropped them from getHostedModels()/shouldBillModelUsage(), so saved workflows still referencing them would fail on a missing-API-key error instead of Anthropic's actual "model retired" error. deprecated:true isn't consumed by the model picker (only copilot's serializer reads it), so restoring them this way costs nothing on hiding from new selection while preserving hosted-key resolution for existing references.

waleedlatif1 · 2026-07-10T07:45:32Z

@greptile

waleedlatif1 · 2026-07-10T07:45:33Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ec43215. Configure here.}

…2.5 disable-thinking gap Found during a final per-model audit round with independent 2-3 source verification on every changed model: - gpt-5.6-sol: restore 'max' reasoning-effort value. Multiple independent sources (OpenAI's own model-guidance docs page, launch announcement, and press coverage) confirm 'max' is a real, newly-launched value exclusive to Sol - not fabricated as originally assessed. Terra and Luna correctly do NOT get 'max' (confirmed Sol-exclusive), so they're unchanged. - gemini-2.5-flash / gemini-2.5-flash-lite (google + vertex): selecting 'none' for thinking level was sending no thinkingConfig at all, which falls back to the API's dynamic default (thinking stays ON for flash) - not actually disabling it, even though both models explicitly support thinkingBudget:0. Now sends an explicit budget of 0 for these two models specifically (gemini-2.5-pro is correctly excluded - it cannot disable thinking at all, floor is 128 not 0).

waleedlatif1 · 2026-07-10T15:29:31Z

@greptile

waleedlatif1 · 2026-07-10T15:29:32Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6b4e02a. Configure here.}

Trim verbose multi-line comments to single concise lines and remove duplication with the TSDoc already on the ANTHROPIC_MIN_BUDGET_TOKENS/ ANTHROPIC_THINKING_OUTPUT_HEADROOM constants. No behavior change - verified against the full test suite (811 files, 11141 tests, all passing).

waleedlatif1 · 2026-07-10T16:04:41Z

@greptile

waleedlatif1 · 2026-07-10T16:04:43Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1103ca5. Configure here.}

greptile-apps Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread apps/sim/providers/models.ts

vercel Bot temporarily deployed to Preview July 10, 2026 07:45 Inactive

cursor Bot reviewed Jul 10, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview July 10, 2026 15:29 Inactive

cursor Bot reviewed Jul 10, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview July 10, 2026 16:04 Inactive

cursor Bot reviewed Jul 10, 2026

View reviewed changes

waleedlatif1 merged commit 7d1c927 into staging Jul 10, 2026
13 checks passed

waleedlatif1 deleted the validate-model-comparison branch July 10, 2026 16:08

Uh oh!

Conversation

waleedlatif1 commented Jul 10, 2026

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

greptile-apps Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

waleedlatif1 commented Jul 10, 2026

Uh oh!

waleedlatif1 commented Jul 10, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Jul 10, 2026

Uh oh!

waleedlatif1 commented Jul 10, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Jul 10, 2026

Uh oh!

waleedlatif1 commented Jul 10, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 10, 2026 •

edited

Loading

cursor Bot commented Jul 10, 2026 •

edited

Loading

greptile-apps Bot commented Jul 10, 2026 •

edited

Loading