GeezerVison is showing github.com — feat(script): migrate agent/router/evaluator LLM keys to BYOK by model prefix by TheodoreSpeaks · Pull Request #5607 · simstudioai/sim
Conversation
Summary
- Extend the block→BYOK migration script with a model-aware mode (
--model-map <modelPrefix>=<providerId>, e.g. grok=xai)
- Lets existing LLM keys entered into
agent/router/evaluator blocks migrate to workspace BYOK, so they keep using their own key instead of falling back to the rotating key once the provider joins the hosted pool
Why a new mode
The existing --map <blockType>=<providerId> only works for dedicated single-provider blocks. LLM-family blocks (agent, router, evaluator) share one model-gated apiKey subblock across every provider, so a block-type map would misattribute every provider's key. The new --model-map only takes the key when the block's selected model starts with the given prefix.
Type of Change
Testing
- Dry run + live run verified against staging; migrated rows well-formed (
iv:encrypted:authTag), created_by set, one key per workspace with full coverage of eligible blocks
bun run check:api-validation:strict passed
- Biome formatting applied
Checklist
- Code follows project style guidelines
- Self-reviewed my changes
- Tests added/updated and passing
- No new warnings introduced
- I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
greptile-apps[bot] reacted with thumbs up emoji
vercel
Bot
commented
Jul 11, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
cursor
Bot
commented
Jul 11, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
PR Summary
Medium Risk
The script inserts encrypted workspace BYOK rows from user API key material; wrong --model-map or missing --require-key-prefix could attribute keys to the wrong provider, though dry-run, conflict handling, and the new guards reduce that risk.
Overview
Extends migrate-block-api-keys-to-byok.ts so LLM-family blocks (agent, router, router_v2, evaluator, translate, guardrails, pi) can migrate shared apiKey values into workspace BYOK using --model-map <modelPrefix>=<providerId> (e.g. grok=xai), only when the block’s selected model matches the prefix. --map is rejected for those block types to avoid mis-attributing one shared key to a single provider.
Adds optional --require-key-prefix <providerId>=<prefix> to skip stale keys left in the shared field after a model switch, and longest-prefix-first resolution so overlapping --model-map entries are deterministic. Startup logging and workspace discovery now account for model mappings alongside existing block-type maps; --map values are trimmed and lowercased.
Reviewed by Cursor Bugbot for commit 5cee10c. Bugbot is set up for automated code reviews on this repo. Configure here.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Greptile Summary
This PR adds model-aware BYOK migration for shared LLM block credentials. The main changes are:
- New
--model-map support for model-prefix to provider mapping.
- Rejection of unsafe
--map usage for LLM-family blocks.
- Optional
--require-key-prefix filtering for stale shared keys.
- Provider ID normalization and trimmed CLI parsing.
- Workspace discovery for LLM-family blocks when model mappings are used.
Confidence Score: 5/5
This looks safe to merge.
- No blocking issues found in the changed code.
Important Files Changed
| Filename |
Overview |
| packages/db/scripts/migrate-block-api-keys-to-byok.ts |
Adds model-prefix based BYOK migration for LLM-family blocks and tightens CLI parsing around provider mapping. |
Reviews (5): Last reviewed commit: "fix(script): resolve overlapping model p..." | Re-trigger Greptile
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
|
|
|
|
// LLM-family blocks share one model-gated `apiKey` subblock across every provider, so the
|
|
|
// key is only attributed to a provider when the block's selected `model` matches a prefix.
|
|
|
const LLM_FAMILY_BLOCK_TYPES = ['agent', 'router', 'evaluator'] as const
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@TheodoreSpeaks what about translate, I know we deprecated vision? and idk if there are more or not
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch — added `translate` and `guardrails` (both use `getModelOptions()` + the shared `getProviderCredentialSubBlocks()` apiKey, same as agent/router/evaluator, so they can hold a grok key). Left `vision` out on purpose: it is deprecated and its curated `VISION_MODEL_OPTIONS` list has no grok models, so it can never hold an xai key. The set is now ['agent','router','evaluator','translate','guardrails'].
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
TheodoreSpeaks
changed the title
feat(byok): migrate agent/router/evaluator LLM keys to BYOK by model prefix
feat(script): migrate agent/router/evaluator LLM keys to BYOK by model prefix
Jul 12, 2026
…er ids, reject mixed mapping, add key-prefix guard
|
Addressed the review feedback in 1a3ef5f8:
- @waleedlatif1 — added
translate and guardrails to the LLM-family set (same getModelOptions() + shared getProviderCredentialSubBlocks() apiKey structure). Excluded deprecated vision since its curated model list has no grok models.
- Greptile (whitespace provider rows) — provider ids are now
.trim()-ed in both --map and --model-map parsers.
- Greptile / Bugbot (mixed mapping duplicates) —
--map targeting an LLM-family block type is now rejected at startup, so an LLM block can never be processed by both the block-type and model-map paths.
- Bugbot (stale keys misattributed) — added an opt-in
--require-key-prefix <providerId>=<prefix> guard. The live xai run will use --require-key-prefix xai=xai- so a stale non-xai key left in the shared apiKey field after a model switch is skipped rather than migrated. (Note: a stale non-grok key would already have been failing that block's grok runs pre-migration, so this closes the remaining edge where it could shadow the hosted rotating key.)
@greptile review
|
greptile-apps[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
Round 2 addressed in 948721f2:
- Bugbot (missing
pi block) — good catch, added pi to LLM_FAMILY_BLOCK_TYPES. It uses getPiModelOptions() (a filter over getModelOptions()) + the shared getProviderCredentialSubBlocks() apiKey, and xai is in PI_SUPPORTED_PROVIDER_IDS, so Pi blocks can hold a grok key.
- Greptile (trim prefix value) —
--require-key-prefix now trims the prefix too, so xai= xai- no longer rejects every valid xai-... key.
@greptile review
|
greptile-apps[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
Round 3 addressed in 02f06325:
- Bugbot (active router type omitted, High) — correct, added
router_v2. router.ts defines both RouterBlock (router, legacy/hideFromToolbar) and RouterV2Block (router_v2, active); both are registered and both use the shared getProviderCredentialSubBlocks() apiKey, so both can hold a grok key. Verified the full set against every block that spreads getProviderCredentialSubBlocks() — it's now agent, router, router_v2, evaluator, translate, guardrails, pi (complete).
- Greptile (canonicalize provider ids) — provider ids are now lowercased in all three parsers (
--map, --model-map, --require-key-prefix), so grok=XAI stores xai and matches the runtime BYOK lookup.
@greptile review
|
greptile-apps[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 02f0632. Configure here.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
Latest re-review triage (HEAD is now 5cee10ca):
- Bugbot — "Prefix map picks shortest match" (Low) — valid, fixed in
5cee10ca. resolveModelProvider now iterates prefixes longest-first, so the most specific mapping wins deterministically regardless of arg order.
- Bugbot — "Active router type omitted" (High) — ⚠️ false positive.
router_v2 was already added in 02f06325; it's on line 87 of LLM_FAMILY_BLOCK_TYPES (agent, router, router_v2, evaluator, translate, guardrails, pi). The comment re-posted the round-2 text against the commit that already fixed it.
- Greptile — "Canonicalize Provider Ids" — already fixed; that comment reviewed the older commit
948721, but 02f06325 lowercases provider ids in all three parsers (lines 65, 101, 156).
@greptile review
|
greptile-apps[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
This branch was successfully deployed
1 active
deployment
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|