GeezerVison is showing github.com — fix(knowledge): stop capping the unpaged knowledge-base list by waleedlatif1 · Pull Request #6771 · simstudioai/sim
Conversation
Summary
- Staging's integration test failed on the first run after fix(knowledge): list knowledge bases on the same authority that creates them #6770 deployed:
GET /api/knowledge returned 500 with Knowledge base list exceeds the 10000 row limit. The same test had passed 28 consecutive runs over the prior two days against the same data — the delta was the code, not the volume.
- fix(knowledge): list knowledge bases on the same authority that creates them #6770 routed the internal list through the workspace read, which carried a 10,000-row cap. Soft-delete cleanup reclaims archived knowledge bases only past a retention window, and not at all where none is configured, so a workspace that archives faster than that window crosses any fixed count on its own.
- The cap could never have worked as a guard. Its throw was gated on
limit === undefined, so it fired only for callers that had not asked for a page — precisely the callers with no cursor to retry with and no way to ask for less. A paged caller never reached it. It also read one row past the cap before throwing, so it refused to serve rows it had already materialized; most of the memory cost was already paid.
- Removed. An unpaged read is unbounded, matching the sibling internal lists (
listTables, workspace files, neither capped). Paged callers keep their page and their nextCursorKeys. This also clears the same latent 500 in three other unpaged callers — the archived list use case, the catalog read, and the VFS name lookup — rather than only the surface that happened to fail.
Two more of the same shape, found while auditing for others:
attachConnectorTypes threw a bare Error above its own row cap, on those same unpaged callers. Archiving a knowledge base archives its connectors, so the growth curve that broke staging could not reach it — but it is the identical construct, and the sibling latestJobsForTables has no equivalent.
- The VFS path lookup read every knowledge base whose name merely contained the term and exact-matched in JS, so a single-row lookup scaled with the workspace. It now queries the exact name and reads two rows, mirroring
findActiveTablesByExactName.
Regression test reproduces the incident: it fails with the cap restored, passes without.
Type of Change
Testing
Tested manually. Full apps/sim suite, type-check, lint, and all 29 audits pass.
Two things worth separate follow-up, neither in this PR:
- The integration test's teardown leaves archived knowledge bases behind, which is how that workspace reached five figures.
sync-engine.ts accumulates every synced document's extracted text into one array with a page-count bound but no byte cap. That is where a bound would actually earn its keep.
Checklist
- Code follows project style guidelines
- Self-reviewed my changes
- Tests added/updated and passing
- No new warnings introduced
- I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
greptile-apps[bot] reacted with thumbs up emoji
vercel
Bot
commented
Aug 17, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
| Project |
Deployment |
Actions |
Updated (UTC) |
docs |
Skipped |
|
Aug 17, 2026 6:11am |

|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
cursor
Bot
commented
Aug 17, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
PR Summary
Medium Risk
Unpaged list endpoints can return very large result sets and heavier connector projections, trading prior hard failures for memory/latency risk on huge workspaces.
Overview
Fixes staging 500s on GET /api/knowledge when a workspace had more than 10,000 knowledge bases (often archived rows with slow cleanup). Unpaged workspace, legacy, and merged list reads no longer enforce MAX_KNOWLEDGE_BASES_PER_WORKSPACE / legacy / connector caps; paging with limit still uses limit + 1 for nextCursorKeys.
Copilot VFS rename/delete no longer loads the whole workspace with a substring search and filters in JS. They call new findActiveKnowledgeBasesByExactName (exact name, limit 2), aligned with findActiveTablesByExactName.
Removed unused constants from constants.ts. Tests cover unbounded unpaged reads, large archived sets, paging, and exact-name lookup.
Reviewed by Cursor Bugbot for commit bc7d114. Configure here.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Greptile Summary
The PR removes fixed caps from unpaged knowledge-base and connector projections while preserving keyset limits for explicitly paged reads. It also replaces VFS name resolution through a workspace-wide search with a bounded exact-name query.
- Makes unpaged workspace and legacy knowledge-base reads unbounded.
- Keeps paged workspace reads at
limit + 1 for cursor generation.
- Adds an exact-name, two-row VFS lookup.
- Updates regression and paging tests.
Confidence Score: 5/5
The PR appears safe to merge because no blocking failure remains.
No blocking failure remains.
Important Files Changed
| Filename |
Overview |
| apps/sim/lib/knowledge/service.ts |
Removes unpaged row and connector projection caps, preserves explicit paging, and adds a bounded exact-name lookup. |
| apps/sim/lib/knowledge/application/knowledge-vfs.ts |
Switches VFS knowledge-base resolution from workspace list filtering to the exact-name service query. |
| apps/sim/lib/knowledge/service.test.ts |
Replaces cap-failure coverage with unbounded-read, cursor paging, exact-name lookup, and oversized archived-list regression coverage. |
| apps/sim/lib/knowledge/constants.ts |
Removes constants for the deleted knowledge-base and connector projection limits. |
Reviews (2): Last reviewed commit: "fix(knowledge): stop capping the unpaged..." | Re-trigger Greptile
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
#6770 routed GET /api/knowledge through the workspace read, which carried a
10,000-row cap. Staging crossed it, and the first list request after the deploy
returned 500 with "Knowledge base list exceeds the 10000 row limit" — against
data that had served fine for days.
The cap could never have worked. Its throw was guarded by `limit === undefined`,
so it fired only for callers that had NOT asked for a page: exactly the callers
with no cursor to retry with and no way to ask for less. A paged caller never
reached it. It also read one row PAST the cap before throwing, so it refused to
serve rows it had already materialized — most of the memory was already spent.
Soft-delete cleanup reclaims archived rows only past a retention window, and not
at all where none is configured, so a workspace that archives faster than that
window crosses any fixed count on its own.
Remove it. An unpaged read is unbounded, matching the sibling internal lists
(`listTables`, workspace files), and paged callers keep their page. That fixes
the same latent 500 in the archived list, the catalog read, and the VFS name
lookup, which are all unpaged too, rather than only the surface that failed.
Two more of the same shape found while auditing for others:
- `attachConnectorTypes` threw a bare Error above its own cap, on those same
unpaged callers. Archiving a knowledge base archives its connectors, so the
growth curve that broke staging could not reach it — but it is the identical
construct, and the sibling `latestJobsForTables` has no equivalent.
- The VFS path lookup read every knowledge base whose name merely CONTAINED the
term and then exact-matched in JS, so a single-row lookup scaled with the
workspace. It now queries the exact name and reads two rows, mirroring
`findActiveTablesByExactName`.
greptile-apps[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
cursor[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit bc7d114. Configure here.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
This branch was previously deployed
1 inactive
deployment
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|