GeezerVison

Two cursor schemes exist, both opaque base64-JSON from `response.ts`. Which one you use is decided by what the read can express, not by taste:

- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort is stamped into the cursor and re-checked on replay, so changing `sortBy` mid-pagination is a 400, not a silently skipped page.

- **Offset** (`decodeOffsetCursor` / `encodeCursor({ offset })`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query.

- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. An offset cursor **must** be stamped with `offsetCursorScope(...)` covering every param that filters or orders the sequence (not `limit`, which only selects how much of it to return). A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results — the exact failure the keyset's sort stamp already prevents.

**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure.

Two cursor schemes exist, both opaque base64-JSON from `response.ts`. Which one you use is decided by what the read can express, not by taste:

- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort is stamped into the cursor and re-checked on replay, so changing `sortBy` mid-pagination is a 400, not a silently skipped page.

- **Offset** (`decodeOffsetCursor` / `encodeCursor({ offset })`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query.

- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. An offset cursor **must** be stamped with `offsetCursorScope(...)` covering every param that filters or orders the sequence (not `limit`, which only selects how much of it to return). A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results — the exact failure the keyset's sort stamp already prevents.

**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure.

Two cursor schemes exist, both opaque base64-JSON from `response.ts`. Which one you use is decided by what the read can express, not by taste:

- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort is stamped into the cursor and re-checked on replay, so changing `sortBy` mid-pagination is a 400, not a silently skipped page.

- **Offset** (`decodeOffsetCursor` / `encodeCursor({ offset })`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query.

- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. An offset cursor **must** be stamped with `offsetCursorScope(...)` covering every param that filters or orders the sequence (not `limit`, which only selects how much of it to return). A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results — the exact failure the keyset's sort stamp already prevents.

**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure.

import { MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE } from '@/lib/uploads/shared/types'

import { validateFileType } from '@/lib/uploads/utils/validation'

import { serializeDate } from '@/app/api/v1/knowledge/utils'

import { decodeOffsetCursor, encodeCursor } from '@/app/api/v2/lib/response'

import {

decodeOffsetCursor,

encodeOffsetCursor,

offsetCursorScope,

} from '@/app/api/v2/lib/response'

export const dynamic = 'force-dynamic'

export const revalidate = 0

operation: knowledgeOperations.listDocuments,

rateLimit: v2RateLimits.publicApi,

errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization,

mapInput: ({ params, query }) => ({

knowledgeBaseId: params.id,

assertedWorkspaceId: query.workspaceId,

enabledFilter: query.enabledFilter,

search: query.search,

limit: query.limit,

offset: decodeOffsetCursor(query.cursor),

sortBy: query.sortBy,

sortOrder: query.sortOrder,

}),

mapInput: ({ params, query }) => {

/**

const cursorScope = offsetCursorScope({

knowledgeBaseId: params.id,

enabledFilter: query.enabledFilter,

search: query.search,

sortBy: query.sortBy,

sortOrder: query.sortOrder,

})

return {

knowledgeBaseId: params.id,

assertedWorkspaceId: query.workspaceId,

enabledFilter: query.enabledFilter,

search: query.search,

limit: query.limit,

offset: decodeOffsetCursor(query.cursor, cursorScope),

sortBy: query.sortBy,

sortOrder: query.sortOrder,

cursorScope,

}

},

useCase: listKnowledgeDocuments,

present: ({ documents, pagination }) => ({

present: ({ documents, pagination, cursorScope }) => ({

data: documents.map(toV2DocumentSummary),

nextCursor: pagination.hasMore

? encodeCursor({ offset: pagination.offset + pagination.limit })

? encodeOffsetCursor(cursorScope ?? '', pagination.offset + pagination.limit)

: null,

}),

}

interface OffsetCursorPayload {

/** The query state the offset counts positions within. */

scope: string

offset: number

export function offsetCursorScope(parts: Record<string, string | boolean | undefined>): string {

return Object.keys(parts)

.sort()

.map((key) => `${key}=${parts[key] ?? ''}`)

.join('&')

export function encodeOffsetCursor(scope: string, offset: number): string {

return encodeCursor({ scope, offset } satisfies OffsetCursorPayload)

export function decodeOffsetCursor(cursor: string | undefined): number {

export function decodeOffsetCursor(cursor: string | undefined, scope: string): number {

if (!cursor) return 0

const offset = decodeCursor<{ offset?: unknown }>(cursor)?.offset

const decoded = decodeCursor<Partial<OffsetCursorPayload>>(cursor)

if (!decoded || decoded.scope !== scope) {

throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE)

}

const { offset } = decoded

if (typeof offset !== 'number' || !Number.isInteger(offset) || offset < 0) {

throw new OrchestrationError('validation', 'Invalid cursor')

}

import { GET, POST } from '@/app/api/v2/skills/route'

const WORKSPACE_ID = 'workspace-1'

const SCOPE = ({

search = '',

sortBy = 'createdAt',

sortOrder = 'desc',

}: Record<string, string> = {}) =>

`search=${search}&sortBy=${sortBy}&sortOrder=${sortOrder}&workspaceId=${WORKSPACE_ID}`

const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_ID, keyId: 'key-1' }

const AUTH = {

principal: PRINCIPAL,

limit: 50,

cursor: undefined,

offset: 0,

cursorScope: SCOPE(),

},

request: expect.anything(),

})

})

it('resumes from the offset cursor and mints the next one while pages remain', async () => {

mocks.list.mockResolvedValueOnce({ skills: [skill], hasMore: true, offset: 2, limit: 2 })

const cursor = Buffer.from(JSON.stringify({ offset: 2 })).toString('base64')

mocks.list.mockResolvedValueOnce({

skills: [skill],

hasMore: true,

offset: 2,

limit: 2,

cursorScope: SCOPE(),

})

const cursor = Buffer.from(JSON.stringify({ scope: SCOPE(), offset: 2 })).toString('base64')

const response = await GET(

request(

expect(response.status).toBe(200)

expect((await response.json()).nextCursor).toBe(

Buffer.from(JSON.stringify({ offset: 4 })).toString('base64')

Buffer.from(JSON.stringify({ scope: SCOPE(), offset: 4 })).toString('base64')

)

expect(mocks.list).toHaveBeenCalledWith(

expect.objectContaining({ input: expect.objectContaining({ limit: 2, offset: 2 }) })

)

})

/**

it('rejects a cursor replayed under a different sort', async () => {

const cursor = Buffer.from(JSON.stringify({ scope: SCOPE(), offset: 2 })).toString('base64')

const response = await GET(

request(

'GET',

`/api/v2/skills?workspaceId=${WORKSPACE_ID}&sortBy=name&cursor=${encodeURIComponent(cursor)}`

)

)

expect(response.status).toBe(400)

expect(mocks.list).not.toHaveBeenCalled()

})

it('rejects a malformed cursor rather than silently restarting at page one', async () => {

const response = await GET(

request('GET', `/api/v2/skills?workspaceId=${WORKSPACE_ID}&cursor=not-a-cursor`)

import { captureServerEvent } from '@/lib/posthog/server'

import { skillOperations } from '@/lib/skills/application/operations'

import { createSkillUseCase, listSkillsUseCase } from '@/lib/skills/application/use-cases'

import { decodeOffsetCursor, encodeCursor } from '@/app/api/v2/lib/response'

import {

decodeOffsetCursor,

encodeOffsetCursor,

offsetCursorScope,

} from '@/app/api/v2/lib/response'

import { toV2Skill, toV2SkillSummary } from '@/app/api/v2/skills/utils'

function skillCursorScope(query: {

workspaceId: string

search?: string

sortBy: string

sortOrder: string

}): string {

return offsetCursorScope({

workspaceId: query.workspaceId,

search: query.search,

sortBy: query.sortBy,

sortOrder: query.sortOrder,

})

export const dynamic = 'force-dynamic'

export const revalidate = 0

auth: v2ApiKeyAuth,

rateLimit: v2RateLimits.publicApi,

errorPolicy: v2OrchestrationErrorPolicy,

mapInput: ({ query }) => ({ ...query, offset: decodeOffsetCursor(query.cursor) }),

mapInput: ({ query }) => {

/**

const scope = skillCursorScope(query)

return {

...query,

offset: decodeOffsetCursor(query.cursor, scope),

cursorScope: scope,

}

},

useCase: listSkillsUseCase,

present: ({ skills, hasMore, offset, limit }) => ({

present: ({ skills, hasMore, offset, limit, cursorScope }) => ({

data: skills.map(toV2SkillSummary),

nextCursor: hasMore ? encodeCursor({ offset: offset + limit }) : null,

nextCursor: hasMore ? encodeOffsetCursor(cursorScope, offset + limit) : null,

}),

import { describe, expect, it } from 'vitest'

import {

decodeOffsetCursor,

encodeOffsetCursor,

offsetCursorScope,

} from '@/app/api/v2/lib/response'

describe('offset cursor scope', () => {

const base = { workspaceId: 'ws-1', search: undefined, sortBy: 'name', sortOrder: 'asc' }

it('resumes a cursor replayed under the same query state', () => {

const scope = offsetCursorScope(base)

expect(decodeOffsetCursor(encodeOffsetCursor(scope, 40), scope)).toBe(40)

})

it('rejects a cursor replayed under a different sort', () => {

const cursor = encodeOffsetCursor(offsetCursorScope(base), 40)

expect(() =>

decodeOffsetCursor(cursor, offsetCursorScope({ ...base, sortBy: 'createdAt' }))

).toThrow(/does not match the requested/)

expect(() =>

decodeOffsetCursor(cursor, offsetCursorScope({ ...base, sortOrder: 'desc' }))

).toThrow(/does not match the requested/)

})

it('rejects a cursor replayed under a different filter', () => {

const cursor = encodeOffsetCursor(offsetCursorScope(base), 40)

expect(() =>

decodeOffsetCursor(cursor, offsetCursorScope({ ...base, search: 'deploy' }))

).toThrow(/does not match the requested/)

expect(() =>

decodeOffsetCursor(cursor, offsetCursorScope({ ...base, workspaceId: 'ws-2' }))

).toThrow(/does not match the requested/)

})

it('treats an absent cursor as page one', () => {

expect(decodeOffsetCursor(undefined, offsetCursorScope(base))).toBe(0)

})

it('rejects a cursor that is not valid base64-JSON', () => {

expect(() => decodeOffsetCursor('not-a-cursor', offsetCursorScope(base))).toThrow()

})

it('rejects an offset that is not a non-negative integer', () => {

const scope = offsetCursorScope(base)

expect(() => decodeOffsetCursor(encodeOffsetCursor(scope, -1), scope)).toThrow('Invalid cursor')

expect(() => decodeOffsetCursor(encodeOffsetCursor(scope, 1.5), scope)).toThrow(

'Invalid cursor'

)

})

/**

it('is unaffected by the page size', () => {

expect(offsetCursorScope({ ...base, sortBy: 'name' })).toBe(offsetCursorScope(base))

})

it('does not depend on the order the parts are written', () => {

expect(offsetCursorScope({ sortBy: 'name', workspaceId: 'ws-1', sortOrder: 'asc' })).toBe(

offsetCursorScope({ sortOrder: 'asc', workspaceId: 'ws-1', sortBy: 'name' })

)

})

sortBy?: DocumentSortField

sortOrder?: SortOrder

tagFilters?: TagFilterCondition[]

/**

cursorScope?: string

export interface ReadKnowledgeDocumentInput {

},

generateRequestId()

)

return { ...result, workspaceId: context.workspaceId }

return { ...result, workspaceId: context.workspaceId, cursorScope: input.cursorScope }

},