GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Show connector errors in the profiler by Karakatiza666 · Pull Request #7491 · feldera/feldera

Show connector errors in the profiler #7491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Karakatiza666 merged 1 commit into main from profiler-connector-errors
Oct 31, 2026

Uh oh!

There was an error while loading. Please reload this page.

Merged

Show connector errors in the profiler #7491

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
6b8e6d5
Show connector errors in the profiler
Karakatiza666 Oct 14, 2026
File filter
Filter by extension

Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.

Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.

Diff view
Diff view

There are no files selected for viewing

2 changes: 1 addition & 1 deletion 2 js-packages/common-ui/README.md
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { SegmentedControl, TabsPanel, PersistentContent, MonacoEditor } from 'co
| Layout | `Drawer` shows a resizable side panel on wide screens (`InlineDrawer`) and an overlay on tablets and phones (`OverlayDrawer`). `InlineDropdown` toggles content under a header. `useIsMobile.svelte` has the screen-size media queries. |
| Layout helpers | `PersistentContent` and `persistentRect.svelte` for content that must survive conditional re-renders (see those files for details). `ANSIDecoratedText` renders ANSI-colored log output. |
| Buttons | `ClipboardCopyButton`. `ClickFeedback` shows a short confirmation state after a click. |
| Pipeline views | `ConnectorMetricsTables` shows the connector statistics of each table and view. Use `aggregateConnectorMetrics` from `common-ui/connectorMetrics` to make its input from a `/stats` response. |
| Pipeline views | `ConnectorMetricsTables` shows the connector statistics of each table and view. Use `aggregateConnectorMetrics` from `common-ui/connectorMetrics` to make its input from a `/stats` response. `ConnectorErrors` shows the error lists of one connector. The caller passes the connector status, from the API or from a support bundle. |
| Health views | `StatusTimeline`, `EventLogList` and `HealthEventList` show a status history, its incidents and the events of one incident. `PipelineEventHistory` combines them for the monitor events of a pipeline, from the API or from a support bundle. |

Plain TypeScript modules that only the web console and the profiler use also
Expand Down
198 changes: 198 additions & 0 deletions 198 js-packages/common-ui/src/lib/ConnectorErrors.svelte
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
<script lang="ts">
import { formatDateTime, formatServerTime } from 'common-lib/format'
import Dayjs from 'dayjs'
import { fade, slide } from 'svelte/transition'
import type {
ConnectorError,
ConnectorErrorFilter,
ConnectorErrorKind,
InputEndpointStatus,
OutputEndpointStatus
} from './connectorMetrics'
import { getCaseDependentName } from './felderaRelation'
import InlineDropdown from './InlineDropdown.svelte'
import SegmentedControl from './SegmentedControl.svelte'
import Tooltip from './Tooltip.svelte'

const inputFilterOptions = ['all', 'parse', 'transport'] as const
const outputFilterOptions = ['all', 'encode', 'transport'] as const

let filterOptionLabels: Record<string, string> = {
transport: 'Transport',
parse: 'Parse',
encode: 'Encode',
all: 'All'
}

let tagsFilter: ConnectorErrorFilter = $state('all')

let {
relationName,
connectorName,
direction,
filter = 'all',
status,
loading = false,
onClose
}: {
relationName: string
/** Endpoint name, prefixed with the relation name. */
connectorName: string
direction: 'input' | 'output'
filter?: ConnectorErrorFilter
/** The connector status with its error lists. `null` while it is not available. */
status: InputEndpointStatus | OutputEndpointStatus | null
loading?: boolean
onClose: () => void
} = $props()

const filterOptions = $derived(direction === 'input' ? inputFilterOptions : outputFilterOptions)
const filterItems = $derived(
filterOptions.map((value) => ({ value, label: filterOptionLabels[value] ?? value }))
)

$effect(() => {
tagsFilter = filter
})

const strippedConnectorName = $derived(
connectorName.slice(getCaseDependentName(relationName).name.length + 1)
)

/** One row of the error list: an error, or a gap. A gap stands for `count` errors that are
* missing between two listed errors, by `ConnectorError.index`, e.g. older errors that the
* pipeline dropped because it keeps only the most recent ones. */
type DisplayItem = { kind: 'error'; error: ConnectorError } | { kind: 'gap'; count: number }
Comment thread

@mihaibudiu mihaibudiu Oct 18, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "gap" a set of missing messages?
Please document this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

function withGaps(errors: ConnectorError[]): DisplayItem[] {
const byTag = new Map<string, ConnectorError[]>()
for (const e of errors) {
const k = e.tag ?? 'other'
const existing = byTag.get(k)
if (existing) {
existing.push(e)
} else {
byTag.set(k, [e])
}
}
const result: DisplayItem[] = []
for (const tagErrors of byTag.values()) {
const sorted = [...tagErrors].sort((a, b) => a.index - b.index)
for (let i = 0; i < sorted.length; i++) {
if (i > 0) {
const gap = sorted[i].index - sorted[i - 1].index - 1
if (gap > 0) {
result.push({ kind: 'gap', count: gap })
}
}
result.push({ kind: 'error', error: sorted[i] })
}
}
return result
}

/** Returns the error lists of the connector by error kind, e.g. `{ parse: [...], transport:
* [...] }` for an input connector. If no errors for an error kind are there the field remains
* with an empty array. */
function getErrorsByKind(
s: InputEndpointStatus | OutputEndpointStatus
): Partial<Record<ConnectorErrorKind, ConnectorError[]>> {
if (direction === 'input') {
return {
parse: (s as InputEndpointStatus).parse_errors ?? [],
transport: s.transport_errors ?? []
}
}
return {
encode: (s as OutputEndpointStatus).encode_errors ?? [],
transport: s.transport_errors ?? []
}
}

const displayItems = $derived.by(() => {
if (!status) {
return [] as DisplayItem[]
}
const errorsByKind = getErrorsByKind(status)
const combined =
tagsFilter === 'all' ? Object.values(errorsByKind).flat() : (errorsByKind[tagsFilter] ?? [])
return withGaps(combined).reverse()
})
</script>

<div class="bg-white-dark flex h-full flex-col gap-2 rounded px-4 pt-4">
<div class="flex items-start justify-between">
<div>
<div class="font-medium">{relationName} · {strippedConnectorName}</div>
</div>
<button class="fd fd-x text-[20px]" onclick={onClose} aria-label="Close"></button>
</div>
<div class="flex flex-nowrap items-center gap-2">
<SegmentedControl
value={tagsFilter}
onValueChange={(v) => (tagsFilter = v as ConnectorErrorFilter)}
items={filterItems}
class="-mt-2"
/>
<span class="text-surface-500">errors</span>
</div>
<div class="-mr-4 scrollbar flex-1 overflow-y-auto pr-4">
{#if loading}
<div class="p-2 text-surface-500">Loading…</div>
{:else if displayItems.length === 0}
<div class="p-2 text-surface-500">
{#if tagsFilter === 'all'}
No errors
Comment thread

@mythical-fred-oss mythical-fred-oss Bot Oct 15, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pitfall (medium): For a bundle collected before include_connector_errors, the tab label shows e.g. Connectors 3 and the table shows the error icon, but clicking it says "No errors". That is misleading during triage. When the status has no parse_errors/encode_errors/transport_errors key at all (undefined, not []), say something like "This bundle was collected without connector error details" instead.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
{:else}
No errors of this type
{/if}
</div>
{:else}
{#each displayItems as item}
{#if item.kind === 'gap'}
<div class="px-2 py-1 text-center text-surface-500 italic">
{#if item.count !== 1}
{item.count} other errors skipped
{:else}
{item.count} other error skipped
{/if}
</div>
{:else}
<InlineDropdown>
{#snippet header(isOpen, toggle)}
<div
class="flex w-full cursor-pointer items-center gap-2 py-2 pr-2"
onclick={toggle}
role="presentation"
>
<div
class="fd fd-chevron-down text-[20px] transition-transform {isOpen
? 'rotate-180'
: ''}"
></div>
<span class="flex-1 overflow-hidden text-nowrap text-ellipsis">
{#if !isOpen}
<span out:fade={{ duration: 100 }}>{item.error.message}</span>
{/if}
</span>
{#if status?.fatal_error != null && item.error.message === status.fatal_error}
<span class="fd fd-circle-x text-error-500"></span>
{/if}
<span class="ml-auto">{formatServerTime(item.error.timestamp)}</span>
<Tooltip placement="top">{formatDateTime(Dayjs(item.error.timestamp))}</Tooltip>
</div>
{/snippet}
{#snippet content()}
<div
class="mr-16 -mb-7 ml-7 -translate-y-7 pb-2"
transition:slide={{ duration: 150 }}
>
{item.error.message}
</div>
{/snippet}
</InlineDropdown>
{/if}
{/each}
{/if}
</div>
</div>
36 changes: 35 additions & 1 deletion 36 js-packages/common-ui/src/lib/connectorMetrics.ts
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,16 @@ export type OutputEndpointMetrics = {
transmitted_records: number
}

/** An error reported by a connector. */
export type ConnectorError = {
/** Sequence number of the error among the errors of one kind: parse, encode or transport. */
index: number
message: string
tag?: string | null
/** RFC3339 timestamp */
timestamp: string
}

export type InputEndpointStatus = {
endpoint_name: string
config: { stream: string }
Expand All @@ -47,6 +57,9 @@ export type InputEndpointStatus = {
barrier: boolean
health?: ConnectorHealth | null
fatal_error?: string | null
/** The most recent errors. Absent when the status was collected without connector errors. */
parse_errors?: ConnectorError[] | null
transport_errors?: ConnectorError[] | null
}

export type OutputEndpointStatus = {
Expand All @@ -57,6 +70,9 @@ export type OutputEndpointStatus = {
paused?: boolean
health?: ConnectorHealth | null
fatal_error?: string | null
/** The most recent errors. Absent when the status was collected without connector errors. */
encode_errors?: ConnectorError[] | null
transport_errors?: ConnectorError[] | null
}

/** The part of a `/stats` response that `aggregateConnectorMetrics` reads. */
Expand All @@ -70,8 +86,12 @@ export type ConnectorStatus = {
}
}

/** An input connector reports parse and transport errors. An output connector reports encode and
* transport errors. */
export type ConnectorErrorKind = 'parse' | 'transport' | 'encode'

/** The kind of connector errors to show when a connector is selected. */
export type ConnectorErrorFilter = 'all' | 'parse' | 'transport' | 'encode'
export type ConnectorErrorFilter = 'all' | ConnectorErrorKind

export type AggregatedMetrics<M, EndpointStatus = {}> = {
aggregate: { metrics: M }
Expand Down Expand Up @@ -100,6 +120,20 @@ export type ConnectorMetrics = {
views: Map<string, AggregatedOutputEndpointMetrics>
}

/** Sum of the error counters that the connector tables show, over all connectors. */
export const countConnectorErrors = ({ tables, views }: ConnectorMetrics): number => {
let count = 0
for (const { aggregate } of tables.values()) {
count +=
(aggregate.metrics.num_parse_errors ?? 0) + (aggregate.metrics.num_transport_errors ?? 0)
}
for (const { aggregate } of views.values()) {
count +=
(aggregate.metrics.num_encode_errors ?? 0) + (aggregate.metrics.num_transport_errors ?? 0)
}
return count
}

/**
* Higher of two connector latencies, ignoring connectors without samples.
*
Expand Down
1 change: 1 addition & 0 deletions 1 js-packages/common-ui/src/lib/index.ts
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ export { default as ANSIDecoratedText } from './ANSIDecoratedText.svelte'
export { default as ClickFeedback } from './ClickFeedback.svelte'
export { default as ClipboardCopyButton } from './ClipboardCopyButton.svelte'
export { default as ConnectorMetricsTables } from './ConnectorMetricsTables.svelte'
export { default as ConnectorErrors } from './ConnectorErrors.svelte'
export { default as Drawer } from './Drawer.svelte'
export { default as EventLogList } from './EventLogList.svelte'
export { default as HealthEventList } from './HealthEventList.svelte'
Expand Down
5 changes: 4 additions & 1 deletion 5 js-packages/profiler-app/src/App.svelte
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<script lang="ts">
import type { ConnectorMetrics } from 'common-ui/connectorMetrics'
import type { ConnectorMetrics, ConnectorStatus } from 'common-ui/connectorMetrics'
import type { PipelineMonitorEvent } from 'common-ui/pipelineHealth'
import {
createLoadGuard,
Expand Down Expand Up @@ -28,6 +28,7 @@
logText: string | undefined
globalMetrics: GlobalMetrics | undefined
connectorMetrics: ConnectorMetrics | undefined
connectorStatus: ConnectorStatus | undefined
runtimeConfig: unknown
pipelineEvents: PipelineMonitorEvent[] | undefined
sourceFiles: SourceFile[] | undefined
Expand All @@ -54,6 +55,7 @@
logText: data.logText,
globalMetrics: data.globalMetrics,
connectorMetrics: data.connectorMetrics,
connectorStatus: data.connectorStatus,
runtimeConfig: data.runtimeConfig,
pipelineEvents: data.pipelineEvents,
sourceFiles: data.sourceFiles
Expand Down Expand Up @@ -124,6 +126,7 @@
logText={profileData.logText}
globalMetrics={profileData.globalMetrics}
connectorMetrics={profileData.connectorMetrics}
connectorStatus={profileData.connectorStatus}
runtimeConfig={profileData.runtimeConfig}
pipelineEvents={profileData.pipelineEvents}
sourceFiles={profileData.sourceFiles}
Expand Down
Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.