GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [connectors] Delta: report the follow phase after a resume past the snapshot by blp · Pull Request #7797 · feldera/feldera

Conversation

@blp

@blp blp commented Nov 3, 2026

Copy link
Copy Markdown
Member

Linked Issue

Closes #7747.

A snapshot_and_follow Delta connector that resumed from a checkpoint taken after its snapshot completed reported input_connector_delta_phase = 0 (loading_snapshot) while it followed the log.

Commit Change
Delta Set the phase to follow when the connector resumes past the snapshot (crates/connectors/src/integrated/delta_table/input.rs). The snapshot completion time and record count are not in the checkpoint, so their help text now says that they cover only the current run.
Iceberg No phase bug: follow_loop sets the phase on every path that follows. Its two snapshot metrics have the same limit as Delta, so their help text gets the same change.

docs.feldera.com/docs/operations/metrics.md is generated from /metrics output. I edited only the four changed rows, to the new help strings.

Describe Manual Test Plan

test_follow now waits for phase 1 after each resume. This covers the snapshot_and_follow_file_test_suspend tests in static-connectors, which run without transactions and with the snapshot and always transaction modes.

Run Result
Without the fix 3 snapshot_and_follow variants fail: timeout waiting for connector phase 1 (current: 0)
With the fix 5/5 follow_file_test_suspend tests pass, two runs

The Iceberg change is text only. I did not run the Iceberg follow tests, which need MinIO and a REST catalog.

Checklist

  • Unit tests added/updated
  • Integration tests added/updated
  • Documentation updated
  • Changelog updated

Breaking Changes?

Mark if you think the answer is yes for any of these components:

  • OpenAPI / REST HTTP API / feldera-types / manager (What is a breaking change?)
  • Feldera SQL (Syntax, Semantics)
  • feldera-sqllib (incl. dependencies fxp, etc.) (What is a breaking change?)
  • Python SDK (What is a breaking change?)
  • fda (CLI arguments)
  • Adapters (including configuration)
  • Storage Format / Checkpoints
  • Others (specify)

Describe Incompatible Changes

None.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
blp added 2 commits November 2, 2026 16:35
@blp
[connectors] Delta: report the follow phase after a resume past the s…
7315d02 
…napshot.

A snapshot_and_follow connector that resumed from a checkpoint taken
after its snapshot completed kept its initial phase, so
input_connector_delta_phase read 0 (loading_snapshot) while the
connector followed the log.

Set the phase to follow in that case. The snapshot completion time and
record count are not in the checkpoint, so document that they cover
only the current run.

Fixes #7747.

Signed-off-by: Ben Pfaff <[email protected]>
@blp
[iceberg] Say that the snapshot metrics cover only the current run.
212fd5d 
The snapshot completion time and record count are not in the
checkpoint. After a resume past the snapshot they read 0, but their
help text said that 0 means the snapshot is not complete. The phase
metric is correct in that case, because the follow loop sets it.

Refs #7747.

Signed-off-by: Ben Pfaff <[email protected]>
@mythical-fred-oss

mythical-fred-oss Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

LGTM.

Ran Result
cargo test -p static-connectors --test connector_suites follow_file_test_suspend 5/5 pass
Same test with the else branch in input.rs reverted snapshot_and_follow_file_test_suspend fails: timeout waiting for connector phase 1 (current: 0). The new assertion catches the regression.
pre-commit run on the changed files pass

The new wait polls with a 20 s timeout and has no fixed sleep, so it should not make the test flaky.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@blp
blp requested a review from anandbraman November 3, 2026 03:00
@blp blp self-assigned this Nov 3, 2026
@blp blp added ft Fault tolerant, distributed, and scale-out implementation connectors Issues related to the adapters/connectors crate QA Testing and quality assurance rust Pull requests that update Rust code enterprise Issue related to Feldera Enterprise features. labels Nov 3, 2026
anandbraman
anandbraman approved these changes Nov 5, 2026
View reviewed changes

@anandbraman anandbraman left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@blp
blp added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit edafe9e Nov 6, 2026
3 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@blp
blp deleted the delta-resume-phase branch November 6, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anandbraman anandbraman anandbraman approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@blp blp

Labels

connectors Issues related to the adapters/connectors crate enterprise Issue related to Feldera Enterprise features. ft Fault tolerant, distributed, and scale-out implementation QA Testing and quality assurance rust Pull requests that update Rust code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[connectors] Delta: snapshot metrics read "loading snapshot" after resuming from a checkpoint

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@blp @anandbraman
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.