GeezerVison is showing github.com — [connectors] Register schema-declared credentials before initialization by bestouff · Pull Request #7801 · feldera/feldera
Conversation
bestouff
commented
Nov 3, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Part of feldera/cloud#2136. Stacked on #7788 (cloud-2136-manager).
Inline credentials in known connector fields can appear in dependency diagnostics. This registers schema-declared credentials before transport validation and raw Avro format parsing, so production logs and error responses replace matching values with [REDACTED].
- Connector modules declare canonical credential paths and supported aliases; transport dispatch is exhaustive. Operational values and safe
Debug projections remain unchanged.
- Every Secret reference remains registered, including references in public or unknown fields. References take precedence over generic credential markers in either initialization order.
- The eight-byte floor and token boundaries remain. Short values stay protected at recognized sources; unknown inline fields and further transformations remain outside the registry guarantee. SQL/configuration in bundles stays unchanged.
Validation: 298 type tests, 74 adapter tests, 4 observability tests, 10 transport/error/controller/panic tests, 3 engine regressions, and 47 manager tests pass. Clippy passes with warnings denied across all five affected crates. Tests exercise real transport constructors, text/JSON logging, error responses, malformed format parsing, aliases, provenance, collisions, and short values. Negative validation passes: disconnecting visitation fails both transport and engine tests; skipping reference registration fails the public/free-form field integration test; disabling provenance precedence fails its unit test. Enterprise/Fremont validation passed; results are recorded in feldera/cloud#2160.
Coverage includes Kafka JWT assertion PEM/passphrase options from bundled librdkafka 2.12.1, token-only URL userinfo, and the controller-to-transport path with short credentials and a reference in a public field. The new declaration tests fail against the old visitors.
The final pushed tree passes all 436 targeted tests, formatting, and Clippy with warnings denied.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
What I ran
| Check |
Result |
cargo test -p feldera-types --lib -- sensitive secret_redaction secret_resolver |
31 passed |
cargo test -p dbsp_connectors --lib -- transport_registration_precedes_diagnostics short_rejected_value_stays_out |
2 passed |
cargo test -p static-connectors -- format_credentials (engine test) |
passed |
pre-commit run --files <changed files> (clippy, fmt, machete, OpenAPI, docs) |
passed |
QA: Kafka JWT-bearer assertion key options (.pem, .passphrase) |
failed, 0 values visited (see inline comment) |
| QA: a boundary-rejected match followed by a valid match, including multibyte text |
passed |
Gates: tests, docs and changelog are included. No unsafe code, no new dependencies, no CI changes. The commit message is fine.
Missing (medium):
- No test for a token-only userinfo URL (
https://<token>@host); see inline comment.
- No test for the two Kafka assertion keys; see inline comment.
- Nothing covers a config whose credential values are shorter than 8 bytes, or whose only Secret reference sits in a non-credential field, going through
resolve_connector_config → input_transport_config_to_endpoint. That path registers the config twice, which is idempotent by design, but no test pins it.
The rest looks solid. Fix the two inline items and this is good to go.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
What I ran (debug build):
| Command |
Result |
cargo test --locked -p feldera-types -- sensitive secret_redaction |
24 passed |
cargo test --locked -p dbsp_connectors --features with-kafka --lib -- credential_registration transport_registration short_rejected |
3 passed |
| Scratch QA tests on Postgres and Redis registration (not committed) |
2 gaps confirmed, see the inline comments |
Gates: tests are present, there is no unsafe code and no new dependencies, and docs and the changelog are updated. I saw no flaky patterns.
Missing tests (medium):
- A Postgres libpq keyword URI where the diagnostic quotes only the password. This currently leaks.
- A Postgres or Redis URI without credentials. It currently comes out as
[REDACTED], which makes connection errors non-actionable.
redact() now scans every registered value linearly whenever a match fails the token-boundary check. No test or benchmark covers a line with many near-miss matches against a large registry.
Pitfall (medium): the docs table says "Connection strings" are covered. Once the keyword-string fix lands, the docs should say that only the password part of a connection string is masked.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Visit known transport and raw Avro credential fields before validation or
format parsing. Keep every Secret reference registered, including values
in public and unknown fields, and give references precedence over generic
credential markers in either initialization order.
Preserve operational configuration and allowlisted Debug output. Cover
aliases, short-value source sanitization, overlapping registrations, JSON
key collisions, real transport construction, logging, error responses,
and the engine format-failure path. Document the matching and bundle
limits.
Signed-off-by: Xavier Bestel <[email protected]>
This branch has not been deployed
No deployments
Reviewers
mihaibudiu
Awaiting requested review from mihaibudiu
At least 1 approving review is required to merge this pull request.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|