GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — [types,connectors] Show resolved secrets in diagnostics as references by bestouff · Pull Request #7787 · feldera/feldera Conversation
|
|
What I ran
Gates: tests, docs, changelog, license ( Missing tests (medium):
Observation (low): the "JSON-escaped" form in the description is really the Rust Otherwise this looks solid as a second layer. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
96233ea to
291eb8b
Compare
46d747e to
c5bae41
Compare
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
What I ran (debug build)
The design looks sound: the panic hook's Missing (medium): uncovered cases.
Pitfall (low): a panic while |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
c5bae41 to
f248b57
Compare
f248b57 to
ad3e1ca
Compare
ad3e1ca to
ef43d4f
Compare
| N_PANICS.fetch_add(1, Ordering::Relaxed); | ||
| prev(info); | ||
| // A payload that copied a resolved secret is printed as its | ||
| // reference; any other payload keeps the default printer. |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
what is a payload "reference"?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It's a sort-of "path" to find the secret, e.g. ${secret:kubernetes:pg/uri}; I'll rewrite that.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| } | ||
|
|
||
| /// Rewrites a formatted log line before it is written. | ||
| pub trait LineRedactor: Send + Sync + 'static { |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I don't know if this belongs to json_logging.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Yes, in its own module it goes.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| secret_redactor().redact_bytes(line) | ||
| } | ||
|
|
||
| /// The registry's own walk: nothing to do while no secret is resolved. |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
je ne comprends rien
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
"Use the registry's JSON walk, not the default one of the trait: it does nothing when no secret is registered." should be better
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| | Output | What is shown instead of a resolved value | | ||
| |---|---| | ||
| | Pipeline log | the reference, `${secret:...}`, for the value as resolved; `[REDACTED ${secret:...}]` for a derived form of it, such as a percent-encoded or escaped copy or the password of a URL value | | ||
| | `deployment_error`, pipeline events, `/status` | the same two forms, from a pipeline on a runtime with this fix | |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
what are the "same two forms"? each row in this table should be self-contained. Someone may not read all rows.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
/status and /stats (no potential for confusion at all here). I'll write both.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| The following limits apply: | ||
|
|
||
| - The pipeline replaces values of 8 bytes or more, where they stand as a |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
the pipeline? the logging system perhaps.
replaces with what?
This is a rather crude heuristic.
Can't we do better?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It is the pipeline's log and status output, and the value is replaced by its reference.
Yes, it's crude but it's only the last layer. It runs after #7786 (removes the values from the error messages), #7801 (registers the fields that each connector declares as credentials), and the LibraryError PR that sanitizes the errors of the client libraries.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
... and once all of this lands I'll see for something more type-based as a followup
Sorry, something went wrong.
There was an error while loading. Please reload this page.
ef43d4f to
2da2fb3
Compare
A user who may reference a Kubernetes Secret need not be allowed to read its value. Register resolved secret values and their common derived forms in a process-wide Aho-Corasick registry as a second layer behind the source fixes. Values shorter than 8 trimmed bytes and environment references are excluded to avoid rewriting unrelated diagnostics. Apply redaction to both ErrorResponse types, connector error history, panic payloads, and text logs. Redact JSON fields before serialization adds another escaping layer. Include object keys and escaped URL password components without changing operational configuration. Test the real JSON formatter with quoted and backslash-containing secrets. Pipeline tests cover a failed referenced Postgres connection and a format error with a short referenced secret. The environment reference test configures a child process instead of mutating the environment of the parallel test process. Related: feldera/cloud#2136 Signed-off-by: Xavier Bestel <[email protected]>
A panic during registration can run while the registry write lock is held. Use a nonblocking read for panic payloads and omit the payload when the registry is unavailable, preserving the location and backtrace. Exercise concurrent registration and replace the ignored panic-hook test with an isolated subprocess that checks both counting and redaction. Removing the nonblocking read reproduces the deadlock; disconnecting the hook filter fails the subprocess regression. Signed-off-by: Xavier Bestel <[email protected]>
2da2fb3 to
ba2fbee
Compare
At least 1 approving review is required to merge this pull request.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.