GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Build the Enterprise edition on the validated AWS-LC FIPS 3.1.0 module by gz · Pull Request #7915 · feldera/feldera

Conversation

@gz

@gz gz commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Enterprise builds link the AWS-LC FIPS 3.1.0 module for TLS, API keys and checksums, and a new --fips flag makes the pipeline manager and pipelines refuse to start when TLS is not in FIPS mode. The docs gain a FIPS 140-3 page; open source builds are unchanged.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

lalithsuresh reacted with heart emoji
All reactions
  • ❤️ 1 reaction
mythical-fred-oss[bot]
mythical-fred-oss Bot reviewed Nov 10, 2026
View reviewed changes
Comment thread crates/feldera-mtls/src/lib.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.

mythical-fred-oss[bot]
mythical-fred-oss Bot reviewed Nov 10, 2026
View reviewed changes
Comment thread crates/pipeline-manager/Cargo.toml Resolved

Uh oh!

There was an error while loading. Please reload this page.

mythical-fred-oss[bot]
mythical-fred-oss Bot reviewed Nov 10, 2026
View reviewed changes
Comment thread Cargo.toml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@mythical-fred-oss

mythical-fred-oss Bot commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Review: FIPS build and --fips flag

What I ran Result
cargo test -p pipeline-manager (api key, checksum, patch-table tests, workspace_preparation after ./build.sh) ✅ pass
same tests with --features feldera-enterprise (builds aws-lc-fips-sys) ✅ pass
cargo check + adhoc executor tests in dbsp_connectors (hash_digest_is_pinned confirms the hash output is unchanged) ✅ pass
ad-hoc #[should_panic] test of assert_fips_crypto_provider on a non-FIPS build ✅ panics as intended
pre-commit run --files <changed> ✅ pass
  • missing (medium), Hard Rule 2: the user-visible behaviour this PR adds, --fips refusing to start, has no test anywhere: not the gate itself, not the HTTPS/exchange-TLS asserts, and not LocalRunner passing --fips to the pipeline (see inline).
  • missing (medium): with_patch_tables is only tested on the platform/gen2 path. The runtime_version branch, which reads runtime_sources/Cargo.toml, has no test, and neither does the error path for a malformed manifest.
  • pitfall (medium): the pipeline-compile step now depends on git sources (feldera forks) through [patch]. Air-gapped or offline compiler servers need those git checkouts cached in CARGO_HOME, or pipeline builds fail. Please confirm that the image pre-fetches them.
  • docs (low-medium): the FIPS page says the runner rejects non-FIPS connectors and logs FIPS mode: pipelines are allowed to use only these connectors. Nothing in this repository implements that, and the local runner does not enforce it with --fips. Please say that this applies only to Kubernetes deployments.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gz added 8 commits November 10, 2026 09:19
@gz
[pipeline-manager] Draw API keys from the AWS-LC random generator.
a7d15a1
API keys are secrets, so FIPS builds must take them from the validated module's DRBG rather than from rand's ChaCha generator. The key format stays: the apikey: prefix and 128 alphanumeric characters, each drawn by rejection sampling so all are equally likely.

rand is now test-only, and static_assertions had no other user.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[pipeline-manager] Checksum programs and binaries with AWS-LC.
d30107c
The integrity checksums of pipeline binaries and program info came from RustCrypto's sha2. They now come from the aws-lc-rs digest the rest of the manager uses, so FIPS builds verify binaries with the validated module. The output is unchanged.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[connectors] Hash with AWS-LC and drop sha2 from the workspace.
b874544
The ad hoc result hash and the Kafka PEM file name were the last direct users of sha2. Both now use aws-lc-rs with the same hex output, and the workspace no longer declares sha2.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[build] Build on the validated AWS-LC FIPS 3.1.0 module.
f915bca
aws-lc-rs 1.18 pulls AWS-LC 4, which is still in comment resolution at the CMVP. aws-lc-rs 1.17.1 ships aws-lc-fips-sys 0.13.11, the crate that bundles AWS-LC FIPS 3.1.0 (certificates #5314 static, #5298 dynamic), so the lock pins both.

rustls 0.23.46, rustls-webpki 0.103.16 and async-nats 0.51.0 require aws-lc-rs 1.18 only for ML-DSA, which 1.17.1 exposes only as unstable. The patch section takes them from feldera's forks, which keep every upstream fix and gate ML-DSA behind a feature that stays off. rustls-webpki moves from 0.103.15 to 0.103.16, which rejects certificate paths through the distrusted emSign intermediates (RUSTSEC-2026-0382).

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[build] Build the Enterprise edition on the AWS-LC FIPS module.
a50d4bb
The feldera-enterprise feature of pipeline-manager and of the connectors turns on fips, so an Enterprise pipeline-manager and the pipelines it builds link the validated module without a separate switch. Every generated workspace carries the dependency overrides of this workspace so the copied Cargo.lock resolves the same sources. install-librdkafka.sh builds AWS-LC-FIPS-3.1.0 into librdkafka when AWS_LC_FIPS=1, which deploy/Dockerfile exposes as a build argument, and it learns the AWS-LC release that aws-lc-sys 0.42.0 vendors. The open source builds and images stay on the non-FIPS AWS-LC.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[pipeline-manager] Pass AWS_LC_FIPS_SYS_STATIC through to pipeline bu…
d71220d
…ilds.

The cargo environment of a pipeline build is cleared, so the variable that makes aws-lc-fips-sys produce the FIPS module as a shared object never reached it and pipelines linked the module statically. The compiler server sets it for the module the pipeline executor ships.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[fips] Add --fips to refuse to start when TLS is not in FIPS mode.
55b35e9
The API server, compiler server and runner take --fips through CommonConfig, and the pipeline takes it through ServerArgs; the local runner passes it on. With the flag a process panics unless its rustls provider and each TLS config it builds are in FIPS mode. Without the flag a build on the FIPS module runs on it all the same and checks nothing.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
[docs] Add a FIPS 140-3 page for the Enterprise edition.
ba76508
The page lists the validated cryptographic modules and their certificates, the fips.enabled Helm setting, the connectors that FIPS mode allows, and what the operator must review.

Signed-off-by: Gerd Zellweger <[email protected]>
@gz
gz force-pushed the v0.382.0-fips branch from 3bf32da to ba76508 Compare November 10, 2026 17:21
@gz
gz requested review from anandbraman and mihaibudiu November 10, 2026 17:29
mihaibudiu
mihaibudiu approved these changes Nov 10, 2026
View reviewed changes

@mihaibudiu mihaibudiu left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approved, but I have lots of questions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread crates/pipeline-manager/src/compiler/rust_compiler.rs
let feldera_manifest = read_file_content(&feldera_manifest_path).await?;
let workspace_manifest = read_file_content(&cargo_toml_target_path).await?;
let patched = with_patch_tables(&workspace_manifest, &feldera_manifest).map_err(|e| {
UtilError::InvalidContent(format!(

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this leak secrets?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| `rustls-webpki` | 0.103.16 | Certificate verification. A [Feldera fork](https://github.com/feldera/webpki/tree/fips-aws-lc-3) of this release, for the same reason |
| `rclone` | 1.76.0 | Built with Go 1.27.3 and `GOFIPS140=v1.0.0`, which makes FIPS mode (`GODEBUG=fips140=on`) its default |

The forks exist to keep the security fixes of the current releases: the last upstream

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we tracking whether these will change?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| With `fips.enabled: true` | Effect |
|---------------------------|--------|
| The chart requires `httpsSecretRef` | Feldera components exchange no data in plaintext |
| The API server, the compiler server, the Kubernetes runner, the pipelines and their coordinators start with `--fips` | Each one exits unless its TLS runs on the module |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"runs on the module" - uses the FIPS module

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
|---------------------------|--------|
| The chart requires `httpsSecretRef` | Feldera components exchange no data in plaintext |
| The API server, the compiler server, the Kubernetes runner, the pipelines and their coordinators start with `--fips` | Each one exits unless its TLS runs on the module |
| The chart refuses the `runtime_version` unstable feature | A pipeline on another runtime version builds without the module, and `cargo` downloads its crates with another TLS library |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does "on another runtime version" really mean here? a pipeline compiled a long time ago?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| The chart requires `httpsSecretRef` | Feldera components exchange no data in plaintext |
| The API server, the compiler server, the Kubernetes runner, the pipelines and their coordinators start with `--fips` | Each one exits unless its TLS runs on the module |
| The chart refuses the `runtime_version` unstable feature | A pipeline on another runtime version builds without the module, and `cargo` downloads its crates with another TLS library |
| The chart refuses `parallelCompilation.sccache.enabled` | `sccache` transfers build artifacts with its own TLS library |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do users care about this?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md

The chart cannot check the items below. Review them for your deployment.

| Item | Detail | What to do |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this table say?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| Item | Detail | What to do |
|------|--------|------------|
| Control-plane database | The connection to PostgreSQL uses TLS on the module when [TLS is configured](helm-chart-reference.md#external-postgresql-tls), note that the password proof (SCRAM-SHA-256) is computed outside the module | Configure TLS for the database connection |
| Checkpoint synchronization | `rclone` in FIPS mode runs TLS and request signing on the Go module. It still computes MD5 checksums, which S3 and Google Cloud Storage use to detect corrupted transfers, not for security | None |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"None" is not a reasonable value for "what to do"

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| Rust UDFs | Crates that a UDF adds may bring their own cryptography | Set `pipeline.allowRustUdfs: false`, or review the crates |
| Compiler server tools | The image holds the crates of the precompiled dependencies. `cargo` and `git` download the crates that a Rust UDF adds, with their own TLS libraries. They do not handle pipeline data | Set `pipeline.allowRustUdfs: false`, or review the crates |
| Clients | Browsers, the `fda` CLI, the Python SDK and ingress controllers use their own TLS libraries | Use FIPS validated clients and ingress |
| Init containers and sidecars | Containers that you add to pipeline pods are outside Feldera | Set `pipeline.allowInitContainers: false`, or review the containers |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does "review the containers" really mean? how does reviewing fix problems?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| Compiler server tools | The image holds the crates of the precompiled dependencies. `cargo` and `git` download the crates that a Rust UDF adds, with their own TLS libraries. They do not handle pipeline data | Set `pipeline.allowRustUdfs: false`, or review the crates |
| Clients | Browsers, the `fda` CLI, the Python SDK and ingress controllers use their own TLS libraries | Use FIPS validated clients and ingress |
| Init containers and sidecars | Containers that you add to pipeline pods are outside Feldera | Set `pipeline.allowInitContainers: false`, or review the containers |
| SQL functions | `MD5` and other hash functions in SQL programs run outside the module to process data | Do not use them to protect sensitive data |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have two kinds of SQL functions; the ones implemented in SQL should not be a problem

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs.feldera.com/docs/get-started/enterprise/fips.md
| Init containers and sidecars | Containers that you add to pipeline pods are outside Feldera | Set `pipeline.allowInitContainers: false`, or review the containers |
| SQL functions | `MD5` and other hash functions in SQL programs run outside the module to process data | Do not use them to protect sensitive data |
| Debugging tools | The security policies of the modules forbid `ptrace`, `gdb`, `strace` and similar tools on a running module. The chart refuses `pipeline.allowProfiling`, but it cannot see tools that you attach yourself | Do not attach debuggers to Feldera containers |
| Advisories against AWS-LC FIPS 3.1.0 | Scanners report CVE-2026-3337 (timing of AES-CCM tag verification) and CVE-2026-4428 (CRL distribution point check) for `aws-lc-fips-sys` 0.13.11. Later AWS-LC versions fix both but have no certificate. Feldera uses neither code path: its TLS offers no AES-CCM cipher suite, and the X.509 code of AWS-LC, which holds the CRL check, does not run. `rustls-webpki` parses certificates and builds the chain, and verifies every signature with the module | None |

@mihaibudiu mihaibudiu Nov 10, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is awfully specific

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mythical-fred-oss mythical-fred-oss[bot] mythical-fred-oss[bot] left review comments
@mihaibudiu mihaibudiu mihaibudiu approved these changes
@anandbraman anandbraman Awaiting requested review from anandbraman

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@gz @mihaibudiu
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.