GeezerVison

pitfall (medium): Further down, the mTLS Requirements say "The components read a renewed certificate without a restart." That line is about the chart-issued control-plane client certificate, but next to this box readers will take it as a contradiction. Please say in one of the two places which certificate each statement covers.

Both places now say which certificate they mean: the server certificate needs a restart, and the control plane client certificate from the chart doesn't.

pitfall (medium): With NodeLocal DNSCache (common on GKE, and optional on EKS), pods resolve names through 169.254.20.10 on the node, not through the kube-dns pods. This rule doesn't match that traffic, and the ipBlock below excludes 169.254.0.0/16, so the pipeline can't resolve any names. Please add a bullet below that says to allow the node-local DNS address on port 53 in that case.

Added a bullet for it. The address is usually 169.254.20.10, port 53, UDP and TCP.

missing (medium): Connectors aren't the only private-network dependency. Pipeline storage, checkpoint sync to in-cluster or VPC object storage (MinIO, S3 interface endpoints), and an in-cluster OTel collector are also blocked. Please widen this bullet to "systems that pipelines use" and list these.

Widened it to systems that pipelines use, and added object storage for checkpoint sync (MinIO, VPC endpoints). I left the OTel collector out. Pipelines don't export OTLP, and in our OTel setup the collector sends data to the API server, not to the pipeline.

missing (medium): This only checks that allowed traffic works. If the CNI doesn't enforce policies (see the note at the top), every check here still passes, so the user believes the pipelines are isolated when they aren't. Please add a negative check, for example from a pipeline pod:

kubectl exec -n feldera <pipeline-pod> -- timeout 3 bash -c '</dev/tcp/<release>-db/5431' && echo "NOT isolated"

This assumes the image has bash. Otherwise, use any TCP probe.

Added it, but against the API server instead of the database. On k3d the db check said "isolated" even with the pipeline policy removed, because the db policy blocks that traffic on its own, so it doesn't test the pipeline policy. The API server check says "isolated" with the policy and "not isolated" without it. The pipeline image has bash and timeout.

pitfall (medium): This rule assumes the pod, service, and node CIDRs are all RFC 1918. Two cases where that does not hold:

• On clusters with non-RFC 1918 pod ranges (for example GKE privately used public IPs), 0.0.0.0/0 lets pipelines reach every pod, including Postgres and the API server.
• A public Kubernetes API endpoint (EKS or GKE public access) is reachable through this rule.

Please add a bullet under "Change the example" saying to add the cluster pod and service CIDRs to except when they are not private.

pitfall (medium): || echo "isolated" reports success on any failure: a wrong pod name, kubectl exec denied by RBAC, no bash/timeout in the image, or a DNS failure. A misconfigured check then looks like isolation. Suggest adding a positive control first, the same command against <release>-compiler-server:8085, which must succeed, so isolated really means the policy blocked the connection.

we can drop the section that describe how to create them with OpenSSL

Dropped.

can we validate what happens with cert expiration in a cluster (bring up, make sure cert expires) and give concrete steps on what needs to be done

tested it on k3d. I gave the server certificate a 1-hour lifetime and let it expire without restarting anything:

• external clients get certificate has expired
• pipelines that are already running keep running
• a pipeline that starts can't download its binary and goes to Unavailable

A kubectl rollout restart of the API server, runner and compiler fixes it, then stop and start any stuck pipeline. These steps are now in "Renew the server certificate". Also, cluster_healthz stayed green the whole time, so it doesn't catch this.

can drop this

Dropped.

Applied.

clarify that this is in case someone uses postgres pod from ourhelm chart which is uncommon

can probably move this down a bit

Moved it under the example and said it only applies to the Postgres pod from the chart.

drop this

applied.

# Connectors: the public internet, without private, shared, loopback,
# link-local (instance metadata), multicast, and reserved addresses.

I think this is a bad defualt here (allow public internet access) for most users.
You probably want to not add anything here as a default and refer to the section below to customize what the pipeline can reach

Removed for pipelines. The API server and the runner still allow public addresses on port 443, because they call the license server (cloud1.feldera.com) and NetworkPolicies can't match a host name. This explains and says to allow only the proxy if they use one.

I would add a recommendation that they configure it e.g., VPC CNI in strict mode

the examples should be structured for default deny

whta about the control plane pods? should we cover them too?

lets do the most restrictive variant for control and data plane in terms of example policy

yes, every component has its own policy now.