GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — dstack-ingress 2.0: nginx → haproxy L4 TCP proxy by h4x3rotab · Pull Request #88 · Dstack-TEE/dstack-examples

Conversation

@h4x3rotab

@h4x3rotab h4x3rotab commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace nginx L7 reverse proxy with HAProxy in pure TCP mode (mode tcp) for TLS termination + transparent L4 forwarding
  • All protocols (HTTP/1.1, HTTP/2, WebSocket, gRPC) work transparently without proxy-layer configuration
  • Multi-domain SNI-based routing via ROUTING_MAP env var replaces nginx config injection
  • Evidence endpoint preserved at /evidences/ via HAProxy payload inspection (opt-out with EVIDENCE_SERVER=false), plus shared volume for backend serving
  • Configurable L4 tuning: MAXCONN, TIMEOUT_CONNECT/CLIENT/SERVER, ALPN
  • E2e test verified on live Phala CVM: HTTP/1.1, HTTP/2, gRPC, TLS 1.3, evidence serving, SNI routing

Key changes

File Change
Dockerfile nginx → haproxy:3.1-bookworm, master-worker mode
entrypoint.sh HAProxy config generation, combined PEM building, evidence server
renewal-daemon.sh nginx -s reload → kill -USR2 (haproxy graceful reload)
build-combined-pems.sh New: concatenate fullchain+privkey for HAProxy PEM format
functions.sh sanitize_target_endpoint accepts bare host:port
docker-compose.multi.yaml ROUTING_MAP env var replaces nginx config injection
e2e-test.sh New: deploys to Phala CVM, tests all protocol paths

Test plan

  • Unit tests pass (test_sanitizers.sh — bare host:port, IP:port)
  • E2e on Phala CVM: HTTP/1.1 through TCP proxy (200)
  • E2e: HTTP/2 via ALPN negotiation to h2c backend
  • E2e: gRPC reflection + unary call through TCP proxy
  • E2e: TLS 1.3, Let's Encrypt staging certificate
  • E2e: Evidence endpoint /evidences/ via payload inspection
  • E2e: Evidence file integrity (sha256sum chain)
  • E2e: SNI routing separates HTTP and gRPC backends
  • E2e: CVM auto-cleanup on exit

🤖 Generated with Claude Code

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@h4x3rotab h4x3rotab mentioned this pull request Mar 26, 2026
dstack-ingress: add WebSocket proxy support #87
Open
h4x3rotab and others added 6 commits March 26, 2026 21:56
@h4x3rotab @claude
dstack-ingress 2.0: switch from nginx to haproxy for L4 TCP proxying
c9a6403
Replace nginx L7 reverse proxy with HAProxy in TCP mode. This eliminates
HTTP-level inspection so all protocols (HTTP, WebSocket, gRPC, arbitrary
TCP) work transparently after TLS termination.

Key changes:
- Dockerfile: nginx base -> haproxy:3.1-bookworm, master-worker mode
- entrypoint.sh: generate haproxy.cfg instead of nginx conf, support
  ROUTING_MAP env var for multi-domain SNI-based routing
- renewal-daemon.sh: graceful reload via SIGUSR2 instead of nginx -s reload
- New build-combined-pems.sh: concat fullchain+privkey for HAProxy PEM format
- functions.sh: sanitize_target_endpoint accepts bare host:port
- Evidence serving: payload inspection on port 443 (default, opt-out via
  EVIDENCE_SERVER=false) + shared /evidences volume for backend serving
- Configurable L4 tuning: MAXCONN, TIMEOUT_CONNECT/CLIENT/SERVER
- Drop L7-only env vars (CLIENT_MAX_BODY_SIZE, PROXY_*_TIMEOUT) with warnings

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@h4x3rotab @claude
Add e2e test script for dstack-ingress on Phala CVM
669268a
Deploys dstack-ingress + nginx backend to a Phala CVM via phala CLI,
verifies TLS termination, TCP pass-through, evidence serving, and
HTTP/2 ALPN negotiation, then cleans up automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@h4x3rotab @claude
Fix e2e test failures from live CVM testing
4630ec0
Fixes discovered during Phala CVM deployment:
- Remove ALPN advertisement from haproxy bind (pure L4 proxy shouldn't
  promise application-layer protocols; caused HTTP/2 mismatch with
  HTTP/1.1 backends). ALPN now configurable via env var.
- Add http-request replace-path in evidence backend to strip /evidences
  prefix (Python http.server saw /evidences/evidences/)
- Fix e2e test: use --resolve for DNS, --http1.1 for curl, robust TLS
  version detection, grep IPs from dig output, echo y for delete prompt
- Update pinned-packages.txt for haproxy base image

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@h4x3rotab @claude
Enhance e2e test with HTTP/2 and gRPC protocol coverage
838ced6
- Add multi-domain test: whoami (HTTP/1.1) + grpcbin (gRPC/H2) backends
- Use ROUTING_MAP with SNI-based routing to test multi-backend support
- Add comma-separated ROUTING_MAP parsing (phala CLI -e flattens newlines)
- Test HTTP/2 against h2c-capable backend (grpcbin), not HTTP/1.1-only whoami
- Verify gRPC reflection, unary call, and SNI routing separation
- All 17 tests pass on live Phala CVM deployment

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@h4x3rotab @claude
Harden haproxy config generation and input validation
a44963b
- Factor common global/defaults/frontend config into emit_haproxy_preamble()
  and emit_evidence_backend() to eliminate duplication between single-domain
  and multi-domain setup functions
- Add sanitize_positive_integer() and sanitize_haproxy_timeout() validators;
  validate MAXCONN, TIMEOUT_CONNECT/CLIENT/SERVER, EVIDENCE_PORT at startup
- Fix renewal-daemon to log why HAProxy reload failed (missing PID file vs
  failed signal) instead of silently swallowing errors
- Move loop-scoped local declarations before the while loop in
  setup_haproxy_cfg_multi()
- Guard e2e test against wildcard DOMAIN input
- Wrap phala deploy in error check so test exits on deployment failure

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@h4x3rotab @claude
Harden build reproducibility and input sanitization
5335ea2
- Pin haproxy base image by digest for reproducible builds
- Add USER root comment explaining why (haproxy defaults to uid 99)
- Add sanitize_alpn() validator; ALPN is now validated at startup
- Require time suffix on haproxy timeouts (bare numbers are milliseconds
  in HAProxy which is almost never what users intend)
- Remove `local` usage outside function scope in renewal-daemon.sh
- Document ALPN env var in README

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@h4x3rotab
h4x3rotab force-pushed the vk/d9b5-dstack-ingress-2 branch from 62b91d3 to 5335ea2 Compare March 26, 2026 21:56
@h4x3rotab
h4x3rotab requested a review from kvinwang March 26, 2026 22:03
kvinwang
kvinwang reviewed Mar 27, 2026
View reviewed changes
Comment thread custom-domain/dstack-ingress/docker-compose.yaml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread custom-domain/dstack-ingress/scripts/entrypoint.sh Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread custom-domain/dstack-ingress/scripts/entrypoint.sh Resolved

Uh oh!

There was an error while loading. Please reload this page.

@h4x3rotab @claude
Harden build reproducibility and input sanitization
c1a57c3
Address PR review feedback:
- Add TODO comments for image digest pinning in compose files
- Use literal domains in multi-domain compose example (avoid env var injection)
- Sanitize ROUTING_MAP domain/target values against haproxy config injection
- Replace single-threaded http.server with ThreadingTCPServer for evidence serving

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@kvinwang kvinwang mentioned this pull request Mar 27, 2026
dstack-ingress: new optional SAN field on certs #86
Open
@h4x3rotab @claude
Replace inline Python http.server with mini-httpd
8ed590f
The evidence file server was using an inline Python ThreadingTCPServer,
which is hacky and not battle-tested for production use. Replace it with
mini-httpd (1.30), a proven static file server by Jef Poskanzer (author
of thttpd). At 123 KB installed, it is the smallest proper HTTP server
in Debian bookworm that handles concurrent connections (fork-per-request).

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@kvinwang
kvinwang merged commit 0c9f5c6 into main Mar 27, 2026
6 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@wwwehr wwwehr mentioned this pull request Apr 15, 2026
DNS based (route53) load balancing and dstack-ingress upstream server support #83
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kvinwang kvinwang kvinwang left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@h4x3rotab @kvinwang
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.