@@ -48,7 +48,9 @@ The API is currently versioned under `/api/v1`. |
48 | 48 | - Redis-backed rate limiting. |
49 | 49 | - Request ID, structured logging, audit logging, and security headers. |
50 | 50 | - Request size limiting and idempotency support. |
51 | | -- Health, liveness, and readiness endpoints. |
| 51 | +- Health, liveness, and readiness endpoints with structured RFC-style responses. |
| 52 | +- Prometheus metrics at `/metrics`. |
| 53 | +- OpenTelemetry distributed tracing (FastAPI, SQLAlchemy, Redis instrumentations). |
52 | 54 | - API route grouping under `/api/v1`. |
53 | 55 | - Async SQLAlchemy persistence. |
54 | 56 | - Alembic database migrations. |
@@ -70,6 +72,8 @@ The API is currently versioned under `/api/v1`. |
70 | 72 | - Passlib for password hashing |
71 | 73 | - Casbin for authorization policies |
72 | 74 | - Redis and fastapi-limiter for rate limiting and token revocation |
| 75 | +- Prometheus for metrics collection |
| 76 | +- OpenTelemetry for distributed tracing |
73 | 77 | - Pytest |
74 | 78 | - Ruff |
75 | 79 | - Poetry |
@@ -94,6 +98,7 @@ The API is currently versioned under `/api/v1`. |
94 | 98 | │ │ ├── schemas/ # Shared response schemas |
95 | 99 | │ │ ├── security/ # JWT, password, revocation, audit helpers |
96 | 100 | │ │ ├── seed/ # Database seeding orchestration |
| 101 | +│ │ ├── telemetry/ # Prometheus metrics and OpenTelemetry tracing setup |
97 | 102 | │ │ └── lifespan.py # FastAPI lifespan hook |
98 | 103 | │ ├── modules/ |
99 | 104 | │ │ ├── authorization/ |
@@ -123,7 +128,10 @@ The API is currently versioned under `/api/v1`. |
123 | 128 | ├── poetry.lock # Poetry lock file |
124 | 129 | ├── alembic.ini # Alembic configuration |
125 | 130 | ├── Dockerfile # Docker image definition |
126 | | -└── docker-compose.yml # Local API, PostgreSQL, and Redis services |
| 131 | +├── docker-compose.yml # Local API, PostgreSQL, Redis, Prometheus, and OTEL collector services |
| 132 | +└── docker/ |
| 133 | + ├── prometheus/prometheus.yml # Prometheus scrape configuration |
| 134 | + └── otel-collector/config.yaml # OpenTelemetry collector configuration |
127 | 135 | ``` |
128 | 136 |
|
129 | 137 | ## Architecture |
@@ -235,13 +243,15 @@ DELETE /api/v1/permissions/{permission_id} |
235 | 243 | GET /health |
236 | 244 | GET /live |
237 | 245 | GET /ready |
| 246 | +GET /metrics |
238 | 247 | ``` |
239 | 248 |
|
240 | 249 | Public routes: |
241 | 250 |
|
242 | 251 | - `/health` |
243 | 252 | - `/live` |
244 | 253 | - `/ready` |
| 254 | +- `/metrics` |
245 | 255 | - `/docs` |
246 | 256 | - `/redoc` |
247 | 257 | - `/openapi.json` |
@@ -307,6 +317,10 @@ ACCOUNT_LOCKOUT_MAX_ATTEMPTS=5 |
307 | 317 | ACCOUNT_LOCKOUT_WINDOW_MINUTES=15 |
308 | 318 | ACCOUNT_LOCKOUT_DURATION_MINUTES=15 |
309 | 319 | LOG_FORMAT=json |
| 320 | +OTEL_ENABLED=false |
| 321 | +OTEL_SERVICE_NAME=fastapi-modulith |
| 322 | +OTEL_EXPORTER_OTLP_ENDPOINT= |
| 323 | +OTEL_EXPORTER_OTLP_HEADERS= |
310 | 324 | EMAIL_PROVIDER=ses |
311 | 325 | AWS_REGION=us-east-1 |
312 | 326 | AWS_ACCESS_KEY_ID= |
@@ -339,6 +353,10 @@ REDIS_URL=redis://127.0.0.1:6379/0 |
339 | 353 |
|
340 | 354 | Production mode requires a non-default `SECRET_KEY`, non-empty database and Redis URLs, JWT issuer and audience values, positive token lifetimes, and explicit CORS origins. |
341 | 355 |
|
| 356 | +OpenTelemetry tracing is off by default. To enable it, set `OTEL_ENABLED=true` and point `OTEL_EXPORTER_OTLP_ENDPOINT` at an OTLP HTTP collector endpoint. |
| 357 | + |
| 358 | +Metrics from Prometheus are always available at `/metrics` and require no additional setup. |
| 359 | + |
342 | 360 | ## Local Setup |
343 | 361 |
|
344 | 362 | The Makefile expects Poetry to create `.venv` inside the repository. Configure that once, then install dependencies: |
@@ -389,6 +407,12 @@ Operational checks: |
389 | 407 | ```text |
390 | 408 | http://localhost:8000/live |
391 | 409 | http://localhost:8000/ready |
| 410 | +
|
| 411 | +Metrics: |
| 412 | +
|
| 413 | +```text |
| 414 | +http://localhost:8000/metrics |
| 415 | +``` |
392 | 416 | ``` |
393 | 417 |
|
394 | 418 | ## Database and Migrations |
@@ -536,14 +560,26 @@ make clean |
536 | 560 |
|
537 | 561 | Before starting Docker Compose, set non-empty `POSTGRES_PASSWORD`, `REDIS_PASSWORD`, and `SECRET_KEY` in `.env`. Compose intentionally fails fast when database or Redis passwords are missing. |
538 | 562 |
|
539 | | -Run API, PostgreSQL, and Redis services: |
| 563 | +Run all services (API, PostgreSQL, Redis, Prometheus, and OpenTelemetry Collector): |
540 | 564 |
|
541 | 565 | ```bash |
542 | 566 | make db-up |
543 | 567 | ``` |
544 | 568 |
|
545 | 569 | The API container applies Alembic migrations before starting Uvicorn. Database seeding remains an explicit `make seed` step. |
546 | 570 |
|
| 571 | +### Docker Services |
| 572 | + |
| 573 | +| Service | Image | Port | Purpose | |
| 574 | +| --- | --- | --- | --- | |
| 575 | +| `api` | `fastapi-modulith:local` | 8000 | FastAPI application | |
| 576 | +| `db` | `postgres:17-alpine` | — | PostgreSQL database | |
| 577 | +| `redis` | `redis:8-alpine` | — | Redis for rate limiting and token revocation | |
| 578 | +| `prometheus` | `prom/prometheus:latest` | 9090 | Metrics scraping and storage | |
| 579 | +| `otel-collector` | `otel/opentelemetry-collector-contrib:latest` | 4318 | OTLP trace ingestion and export | |
| 580 | + |
| 581 | +Tracing is automatically enabled in Docker Compose — the API sends spans to the `otel-collector` service via OTLP HTTP. Prometheus scrapes `/metrics` from the API every 15 seconds. |
| 582 | + |
547 | 583 | Stop services: |
548 | 584 |
|
549 | 585 | ```bash |
@@ -663,6 +699,8 @@ Legend: `Implemented` means code exists in the repository. `Partial` means code |
663 | 699 | | OpenAPI Authentication | Required | Implemented | Swagger OAuth2 auth is configured, and docs/OpenAPI endpoints are disabled when `APP_ENV=production`. | |
664 | 700 | | Health Check Endpoint | Required | Implemented | `/health` endpoint returns service health. | |
665 | 701 | | Readiness/Liveness Endpoints | Required | Implemented | Adds `/live` and `/ready` operational endpoints. | |
| 702 | +| Prometheus Metrics | Recommended | Implemented | Request count, latency histogram, active requests at `/metrics`. | |
| 703 | +| Distributed Tracing (OpenTelemetry) | Recommended | Implemented | FastAPI, SQLAlchemy, and Redis instrumentations with OTLP HTTP export. Configurable via `OTEL_ENABLED`. | |
666 | 704 | | Request Size Limiting | Required | Implemented | `LimitRequestSizeMiddleware` rejects oversized write requests. | |
667 | 705 | | Idempotency Support (for applicable POST endpoints) | Optional but valuable | Implemented | Supports `Idempotency-Key` replay caching for POST responses. | |
668 | 706 | | Database Migrations | Required | Implemented | Alembic is configured with migration commands in the README and Makefile. | |
|