InsightStream is an enterprise-grade, real-time data ingestion and analytics visualization platform. The application leverages a distributed architecture to process large-scale financial CSV data asynchronously in the background, minimizing main-thread execution time and providing users with live dashboard statistical updates via server-driven UI swaps.

• Asynchronous Processing Pipeline: High-throughput data parser powered by Celery background workers and Redis event brokers.
• Real-Time UI State Updates: Live data status updates and metric calculations driven by HTMX polling and Out-of-Band (OOB) structural swaps—zero page refreshes required.
• Dynamic Visual Trend Modeling: Interactive time-series line graphing tracking daily aggregated revenue points via Chart.js.
• Defensive Fault-Tolerance Engine: Robust error isolation mechanics that capture, report, and render data serialization exceptions directly on the UI log rows without system interruption.
• Enterprise Data Streaming Exporter: One-click, database-to-client memory-efficient data compiler streaming raw data into formal .csv formats.

• Backend Framework: Django (Python 3)
• Task Queue & Broker: Celery + Redis
• Primary Relational Database: PostgreSQL
• Containerization Engine: Docker + Docker Compose
• Frontend UI Layer: HTMX, Bootstrap 5, Chart.js

The executive control command center displaying active rolling financial totals, real-time logging, caught processing errors, and responsive trend charts.

The centralized backend data management layer configured to monitor base tables, user authentication levels, and automated periodic scheduling tasks.

 [Client Browser] ──(Uploads File)──> [Django App Engine] ──(Enqueues Task)──> [Redis Broker]
        ▲                                     │                                     │
        │ (HTMX Polling & OOB Swaps)          ▼ (Creates Record)                    ▼
 [Pristine Dashboard View] <────── [PostgreSQL Database] <───(Parses Data)─── [Celery Worker]

Follow these steps to spin up the entire distributed infrastructure locally on your machine using Docker Compose.

Ensure you have the following installed on your system:

• Docker Desktop (Engine v20+)
• Docker Compose V2

1. Clone the Repository

git clone [https://github.com/rajneeshpython/insightstream.git](https://github.com/rajneeshpython/insightstream.git)
cd insightstream

2. Spin Up the Docker Infrastructure Run the compose engine in detached mode. This automatically pulls images and initializes PostgreSQL, Redis, the Django Web App, and the Celery Background Worker simultaneously:

docker-compose up --build -d

3. Run Database Schema Migrations Apply database tables to your live PostgreSQL container instance:

docker-compose exec web python manage.py migrate

4. Create an Admin Superuser Account To access the Django Administration portal shown in the showcase, generate an admin credential profile:

docker-compose exec web python manage.py createsuperuser

5. Explore the Applications Once initialization completes, you can access the entry nodes directly:

• Main Analytics Dashboard: 🌍 http://localhost:8080/
• Django Administration Portal: 🔐 http://localhost:8080/admin/

To gracefully stop and remove all application service containers, run:

docker-compose down

product_name,quantity_sold,unit_price,sale_date
Serverless Database Unit,8,120.00,2026-06-21
CDN Edge Pop License,15,45.00,2026-06-21
IAM Security Audit Tool,3,350.00,2026-06-22

product_name,quantity_sold,unit_price,sale_date
Malicious Exploiter Pod,TEN,150.00,2026-06-23
Corrupted Memory Buffer,4,FREE,2026-06-23