Exact financial models for AI infrastructure decisions. Make the LLM do the modeling — but never the arithmetic. Live site: copyleftdev.github.io/rational
This repo is the companion to the guide "The AI Cost Modeling Handbook." Every number here is produced by a two-stage pipeline:
- Research agents pull live, cited provider prices and quality benchmarks (observed dates recorded in
data/). agent-calc— an AI-native exact rational computation kernel — does the arithmetic. No float drift, no LLM hallucinating a multiplication. Where a solver must use floats (LP), the result is re-verified in exact rationals.
The point: a cost model you can audit. Re-run any chapter and you get bit-identical numbers, with the source of every input price one file away.
All chapters share one production-agentic workload profile (Hermes-Agent-style tool loops):
| Component | Share of tokens | Why |
|---|---|---|
| Fresh input | 21.25% | New context each turn |
| Cached input | 63.75% | System prompt + skills + history re-sent every tool-loop step (75% of input is a cache hit) |
| Output | 15% | Agents emit little text, mostly tool-call args |
blended $/1M = 0.2125·in + 0.6375·cached + 0.15·out
| # | Model | Money question | agent-calc domain | Status |
|---|---|---|---|---|
| 00 | Token-provider ranking | Cheapest provider per quality unit to drive an agent | rational |
✅ data/ |
| 01 | Multi-provider routing LP | Optimal traffic split minimizing spend under a quality + privacy floor | linear + eval |
✅ |
| 02 | Self-host vs. serverless | Utilization a GPU node must hold to beat per-token API | finance, eval |
✅ |
| 03 | The reasoning-token tax | Cost per solved task once you count thinking tokens | eval, solve |
✅ |
| 04 | The retry cascade | Cheap→premium escalation: near-total coverage at a fraction of frontier cost | stats, eval |
✅ |
| 05 | NPV of waiting | Reserved-capacity prepay vs. riding the price decline | finance |
✅ |
| 06 | Prompt-cache ROI | Reuse count where caching pays off vs. write premium | eval, interval |
✅ |
| 07 | Agent-loop compounding | Why turn N costs O(N²) without compaction | polynomial |
✅ |
- Site — a kinetic single-page presentation of the findings in
docs/(light-mode, self-drawing SVG charts). Deploy via Settings → Pages → branchmain, folder/docs. Serve locally withpython3 -m http.server -d docs. - Article — the full write-up in
article/the-ai-cost-modeling-handbook.md, all eight chapters as one narrative with charts. Regenerate figures withpython3 article/charts/make_charts.py.
make ch01 # optimal routing LP frontier make all # every chapter
Every model script is self-contained Python that shells out to agent-calc. Inputs live in data/ with observation dates and source URLs.