Cost and latency pull opposite ways
Flagship reasoning models often cost more and respond slower. Flash/mini/nano tiers are cheaper and usually snappier—but may need more retries or weaker answers for hard tasks.
Use a latency budget
Decide the maximum p50 latency your product can accept, then pick the cheapest model that stays under that budget for your token sizes. Measure your own latencies when you can; planning defaults are only a starting point.
Optional: price user time
If every extra second of wait has a dollar impact (support agents, checkout flows), add a per-second penalty so effective cost reflects both API spend and delay.
FAQ
Is CentsPerToken latency data a benchmark?
No. Defaults are editable planning figures. Replace them with your measured p50 before making production routing decisions.