Cost formula
Effective cost = token cost + (timeout retry rate × retry cost) + (latency penalty value if modeled); Score = weighted rank of normalized cost and latency against your SLO target
Calculator
Latency vs cost tradeoff calculator
Compare selected models on the same token workload. Set a latency budget, optionally price user-time penalties, and see which model wins on cost, speed, or cost×latency score.
Latency SLO
GPT-5 nano
$23.50450ms · under 2,000ms
Cheapest under budget · score 0.0001
GPT-5 nano$23.50
GPT-5 mini$117.50
Gemini 2.5 Flash$145.00
$ / second of wait$0.00
Wait $ / req (pick)—
Wait business total—
Fastest planningClaude Haiku 4.5
SLO
Workload & budget
Compare up to 6 models on API cost vs planning latency.
Tradeoff
Ranked models
Score = cost per request × latency in seconds.
1$23.50
GPT-5 nano
2$117.50
GPT-5 mini
3$145.00
Gemini 2.5 Flash
4$350.00
Claude Haiku 4.5
Best cost×latencyGPT-5 nano
Per-request (pick)$0.000235
Effective total (pick)$23.50
Latencies are editable planning defaults, not measured benchmarks. Score = cost per request × latency in seconds. Updated 2026-07-31. Approximate static list prices for planning only. Always verify against each provider’s official pricing page before production budgeting.
Guide
How to get value from this calculator
Compare models by both API cost and planning latency to find options that meet your SLO without overspending. Faster frontier models may cost more per token but reduce timeout retries; cheaper models may save money while missing latency targets. Use this to inform routing and model selection tradeoffs.
Why it matters
Choosing purely on per-token price ignores retry storms and user churn from slow responses. Latency-aware cost planning helps you pick models that are cheap enough and fast enough for production SLAs.
How to use it
- Set your latency SLO (e.g., p95 time to first token or total response time).
- Enter input and output tokens per request for your workload.
- Compare candidate models on list price, latency tier, and $/second wait.
- Flag models that miss your SLO or exceed your cost ceiling.
- Factor retry cost if slow models increase timeout and retry rates.
- Validate latency assumptions with staged load tests—not benchmarks alone.
Planning tips
- Time to first token matters for streaming UX; total latency matters for batch-like tasks.
- Smaller models are often faster and cheaper—test whether they meet quality before defaulting to frontier.
- Regional endpoint selection affects latency independently of model price.
- Retries on timeout double token cost—include failure rates in the latency-cost tradeoff.
- Use $/second wait to put a dollar value on missing your latency SLO.
- Re-evaluate when providers release faster model variants at different price points.
Frequently asked questions
Where do latency numbers come from?
CentsPerToken uses planning-tier latency estimates for comparison, not live measurements. Run your own benchmarks for binding SLO decisions.
Should I optimize for cost or latency?
Depends on your product. Interactive apps usually prioritize latency within a cost ceiling; offline jobs prioritize cost. This tool helps you see both dimensions.
Are token prices exact here?
No. List prices are approximate planning figures. Verify official pricing alongside your latency testing.
Does streaming affect cost?
Streaming does not change per-token pricing but affects perceived latency and early-cancel behavior. See the streaming cost calculator for cancel-related savings.