Cost formula
Effective monthly cost = success-path cost × (1 + retry rate × avg retries per failure) + (failure token cost × failure rate); Inflated cost % = (effective cost − success cost) / success cost × 100
Calculator
Retry & failure cost calculator
Real traffic isn't 100% successful. Use this planner to estimate how rate limits, timeouts, and partial billing turn intended jobs into extra attempts—and extra API cost.
Retry overhead
Expected cost with retries
$155.40+$5.40 vs ideal
GPT-5 mini · ~1.11 attempts / job · 99.9% complete
Ideal$150.00
With retries$155.40
Success / attempt$0.0015
Fail / attempt$0.0005
Cost / job$0.001554
Waste on fails$5.55
Workload
Failure mode & volume
Common when the request dies before much output
Waste waterfall
Ideal vs retries
Stack bars show ideal spend, retry-inflated total, and waste on failed attempts.
~99,900 jobs complete · 100 abandoned · 11,100 failed attempts.
Sensitivity · overhead by success rate
70%$16.80 · $166.80
80%$11.20 · $161.20
90% (current)$5.40 · $155.40
95%$2.61 · $152.61
99%$0.5049 · $150.50
Billing on failures varies by provider and error type—use presets as planning bounds, not invoices. Updated 2026-07-31. Approximate static list prices for planning only. Always verify against each provider’s official pricing page before production budgeting.
Guide
How to get value from this calculator
Estimate how retries, timeouts, and partial failures inflate LLM API spend above nominal success-path costs. Failed requests that still bill tokens, exponential backoff retries, and duplicate submissions can add 5–30% to effective monthly cost. Model failure rates explicitly instead of treating them as noise.
Why it matters
Retry logic designed for reliability silently multiplies token bills. Finance sees higher invoices while engineering sees normal per-success token counts—retry cost explains the gap.
How to use it
- Enter success-path input and output tokens and monthly request volume.
- Set expected failure or timeout rate as a percentage of requests.
- Set average retry attempts per failed request before success or abandon.
- Add tokens billed on failures if your provider charges for partial completions.
- Read the waste waterfall from success-path cost through retries and failed tokens.
- Test lower failure rates via timeout tuning, smaller models, or circuit breakers.
Planning tips
- Idempotency keys prevent duplicate chargeable requests on client retries.
- Lower max_tokens on retry attempts when failures are truncation-related.
- Circuit breakers stop retry storms during provider outages—the most expensive failure mode.
- Use the waste waterfall to show finance why invoice spend exceeds success-path token math.
- Log billed tokens per attempt, not per successful response, for accurate COGS.
- Include eval and CI retry traffic in monthly failure-rate assumptions.
Frequently asked questions
Do providers charge for failed requests?
Often yes for tokens generated before failure. Policies vary—check your provider. This calculator lets you model partial token billing on failures.
What failure rate should I assume?
Start with 2–5% for production if you lack data. Measure actual timeout and error rates from logs and update quarterly.
Are inflated cost estimates exact?
No. Results depend on your assumptions and approximate list prices. Use as a planning range, not a guaranteed invoice figure.
How do retries interact with output token pricing?
Each retry attempt generates new output tokens billed at output rates. High retry counts disproportionately inflate output-side spend.