Ideal cost assumes every attempt succeeds
Most budgets multiply tokens × price × intended jobs. That ignores the second (and third) attempt after a timeout or 5xx—each of which may still bill input or partial output.
Not every failure costs the same
A 429 rate-limit response is often free but still burns latency and queue capacity. A mid-stream disconnect may bill the prompt plus tokens already generated. Treat billing mode as a planning bound, not a universal rule.
Retries raise completion rate and spend
With a per-attempt success rate p and max attempts N, job completion probability is 1 − (1 − p)^N. Expected attempts and cost grow with every failed try you allow. Cap retries and prefer idempotent, cheaper fallbacks when reliability matters more than one model.
FAQ
Should I always retry on 429?
Usually yes with backoff—but 429s often aren’t billed. The cost is latency and delayed capacity, not token meters. Pair retries with the rate-limit planner.