What gets cached
Providers typically discount a stable prefix—system instructions, tool schemas, or shared documents—when later requests reuse the same bytes. Dynamic user text still bills at the normal input rate.
Hits, misses, and writes
A high hit rate is where savings appear. Some providers also charge a cache-write premium when a prefix is first stored. If your traffic is bursty or prefixes rotate often, write costs can erase gains.
Plan before you ship
Use the Prompt cache savings calculator with your prefix size, dynamic tokens, and expected hit rate. Compare models that publish cached-input rates before locking architecture.
FAQ
Is prompt caching the same as prompt compression?
No. Compression shortens the prompt. Caching keeps a long prefix but bills repeats cheaper when the provider supports it.