If you send the same instructions every time, cut costs with prompt caching

When you add an AI feature to a service, the same content repeats in every request. A role description, answer rules, and reference material such as a product list or manual sit at the very front each time, and only the user's question changes at the end. That repeated front part is billed as input every single time.

We briefly introduced caching in our post on estimating LLM API costs. This time, we have organized the principles and design criteria you need when actually adding prompt caching, based on Anthropic's and OpenAI's official documentation.

What prompt caching stores

Prompt caching is a feature where the server temporarily stores the front part (prefix) of a request and reuses it when the next request starts with exactly the same content. The reused part is processed faster, and its input price is discounted.

The key point is that it must be "identical from the beginning." Anthropic's documentation explains that for a cache hit, the prompt must be 100% identical up to the marked point, and OpenAI's documentation likewise says the entire prefix must match for the cache to hit.

Comparing the two providers

Writing to the cache for the first time costs extra. Anthropic charges 1.25× base input (2× for the 1-hour cache), and OpenAI also lists 1.25× for newer models (GPT-5.6 and later). So the benefit is small for prompts used only once, and large for features that reuse the same prefix many times. The numbers vary by model, so check the pricing for the model you use in each provider's documentation before adding it.

Ordering your prompt so the cache hits

Most of the caching benefit comes down to prompt order.

• Put what never changes at the very front: role description, answer rules, tool definitions, reference material
• Put what changes at the very end: the user's question, today's date, per-user information
• Keep timestamps out of the prefix: Anthropic's documentation warns that if you place a cache breakpoint on a block containing a value that changes every request (such as the current time), it will be written to the cache every time and never reused
• Don't change tool definitions often: on Anthropic, changing tool definitions invalidates all the cache that follows them

Checking whether the cache actually hit

Adding it is not the end — you need to confirm it with the usage values returned with each response.

• Anthropic: cache_creation_input_tokens (amount newly stored) and cache_read_input_tokens (amount reused). If both are 0, caching did not happen
• OpenAI: check the number of cached input tokens shown in the response usage

Log these values for the first few days after deployment. If almost nothing is being reused, start by checking whether a value that changes per request has slipped into the front of the prompt. Note that on Anthropic, caches are stored separately per organization (workspace).

Summary: the order to add it

1. First measure how long the repeated prefix is in each request (no benefit if it is shorter than the minimum length)
2. Rearrange the prompt in "fixed → variable" order
3. On Anthropic, turn on cache_control; on OpenAI, confirm the model is supported
4. Log the cache usage values in responses to confirm the effect

If a feature sends the same instructions over and over every day, caching is a way to reduce both cost and waiting time without changing the feature itself.

References

• Anthropic, Prompt caching: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
• OpenAI, Prompt caching: https://developers.openai.com/api/docs/guides/prompt-caching

SOL LABS AI

We build web apps, mobile apps and business systems, then add and run the AI they need. We build and operate our own meeting notes, contract and company analysis AI.

See our projects