Prompt caching

When you send a prompt that begins with the same text as a recent one, the provider can serve the shared opening from cache instead of processing it again. Those tokens are billed at a reduced input rate, and the response tells you how many were cached.

There is nothing to switch on. No parameter, header, or SDK feature: send the same prefix and the discount applies when a cache hit happens.

What it costs

Cached input tokens are billed at the rate below instead of the normal input rate. Output tokens are never discounted.

A worked example on glm-5.2. A 9,372-token prompt where 9,344 tokens came from cache is billed as:

    28 uncached x €1.10/1M  =  €0.0000308
 9,344 cached   x €0.275/1M =  €0.0025696
                               ----------
                               €0.0026004

against €0.0103 at the undiscounted input rate, a 75% saving on the prompt. Completion tokens are charged on top at the normal output rate.

Reading it from the response

Cached tokens appear on the OpenAI-standard field usage.prompt_tokens_details.cached_tokens. It is always present, and 0 when nothing was served from cache.

"usage": {
  "prompt_tokens": 9372,
  "completion_tokens": 8,
  "total_tokens": 9380,
  "prompt_tokens_details": { "cached_tokens": 9344 },
  "completion_tokens_details": { "reasoning_tokens": 13 }
}

prompt_tokens is the total prompt size, cached and uncached together, so the tokens you paid full price for are prompt_tokens - cached_tokens.

Streaming reports the same field in the same place. Usage arrives once, on the final chunk, which carries an empty choices array. See Chat Completion (Streaming).

How to get cache hits

Caching works on a prefix, so the saving comes from what your requests share at the start.

• Put the stable material first: system prompt, instructions, reference documents, schemas. Put the part that changes, such as the user's latest message, last.
• Keep the prefix byte-identical. A changed word, a timestamp, or a reordered block near the start ends the shared prefix and everything after it is reprocessed.
• Multi-turn conversations cache naturally, since each turn repeats the transcript so far.

Two caveats worth knowing:

• Hits are opportunistic. Providers do not guarantee a cache hit, and the same prompt can miss on one request and hit on the next. Treat the discount as an optimisation, not a rate you can budget on exactly.
• Each compression model keeps its own cache. glm-5.2-caveman and glm-5.2-honey carry different built-in instructions, so identical user content sent to one gains nothing from the other having cached it. The same applies between them and plain glm-5.2.

Related

• Token Pricing
• Compression models
• API credits