> ## Documentation Index
> Fetch the complete documentation index at: https://platform.stepfun.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt cache best practices

When building multi-turn conversations or fixed persona Q\&A, you pass conversation history on every turn. As turns grow, repeated content accumulates.

Stepfun provides context caching: each request’s input is cached, and repeated requests automatically hit the cache. This speeds up inference (up to 90% faster time-to-first-token in some cases) and reduces cost. **Cached input tokens are billed at the model’s cache-hit input rate; discounts vary by model.** For example, `step-5-preview` costs \$0.10 / 1M cached input tokens, 10% of its cache-miss input rate (\$1.00 / 1M tokens). See [Pricing and Rate Limits](/docs/en/guides/pricing/details) for current prices.

## Supported models

Prompt caching is supported on `step-5-preview`, `step-3.7-flash`, `step-3.5-flash`, and `step-3.5-flash-2603`. For other models, check the corresponding model documentation and pricing table.

## How prompt caching works

Caching automatically activates when a request exceeds 256 tokens. When you call the API:

1. **Cache lookup**: the system checks for a cached prefix of your prompt.
2. **Cache hit**: if found, the cache is reused and only the uncached portion is processed.
3. **Cache miss**: if not found, the request is processed normally and the prompt prefix is cached for future use.

Cache hit rates are higher for prompts with multiple examples, multi-turn history, or long context/background sections. Eviction uses LRU: during peak traffic, unused cache entries are evicted more aggressively; off-peak they live longer.

## How to tell if caching was used

A positive `response.usage.cached_tokens` value indicates a cache hit and reports the number of cached input tokens. A value of 0 means no input tokens were served from cache.

The following is an illustrative response; actual output depends on the request:

```json theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
{
    "id": "22ebae159c8f10c8657253671c8f7f17.6de8d4f65f5ba3dddeb693a1aa83de1e",
    "object": "chat.completion",
    "created": 1730963954,
    "model": "step-5-preview",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "xxx"
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "cached_tokens": 512,
        "prompt_tokens": 591,
        "completion_tokens": 120,
        "total_tokens": 711
    }
}
```

In this example, total input remains 591 tokens: 512 are billed at the cache-hit input rate, and 591 - 512 = 79 at the cache-miss input rate. Caching does not reduce the actual input token count or make cached tokens free. Input cost is `(79 × cache-miss input rate + 512 × cache-hit input rate) / 1,000,000`, with rates expressed in USD per 1M tokens. Output tokens are billed separately.

## Cached content

Caching uses 256 tokens as the minimum unit. Prompts shorter than 256 tokens will not hit the cache.

The following content can be cached:

* **Conversation messages**: the full array, including system prompt, user messages, and assistant messages.
* **Images**: images within user messages, as long as the same images are reused.
* **Video**: videos within user messages, with the same constraint.
* **Tool calls and results**: tool calls inside the conversation and their outputs.

## Prompt caching tips

* Put static or repeated content at the beginning of the prompt, and place dynamic parts at the end.
* Monitor ratios of `cached_tokens` vs `prompt_tokens`, latency, and hit rate to optimize prompt structure.
* Cache entries unused recently may be evicted. Keep the prompt prefix stable to improve hit rate.
* For long prompts, send them during off-peak hours to reduce evictions and improve hit rate.

## Troubleshooting cache misses

If a request is not caching as expected, check the following:

1. Ensure the cached portion stays identical across calls.
2. Verify calls are not spaced so far apart that the cache expires.
3. Provide at least 256 tokens of input.

## FAQ

### Does caching affect model quality?

Caching does not affect model quality. Each generation still uses the full prompt.

### Do I pay extra for prompt caching?

There is no separate paid cache feature to enable. Uncached input tokens, including the first write to the cache, use the cache-miss input rate; cached input tokens use the cache-hit input rate. For `step-5-preview`, the cache-hit input rate is 10% of the cache-miss input rate. See [Pricing and Rate Limits](/docs/en/guides/pricing/details).

### Can I clear the cache manually?

Manual cache clearing is not supported. Modify the prompt if you need to avoid cache hits.

### Can I force every request to hit the cache?

Guaranteed cache hits are not supported. If you need this capability, contact us to discuss your scenario.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.