OpenRouter Prompt Caching: Repeated Context, Provider Support, and Cost Optimization Explained
- 1 day ago
- 16 min read
OpenRouter prompt caching lowers inference cost when repeated context is arranged so that supported providers can reuse stable prompt material across later requests, which makes the feature most valuable in workflows where long instructions, source packs, schemas, tool definitions, policies, or document context appear again and again.
The important distinction is that OpenRouter does not create one universal prompt cache across every model and provider, because prompt caching depends on provider-side support, routing continuity, request structure, cache-control syntax, time-to-live behavior, and measured cache usage.
Repeated context becomes economically useful only when the application keeps the reusable part stable, sends the request to a compatible provider route, avoids breaking the prefix with dynamic metadata, and checks whether cached tokens are actually being read rather than assuming that long prompts automatically become cheaper.
For production teams, prompt caching belongs in the same design conversation as model selection, provider routing, session identity, privacy settings, response caching, and cost analytics, since each of those choices changes whether repeated context saves money or simply adds another hidden assumption to the workflow.
·····
OpenRouter prompt caching reduces cost when repeated context remains reusable.
Prompt caching is designed for requests where a large portion of the prompt remains the same while the final question, latest user message, retrieved item, or task instruction changes.
That pattern appears in customer-support assistants that reuse policy rules, coding agents that reuse repository context, legal review workflows that reuse a contract or clause library, research systems that reuse a source pack, and product-enrichment pipelines that reuse taxonomy rules.
The first request gives the provider the long context, while later requests can receive a discount on the cached portion when the provider recognizes that the same material has already been processed.
This does not make every request cheaper, because new input still costs money, generated output still costs money, reasoning or tool-related costs still apply where relevant, and cache writes can have their own pricing depending on the provider.
The economic value comes from repeated reuse, so a one-time long prompt over a document rarely benefits as much as a multi-turn session or batch workflow where the same context supports several outputs.
........
Prompt Caching in Repeated-Context Workflows.
Workflow pattern | Repeated context | Variable content |
Customer-support assistant | Policy rules, tone guide, escalation matrix | New ticket or customer message |
Coding assistant | Repository instructions, architecture notes, selected files | New question, error, or patch request |
Legal review assistant | Contract template, clause library, review rubric | New clause or counterparty note |
Finance analysis assistant | Reporting framework, chart of accounts, memo format | New period, metric, or variance |
Product catalog assistant | Taxonomy, attribute rules, normalization guide | New product title or description |
Research assistant | Source pack, citation format, analysis framework | New question or section request |
Agent workflow | Tool definitions, system prompt, task state | New tool result or next step |
·····
Prompt caching works best when stable material stays at the front of the request.
Most prompt-caching systems reward stable prefixes, which means the beginning of the request needs to remain as consistent as possible across calls that are expected to share a cache.
System instructions, role rules, output schemas, tool policies, few-shot examples, document packs, and source material generally belong before the changing user question or latest tool result.
Dynamic material such as timestamps, request identifiers, user-specific variables, fresh retrieval fragments, and transient status messages can break cache reuse when inserted before the stable content.
The practical prompt-layout problem is therefore simple but easy to miss, because a small changing line placed at the top of a system message can make the provider treat the full context as new.
Cache-friendly design separates the stable prefix from the dynamic tail, then keeps serialization, message order, and tool definitions consistent enough that later requests still resemble the earlier request at the point where caching matters.
........
Prompt Layout for Cache-Friendly Requests.
Prompt area | Cache-friendly content | Cache-breaking content |
System or developer message | Stable role, rules, style, taxonomy, tool policy | Request IDs, timestamps, user-specific dynamic data |
Large reference block | Static documentation, product catalog, policy pack | Frequently changing query results |
Few-shot examples | Stable examples and labels | Examples regenerated per user |
Conversation history | Stable prior context in a continuing session | Reordered or rewritten history |
RAG context | Reused source pack or stable document | Fresh retrieval fragments placed before stable material |
User request | Current question or task | Belongs after stable reusable content |
Tool output | Latest external result | Better near the end unless intentionally cached |
·····
Provider support determines whether caching happens automatically or needs explicit controls.
OpenRouter routes requests across providers, while prompt caching depends on whether the selected provider and model support automatic caching, implicit caching, or explicit cache-control breakpoints.
Some providers handle caching automatically when the prompt is large enough and the prefix remains stable, which reduces implementation work but gives the developer less fine-grained control over which block is cached.
Other providers require explicit cache markers on content blocks, which gives the application more control but also creates responsibility for breakpoint placement, provider compatibility, and request-shape consistency.
This provider variation matters because an application can use the same OpenRouter interface while receiving different caching behavior depending on the model route behind the request.
A cost review that ignores provider-specific cache support can misread the workflow, especially when one route creates cache hits while another route processes the same prompt as ordinary input.
........
Provider Prompt-Caching Support on OpenRouter.
Provider family | Caching pattern | Practical note |
OpenAI | Automatic | Minimum prompt size applies and cache reads use discounted input pricing |
Grok | Automatic | No extra configuration required in supported routes |
Moonshot AI | Automatic | No extra configuration required in supported routes |
Groq | Automatic for supported Kimi K2 models | Model support is narrower |
DeepSeek | Automatic | Cache write and read pricing differ from ordinary input pricing |
Alibaba Qwen | Explicit breakpoints | Cacheable block needs cache_control |
Anthropic Claude | Automatic or explicit breakpoints | Route and API shape affect support |
Google Gemini | Implicit caching with breakpoint guidance | Stable prefix design and final-breakpoint behavior matter |
·····
Sticky routing keeps cache-friendly sessions closer to the provider that warmed the cache.
Prompt caching becomes less useful when a request warms a provider cache and the next request goes to a different provider endpoint that has never seen the repeated context.
OpenRouter addresses this with sticky routing, which improves cache continuity by sending later requests in the same cache-friendly flow back to the provider that previously served the request when doing so is economically useful.
This matters most in multi-turn chats, coding sessions, research agents, document-review flows, and any application where a large context remains active across several turns.
Routing still involves trade-offs, because provider fallback protects uptime while narrower routing improves cache predictability.
When a manual provider order is specified, explicit routing priorities can take precedence over cache-oriented stickiness, which means caching and provider policy need to be designed together rather than treated as separate optimization layers.
........
Sticky Routing Effects on Prompt Caching.
Routing condition | Cache consequence | Deployment implication |
Same model and same sticky provider | Higher chance of cache reuse | Useful for repeated-context sessions |
Manual provider order | Explicit order takes priority | Cache routing becomes more controlled but less automatic |
Provider unavailable | Fallback can serve the request | Uptime improves while cache continuity can drop |
Router model selected | Sticky routing can pin resolved model and provider | Prevents model changes across a session |
Cache-read price not cheaper | Sticky routing does not activate | Routing avoids a cache path that would not save cost |
Different conversation key | Different sticky route can be used | Session identity matters for agents and apps |
·····
Session IDs give agents and conversations a clearer cache identity.
Default sticky routing can infer conversation identity from early messages, but agent workflows often change their state as the task progresses, which makes explicit session identity more reliable.
A stable session ID gives OpenRouter a clearer signal that later requests belong to the same conversation, repository task, document review, support ticket, or research job.
This is especially useful when the first messages are not perfectly stable, when agents rewrite their plans, or when a workflow starts with a router model and needs the resolved provider to remain consistent.
Session IDs do not replace good prompt layout, because a stable identifier still needs a stable reusable prefix or explicit cache block in order to produce meaningful savings.
The best pattern combines a stable session key with stable instructions, stable source packs, predictable tool definitions, and dynamic content placed after the reusable context.
........
Session Identity for Cached Workflows.
Workflow | Useful session key | Reason |
Chat assistant | Conversation ID | Keeps follow-up questions on the same cache route |
Coding agent | Repository task ID | Preserves cached repo instructions and source context |
Research agent | Research job ID | Keeps source pack and tool instructions warm |
Document-review flow | Document ID plus review session | Reuses long contract or report context |
Support workflow | Ticket ID | Keeps policy prompt and ticket state aligned |
Batch processing | Template version plus batch ID | Avoids mixing unrelated cache routes |
Router model workflow | Session ID with router model | Pins both resolved model and provider |
·····
Anthropic Claude caching through OpenRouter depends on breakpoints, API shape, and provider route.
Claude caching through OpenRouter needs special handling because automatic top-level caching and explicit content-block breakpoints serve different workflow needs.
Top-level caching works well when a conversation grows over time and the application wants the reusable context to move naturally with the chat history.
Explicit breakpoints fit workflows where a particular large block, such as a contract, policy pack, codebase excerpt, CSV, source bundle, or research file, deserves caching while the current question remains dynamic.
Provider route also changes behavior, because direct Anthropic routing, Bedrock routing, Vertex routing, and API choice do not always support the same cache-control pattern.
For applications that need precise control over which source block becomes reusable, explicit breakpoints provide a more deliberate design than relying on automatic behavior across a growing conversation.
........
Anthropic Caching Choices Through OpenRouter.
Need | Better caching pattern | Reason |
Multi-turn conversation | Top-level cache_control | Breakpoint advances as conversation grows |
Large stable document | Explicit content-block breakpoint | Only the expensive block is marked |
Bedrock or Vertex route | Explicit breakpoint | Top-level automatic caching can be limited by route |
Responses API workflow | Top-level automatic caching | Fine-grained breakpoints are not exposed in the same way |
Long session beyond a few minutes | Longer TTL where supported | Higher write cost can be offset by repeated reads |
Fine-grained context control | Chat Completions or Anthropic Messages | More direct breakpoint placement |
·····
Cache TTL changes the break-even point for repeated context.
A prompt cache has value only when the repeated context is reused before the cached material expires.
Short time-to-live windows work for immediate follow-up questions, tight review loops, batch processing, and agent steps that happen close together.
Longer TTLs become useful when the same large context remains active across a longer session, although the cache write can cost more and the break-even point depends on how many later reads occur before expiration.
This makes TTL a workflow decision rather than a purely technical setting.
A document-review batch that asks ten questions in five minutes has a different caching profile from an executive-review workflow where a user returns to the same source pack over an hour.
The correct TTL depends on reuse frequency, provider support, write cost, expected session length, and whether the application can schedule related work close together.
........
TTL and Cost-Optimization Trade-Offs.
Usage pattern | TTL concern | Cost implication |
One follow-up within minutes | Short TTL is usually enough | Avoids higher write cost |
Long agent session | Longer TTL can help where supported | Reduces repeated cache writes |
Batch questions over the same document | TTL needs to cover the batch window | Cache reads need to happen before expiry |
Infrequent repeated requests | Cache can expire before reuse | Little or no saving |
Gemini workflow | Short cache window matters | Related questions benefit from tighter grouping |
Anthropic explicit caching | TTL choice affects economics | Match cache lifetime to session length |
·····
Automatic caching and explicit breakpoints solve different prompt-design problems.
Automatic caching fits workflows where the whole conversation or stable prefix grows naturally, while explicit breakpoints fit workflows where the developer knows exactly which block is expensive and reusable.
A chat assistant with long history benefits from automatic handling because the relevant reusable context moves with the conversation.
A legal assistant reviewing the same agreement against different questions benefits from marking the agreement as the cacheable block, because each question can change while the contract remains reusable.
A coding workflow might cache repository instructions and selected source context while leaving the latest error message outside the cache boundary.
Breakpoint placement matters because a marker placed too early can exclude useful repeated material, while a marker placed too late can include dynamic content that changes on every request.
The practical goal is to cache the stable material that is expensive to process, without pulling in the current question, transient tool output, or request-specific metadata.
........
Automatic Caching and Explicit Breakpoints.
Caching style | Works well when | Main risk |
Automatic top-level caching | Conversation history grows and remains relevant | Provider route can narrow depending on model family |
Explicit block caching | One large source block repeats across many queries | Breakpoint placement can miss reusable context |
Provider implicit caching | Stable prompt prefix is maintained | App changes can break cache without visible code changes |
Gemini final breakpoint | One final cached block is enough | Earlier breakpoints can have limited effect |
Anthropic multiple breakpoints | Several large reusable sections need marking | Breakpoint limits require prioritization |
No explicit caching | Provider handles cache automatically | Less control over what becomes reusable |
·····
Prompt caching and response caching solve different cost problems.
Prompt caching reuses repeated context while still allowing a new answer to be generated, which makes it suitable when the same long prompt prefix supports different user questions or workflow steps.
Response caching reuses the entire prior response for an identical request, which makes it suitable for deterministic repeated calls, tests, retryable agent steps, monitoring checks, or situations where the same request should return the same output.
The distinction matters because response caching can skip the provider call entirely, while prompt caching still calls the provider and only reduces the cost of the cached prompt portion.
Prompt caching fits long-context workflows with changing outputs, while response caching fits identical requests where replaying the previous output is acceptable.
Applications that confuse the two can either miss prompt savings on repeated source material or accidentally replay an answer where a fresh completion was expected.
........
Prompt Caching Compared With Response Caching.
Feature | Prompt caching | Response caching |
Cache location | Provider infrastructure | OpenRouter edge layer |
Reuse pattern | Same or similar prompt prefix with new output | Identical request and identical response |
Provider support | Depends on provider and model | Model-agnostic across supported endpoints |
Billing effect | Reduces cached prompt-token cost | Cache hits have zero billable usage |
Output freshness | New completion still generated | Prior response is replayed |
Best use | Long repeated context and multi-turn work | Tests, retries, deterministic repeated requests |
Privacy consideration | Provider-side prompt cache behavior | Temporarily stores response data at OpenRouter edge |
·····
Privacy and ZDR settings change which caching modes belong in a workflow.
Caching has privacy implications because different cache types retain or reuse different material in different places.
Provider prompt caching generally concerns repeated prompt context inside provider infrastructure, while OpenRouter response caching temporarily stores the generated response at the OpenRouter layer for replay.
That distinction matters for Zero Data Retention workflows because response caching requires temporary response storage, whereas provider in-memory prompt caching can be treated differently depending on the endpoint policy and routing configuration.
Privacy-sensitive systems need to decide which cache types fit the workload before enabling them, especially when prompts contain customer data, confidential documents, legal material, source code, financial forecasts, or regulated records.
The caching decision belongs beside provider policy, data-retention settings, ZDR routing, API-key design, logging controls, and response visibility rather than being treated only as a cost feature.
........
Caching and Privacy Boundaries.
Caching type | Data retained | Privacy implication |
Provider prompt caching | Repeated prompt data in provider cache | Depends on provider policy and endpoint behavior |
OpenRouter response caching | Full response stored at edge until TTL expiry | Not suitable where response storage is prohibited |
Cache metrics | Token and cost details | Useful for audits without storing full prompt content |
Activity log | Generation and cache indicators | Supports monitoring and cost analysis |
API-key scoped response cache | Cached response tied to one API key | Key rotation changes cache continuity |
Provider fallback | Request can move to another endpoint | Cache hit can disappear and provider policy can change |
·····
Cache metrics reveal whether the workflow is saving money.
Prompt caching needs measurement because lower cost cannot be inferred reliably from the presence of long context or cache-control fields.
OpenRouter exposes usage details such as cached tokens, cache-write tokens, cache discounts, provider served, model resolved, generation ID, and related cost fields, which give teams a way to see whether the expected cache behavior actually occurred.
The most useful analysis compares prompt-heavy routes before and after cache-friendly restructuring, while separating workflows by model, provider, API key, session type, and prompt template.
Cache-write tokens show where the application paid to create or refresh a cache, while cached tokens show whether later requests reused that material.
Cache discount then shows the economic effect, including cases where writes cost more at first and reads create savings later.
Without these metrics, prompt caching becomes guesswork, especially in deployments where provider fallback, dynamic prompt builders, or inconsistent session handling can silently reduce cache hits.
........
Cache Metrics for Cost Review.
Metric | What it shows | Operational use |
cached_tokens | Tokens read from provider cache | Confirms cache reuse |
cache_write_tokens | Tokens written into cache | Shows first-write cost and refresh events |
cache_discount | Cost saving or surcharge from caching | Measures net economic effect |
Provider served | Endpoint used for the request | Confirms sticky routing behavior |
Model resolved | Actual model under router workflows | Detects model changes that break continuity |
API key | Which app or tenant generated traffic | Separates cache performance by workload |
Session ID | Sticky routing identity | Connects cache behavior to conversations or agents |
Generation ID | Individual request record | Supports detailed audits and debugging |
·····
Cost optimization depends on isolating stable context before measuring the result.
The strongest savings usually appear when a large stable prompt prefix is reused many times while only a small dynamic section changes.
That means the optimization work begins with identifying prompt-heavy traffic, not with adding cache controls everywhere.
After the expensive routes are visible, the application can separate stable instructions from dynamic data, move reusable source packs earlier, add explicit breakpoints where providers require them, use session IDs where continuity matters, and tune TTL according to session length.
Only after those changes does measurement show whether the cache writes, cache reads, fallback behavior, and total token costs justify the design.
A workflow that asks ten questions over the same document can benefit significantly, while a workflow that sends unrelated long documents once each will not gain much from prompt caching even if the prompts are large.
........
Cost-Optimization Workflow for Prompt Caching.
Step | Purpose | Evidence to check |
Identify prompt-heavy routes | Find workflows with large repeated context | Cost by model, API key, and generation |
Separate stable and dynamic content | Protect the cacheable prefix | Prompt template review |
Add session identity | Improve sticky routing continuity | Provider consistency across turns |
Enable explicit caching where needed | Mark large reusable blocks | cache_write_tokens and later cached_tokens |
Match TTL to workflow length | Avoid repeated writes | Cache-write frequency |
Monitor provider fallback | Detect cache-breaking route changes | Provider and model logs |
Compare net cost | Include write cost, read savings, retries, and output | cache_discount and total cost |
·····
Multi-turn and document-heavy workflows benefit most from prompt caching.
Prompt caching becomes most valuable when the same large context supports several outputs before the cache expires.
Long document Q&A, contract review, research drafting, coding sessions, customer-support threads, policy assistants, and agent loops all follow this pattern because the expensive source material or instruction layer stays constant across turns.
A single extraction request over one document has limited caching value because there is no later request to reuse the prompt.
A document workflow that asks for summary, risks, obligations, missing clauses, negotiation points, and a final memo has a stronger caching case because the same source block supports several answers.
Batch workflows can also benefit when the same schema, taxonomy, or instruction pack applies across many records, provided that the dynamic record data stays after the stable reusable prompt material.
........
Workflows With Strong Prompt-Caching Potential.
Workflow | Why caching helps | Design concern |
Long document Q&A | Same document supports many questions | Cache source block and keep question dynamic |
Coding session | Same repo context appears across turns | Use session ID and stable project instructions |
Research assistant | Same source pack supports multiple sections | Cache curated source material |
Customer support | Same policy layer supports many replies | Keep user ticket after stable rules |
Batch extraction | Same schema and rules apply repeatedly | Cache schema, examples, and taxonomy |
Agent retries | Same system prompt and tool policy repeat | Avoid changing tool definitions mid-run |
Legal review | Same contract context receives multiple checks | Mark large contract body for explicit caching |
·····
Provider routing needs to balance uptime, cache continuity, and policy control.
OpenRouter’s routing flexibility is valuable because it helps applications reach available providers, manage cost, and maintain uptime.
Caching adds another dimension because cache continuity often improves when repeated requests stay on the same compatible provider route.
An uptime-first deployment can allow broader fallback, accepting that provider changes might lose cache benefits during outages or rate limits.
A cache-optimized deployment can use session identity, provider order, or provider filters to keep repeated context on a route that supports the expected caching behavior.
A privacy-sensitive deployment adds another constraint, because ZDR, data-collection policy, provider retention, and endpoint choice can be more important than cache savings.
The practical design question is not whether routing or caching matters more, but which priority dominates for each workload.
........
Routing Choices for Cached Deployments.
Deployment priority | Routing pattern | Cache effect |
Maximum uptime | Allow fallback across providers | Cache can be lost during fallback |
Stable cache behavior | Use session ID and avoid unnecessary route changes | Higher cache continuity |
Specific provider economics | Use provider order or provider filters | More predictable cache pricing |
Anthropic via Bedrock or Vertex | Use explicit breakpoints rather than top-level automatic caching | Preserves compatibility |
Router model workflow | Use session identity | Pins resolved model and provider |
Privacy-sensitive routing | Combine ZDR or data policy controls with cache review | Cache behavior must match privacy requirements |
·····
Cache misses often come from small request differences.
A cache miss does not always come from an obvious architectural change, because small differences in prompt construction can make a repeated request appear new.
Changing timestamps, request IDs, user-specific metadata, message order, JSON serialization, tool definitions, streaming mode, model route, provider fallback, or prompt-prefix placement can reduce cache hits without changing the visible application behavior.
This is especially common when prompts are built through middleware, SDK wrappers, observability layers, template engines, RAG pipelines, or agent frameworks that add metadata before the stable content.
The fix is not only to enable caching but to inspect the final request body and confirm that the reusable part of the prompt remains identical where the provider expects it.
Cost analytics and generation-level records become necessary because developers often cannot see cache-breaking differences from the user-facing prompt alone.
........
Common Cache-Breaking Patterns.
Pattern | Why it hurts caching | Fix |
Timestamp in system prompt | Changes stable prefix every call | Move timestamp to later dynamic section |
Random request ID inside prompt | Makes repeated context unique | Keep IDs in metadata where possible |
Dynamic RAG before stable rules | Shifts the prefix before cacheable content | Put stable instructions first |
Reordered JSON body | Breaks response-cache identity | Use deterministic serialization |
Mixed streaming modes | Creates separate response-cache keys | Keep endpoint mode consistent |
Changing tool definitions | Alters repeated prompt structure | Version tools deliberately |
Provider fallback | Moves request away from warm cache | Use session ID and review routing policy |
·····
Response caching belongs in the strategy only when identical answers are acceptable.
Response caching is useful enough to include in a caching strategy, but it should not be mistaken for prompt caching.
When the request is identical and the previous answer remains acceptable, response caching can avoid the provider call and replay the stored output.
That fits unit tests, deterministic examples, monitoring checks, retryable agent steps, and repeated calls where freshness is not required.
It does not fit workflows where the same source context needs a new analysis, a different memo section, a different user answer, or a fresh completion.
The clean operational distinction is that prompt caching reuses context while response caching reuses the whole answer.
........
When to Use Each Cache.
Situation | Better cache | Reason |
Same policy prompt, different tickets | Prompt caching | Context repeats while output changes |
Same unit-test request, same expected output | Response caching | Entire response can be reused |
Same document, many questions | Prompt caching | Source block repeats |
Same failed agent step on retry | Response caching | Identical prior step can replay |
Same system prompt across users | Prompt caching | Stable instructions reduce repeated input cost |
Same health check request | Response caching | Provider call can be skipped |
Same source pack with new memo sections | Prompt caching | New outputs depend on reused source material |
·····
OpenRouter prompt caching needs workflow design rather than a single switch.
OpenRouter prompt caching creates value when repeated context is engineered deliberately, because the provider must receive recognizable stable material through a compatible route before cache reads can reduce cost.
The workflow needs stable prompt prefixes, explicit breakpoints where provider support requires them, session identity for multi-turn continuity, TTL choices aligned with reuse timing, and routing policies that do not accidentally move requests away from the warmed cache.
Provider support determines whether caching is automatic or explicit, while cache metrics confirm whether the application is producing cached reads, repeated writes, provider fallback, or no meaningful saving.
Privacy and ZDR settings also belong in the design because provider prompt caching, OpenRouter response caching, and logging each create different data-handling implications.
The practical rule is to cache the material that stays the same, move changing content later, keep related requests on compatible provider routes, and measure cached tokens and net cost before treating caching as a production optimization.
·····
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
·····