Gemini API implicit caching returns 0 cached tokens: why cache hits stop and how to get them back

Last verified: November 14, 2026
Developers running long prompts on Gemini 4 Argon, Gemini 4 Hydrogen and other recent Gemini models expect repeated requests to come back cheaper, because Google caches the start of a prompt automatically and bills the reused part at a steep discount. Many of them now see the opposite: the response reports no cached tokens at all, even when the request starts with exactly the same instructions and documents as the one sent a minute earlier, and the bill grows as if nothing had been reused. The problem shows up most often in agent loops, chat backends with long system instructions and batch jobs that send one large reference document with many small questions.
Implicit caching is best effort, and any change at the start of the prompt breaks it. Google's documentation says the feature carries no cost-saving guarantee, works only above a minimum prompt size, matches the beginning of the request and keeps entries for a limited time. Most misses come from content that changes near the top of the prompt, from prompts below the minimum, from requests spaced too far apart or from code that still tries to use explicit caching, which Gemini 4 models reject. Since late October developers on Google's official forum have also reported hit rates on Gemini 4 Argon falling from above 90% to about 25% with no change on their side, and Google has asked for request IDs without confirming a cause.
··········
THE GEMINI API CACHE-MISS SYMPTOM: WHAT ZERO CACHED TOKENS LOOKS LIKE.
The only place a cache hit becomes visible is the usage block of each response. In the Generate Content API the field is `cachedContentTokenCount` inside `usageMetadata`, described in the API reference as the number of tokens in the cached part of the prompt. The total prompt size stays in `promptTokenCount`, which still includes the cached tokens, so a hit shows a large prompt count together with a large cached count, and a miss shows the same prompt count with no cached count.
A Google staff member confirmed on the developer forum in October 2026 that the field is left out of the response when zero tokens came from the cache. Code that expects the key to always be present can crash or log nothing, and many dashboards record the missing key as an error instead of a miss. The Interactions API reports hits in a different place: the caching page lists `usage.total_cached_tokens` for the Python and JavaScript SDKs.
........
Where you look | Field | What a miss looks like |
Generate Content API, REST | usageMetadata.cachedContentTokenCount | Key missing from the response |
Generate Content API, REST | usageMetadata.promptTokenCount | Unchanged, still the full prompt size |
Generate Content API, REST | usageMetadata.cacheTokensDetails | Missing, it lists cached tokens by modality |
Interactions API, Python and JavaScript | usage.total_cached_tokens | Zero or missing |
........
The illustrative response below shows the shape of a hit on a long prompt. The numbers come from a public reproduction on Gemini 4 Hydrogen, where about 93% of a 28,906-token request was read from the cache.
"usageMetadata": {
"promptTokenCount": 28906,
"cachedContentTokenCount": 26857,
"candidatesTokenCount": 412
}··········
GEMINI API IMPLICIT CACHING RULES FOR GEMINI 4 MODELS.
Implicit caching is switched on by default for Gemini 2.5 and every newer model, with nothing to configure. Google's caching page sets a minimum prompt size per model and gives two pieces of advice: put large, common content at the beginning of the prompt, and send requests with a similar beginning within a short time. Cached input on Gemini 4 Argon is discounted by 95% according to the model's "What's new" page, which is why a run of misses can multiply the input cost of a long-context application. Current per-token rates are in our Gemini API pricing guide.
........
Rule | Gemini 4 Carbon, Argon, Hydrogen | Gemini 3.x and 2.5 |
Implicit caching | On by default | On by default |
Minimum prompt for a cache hit (Gemini API) | 4,096 tokens | 4,096 tokens for 3.x listed models, 2,048 for 2.5 |
Explicit caching (cachedContents) | Not supported, returns an error | Supported on generateContent, beta |
Cost-saving guarantee | None for implicit caching | Guaranteed only with explicit caching |
Multi-turn reuse | Full previous request cached only if the new request adds at least 2,050 tokens | No such rule documented |
Retention | No fixed window stated | Explicit caches default to a 1-hour TTL |
........
The multi-turn rule is specific to Gemini 4. Model output, thoughts and function results all count toward the 2,050 added tokens, so most agent turns clear it. When a turn adds less, Google says the last 2,050 tokens of the previous request are processed again at the standard input rate. That matches the public Hydrogen test, where even exact repeats left a suffix of 2,049 tokens uncached.
Google Cloud's documentation for the same models on its enterprise platform adds two details that the Gemini API page does not state. A cached prefix there expires no later than 24 hours after creation, most entries are evicted earlier because memory capacity is limited, and cached content is never shared across projects. Minimums also differ on that platform: Gemini 3.7 Flash and 3.8 Flash need 6,144 tokens there for implicit caching.
··········
[[ADPLUS_300x250_1]]
WHY THE GEMINI API RETURNS NO CACHED TOKENS: CAUSES, CHECKS AND FIXES.
Every documented cause comes back to the same mechanism: the cache stores the beginning of a request, and a hit needs a later request that starts with the same tokens while the entry is still stored. The table lists the causes in the order to check them.
........
Cause | How to check | Fix |
Prompt below the model's minimum | promptTokenCount under 4,096 on Gemini 4 | Accept no caching, or move the shared material into one longer fixed block |
Changing text near the start | Diff the first part of two consecutive request bodies | Keep systemInstruction static, put per-request data in a final part of contents |
Gemini 4 turn adds fewer than 2,050 tokens | Compare promptTokenCount between turns | Expected behavior, the last 2,050 tokens are billed at the normal rate |
Entry evicted between requests | Misses cluster after gaps or at low volume | Send similar requests close together, judge hit rates on real traffic |
Tools or response schema changed between calls | Diff the tools list and responseSchema | Send identical tools and schema in the same order (reported by users) |
Old explicit-caching code on Gemini 4 | Errors from caches.create or cached_content | Remove explicit caching calls, rely on implicit caching |
Missing field read as a failure | Code assumes cachedContentTokenCount always exists | Treat a missing key as zero cached tokens |
Server-side drop with no code change | Hit rate fell on Argon or Hydrogen in November | Collect responseIds, UTC times and project ID, report them to Google |
........
··········
FIXING GEMINI IMPLICIT CACHE MISSES STEP BY STEP.
Start by measuring. Log `promptTokenCount` and `cachedContentTokenCount` for every request, with the model name and a UTC timestamp, and store a missing cached count as zero. A Google staff member said in October that a test of about four requests is too small for the best-effort cache to warm up, so judge the hit rate on at least a few dozen calls of real traffic.
Then make the start of every request byte-for-byte identical:
1. Put the fixed material first: the system instruction, the tool definitions and any long reference document that every request shares.
2. Remove anything that changes per request from the system instruction. Timestamps, user names, request IDs, session counters and retrieved search results all belong at the end of the request.
3. Following the staff advice on the forum, keep `systemInstruction` 100% static across requests and send dynamic per-request content as a trailing part inside `contents`.
4. Serialize the request the same way every time. Tools in a different order, a reformatted JSON schema or a reordered list of files produce a different token sequence, and the cache matches tokens.
5. In multi-turn conversations, resend the earlier turns exactly as they were returned. Google's Gemini 4 Argon migration page asks clients to pass the full history including thought signatures; a client that trims, summarizes or rewrites older turns sends a new beginning and loses the cached copy.
Next, check the timing. Implicit entries are kept for a limited and unstated time, so a job that sends one request every half hour will rarely hit. Grouping requests that share a large document into a short burst gives the cache a better chance, and on Google Cloud the documentation states that entries can stay in a slower durable tier after memory eviction.
For code moved from Gemini 2.5 or Gemini 3, look for `caches.create`, `cachedContents` and the `cached_content` setting in `GenerateContentConfig`. Google's Argon migration notes say explicit caching is not supported on Gemini 4 Argon and returns an error, and the caching page says the same for later Gemini 4 models. A workload that needs guaranteed savings can stay on a supported Gemini 3 model with an explicit cache, which defaults to a one-hour lifetime unless you set another TTL. Gemini 2.5 Pro and 2.5 Flash calls are rerouted to newer models from December 1, as described in our guide to Gemini model names and retired IDs.
When all of the above is in place and hits still collapse, record the evidence Google asks for. In the November thread on Gemini 4 Argon, a Google staff member requested the affected `responseId` values, UTC timestamps and the Project ID through a form linked in the reply. Keep pairs of requests that should have matched, with their token counts, because they make the report reproducible.
··········
WHAT GOOGLE DOCUMENTS AND WHAT GEMINI DEVELOPERS ONLY REPORT.
Several explanations that circulate in forum threads have no confirmation from Google. The table separates them from the documented rules.
........
Statement | Status |
Implicit caching has no cost-saving guarantee | Documented, repeated by Google staff in November |
4,096-token minimum for Gemini 4 models | Documented on the caching page |
Explicit caching unsupported on Gemini 4 | Documented, returns an error |
Missing cachedContentTokenCount means zero cached tokens | Confirmed by Google staff on the forum |
Long system instructions are harder to cache because of chunking and routing | Said by a Google product manager in October |
Argon hit rate fell from 93% to 25% between Oct 25 and Nov 9 | Reported by one developer with logs, no cause confirmed |
Adding or removing a response schema caused streaks of misses | Reported by developers, not documented |
A one-character change in the last user message lost the whole cache on Hydrogen | Reported in a public reproduction, unexplained |
Cache cuts at a fixed boundary after systemInstruction and tools | Reported on Hydrogen, unexplained |
........
The Argon report, filed on the official Google AI Developers Forum, describes misses that arrive in streaks: after a hit, the next request hit 83% of the time, after a miss only 34%. Cached counts were all or nothing, either most of the prompt or none of it, and input token counts moved by about 700 tokens the client had not sent. Reports on Gemini 4 Hydrogen followed on November 14 with the same pattern. A Google product manager said in October that caching improvements had been released for newer models and would reach older ones, and that the documentation on implicit caching could be expanded; neither statement names a date.
··········
MONITORING GEMINI API CACHE HITS AFTER MODEL AND SDK UPDATES.
Cache behavior changes with models and SDK releases, and the minimums in Google's table have moved before. A hit-rate metric catches a regression on the day it starts, instead of at the end of the billing month. The snippet below is illustrative and uses the Python SDK's snake-case names for the documented fields.
usage = response.usage_metadata
cached = usage.cached_content_token_count or 0
ratio = cached / usage.prompt_token_count
log_metric("gemini_cache_ratio", ratio, model=MODEL_ID)Alert on the ratio for each model, and compare it with the share of requests above the minimum prompt size, because a change in prompt length often explains a drop before any server issue does. After each update to prompts, tools or schemas, look at the first hour of traffic: a reordered tool list or a new field near the top of the system instruction shows up there as a sudden fall to zero. Rate limits and 503 errors can also distort retries and timing, as covered in our guide to Gemini API rate limits and 429 fixes and our page on the Gemini API 503 high-demand error.
··········
GEMINI API IMPLICIT CACHING: WHAT DEVELOPERS CAN CONTROL.
On Gemini 4 models the only cache available through the Gemini API is the automatic one, and Google promises no hit rate, no retention window and no savings. Developers control the size of the shared prefix, keep it identical from request to request, send related requests close together and read the usage field correctly, and those steps fix most zero-cache results that come from the client side. The November drops on Argon and Hydrogen remain unexplained, with Google collecting request IDs and no fix announced; teams whose costs depend on guaranteed reuse can keep that workload on a supported Gemini 3 model with an explicit cache until Google documents more about how Gemini 4 caching decides what to keep.
··········
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
datastudios.org




