top of page

Gemini API tool retrieval loads tool schemas on demand at no extra cost

1 hour ago
9 min read
Gemini logo with the headline Gemini API tool retrieval loads tool schemas on demand at no extra cost over a night city skyline, Data Studios original cover

Google has added tool retrieval to the Gemini API, a preview feature that lets an agent keep a large library of functions available without sending every full parameter schema to the model on every turn. The feature was announced on November 19, 2026 in a post on the Google blog written by Lucia Loher, product manager for the Gemini API, and Nora Kassner, research scientist at Google DeepMind. It works only in the Interactions API, requires the v1beta API version, and carries no separate fee: tokens are billed at the model's standard input, cached and output rates.


The mechanism is simple to describe. A developer adds a tool of type `tool_search` to the request and marks the functions that should stay out of the prompt with `defer_loading: true`. Gemini then sees only the name and a short description of each deferred function. When the model decides it needs one of them, it searches the catalogue, the API loads the full schema, and the model emits the function call, all inside a single API request. Each of those steps is returned in `interaction.steps` as a `tool_search_call`, a `tool_search_result` and a `function_call`, so the developer can see what was searched and what was loaded.


Google's tool retrieval documentation lists six supported models: `gemini-4-carbon`, `gemini-4-neon`, `gemini-4-hydrogen`, `gemini-4-argon`, `gemini-3.8-flash` and `gemini-3.6-flash`. The blog post describes the same scope as Gemini 3.6 and later, including all Gemini 4 models. Calling the feature on any other model returns an error stating that deferred loading is not supported for that model, and the older `generateContent` method does not support it at all.


··········


HOW GEMINI API TOOL RETRIEVAL WORKS IN THE INTERACTIONS API.


The default mode is server-side retrieval. Gemini runs the search on Google's side, so the developer does not need to build, host or update a search index. Every deferred function contributes only its name and its `short_description` field, or the first sentence of its `description` when no short description is given, to the prompt. The full parameter schema stays out of context until the model asks for it. Google recommends keeping frequently used tools loaded normally and deferring the long tail of functions that are called only occasionally.


The second mode is client-side retrieval. Here the developer declares a `tool_search` with `execution: "client"`, together with a name, a description and the parameters the search accepts. When the model calls it, the application runs its own lookup, which can be a tool registry, an embedding index or a permission filter, and returns the matching function declarations in `function_result.result`. Google says this adds one round trip per search, and positions it for dynamic registries, namespaces and per-user tools, including tools that only become available partway through a conversation. Server-side and client-side retrieval cannot be combined in the same request, and the `defer_loading` flag applies only to the server-side mode.


Remote MCP servers fit into the same model. Setting `defer_loading: true` on an `mcp_server` entry, together with a server-side `tool_search`, defers every tool exposed by that server. Calls to deferred MCP tools still run on the remote server and come back as `mcp_server_tool_call` and `mcp_server_tool_result` steps. For teams that already connect agents to several MCP servers, this is the most direct use of the feature, because a single flag covers an entire server instead of a list of individual functions.


The documentation sets several firm rules. Only one `tool_search` entry is allowed per request. Once a schema has been loaded it stays in context for the rest of the conversation and cannot be unloaded. The model may issue several searches in one turn. Because the API does not persist tool declarations, the full `tools` array, deferred definitions included, has to be sent again on every request, including follow-up turns. In stateless use with `store=false`, the `tool_search_call` and `tool_search_result` steps must be replayed in their original order with their `signature` fields untouched: dropping the call or stripping a signature returns a 400 error, while dropping the result silently removes the loaded schemas from context. A query that matches no deferred tool returns a text response, not a structured error, so agent code that only checks the interaction status can miss a failed lookup. Google advises inspecting `interaction.steps` for `function_call` entries instead.


··········


GEMINI API TOOL RETRIEVAL PRICING AND TOKEN BILLING.


Google does not charge for tool retrieval as a feature, but it does change where tokens are spent, and the documentation is precise about the accounting. Deferred tools add only their names and short descriptions to `total_input_tokens`. Parameter schemas are not billed until the model loads them, after which they are billed as input for the rest of the conversation. Enabling `tool_search` adds what the blog post calls a small, fixed block of input tokens; Google has not published its size.


The cost that is easiest to overlook is the search itself. According to the documentation, each server-side search makes an extra model call, and the prompt tokens of that call are reported in `total_tool_use_tokens` and billed at the input rate. Retrieved schemas are appended inline instead of being inserted at the top of the prompt, which keeps the cached prefix stable, so implicit caching continues to apply once the prefix reaches the model's minimum cacheable size. Google also warns that deferring tools can shrink a prompt below that minimum, and that static loading may cost less for small libraries or for long contexts that trigger a search on every turn. The blog post closes the pricing discussion by telling developers that savings depend on their traffic and should be measured.


For readers comparing these numbers with model prices, the Gemini API pricing page lists Gemini 4 Argon, one of the supported models, at $2.00 per 1M input tokens, $10.00 per 1M output tokens and $0.10 per 1M cached input tokens through November 30, 2026, rising to $4.00, $20.00 and $0.20 starting December 1, 2026. Batch prices are half the Standard rates in both periods. Argon supports implicit caching only.


........


Gemini 4 Argon, paid tier (USD per 1M tokens)

Through Nov 30, 2026

From Dec 1, 2026

Standard input

$2.00

$4.00

Standard output (incl. thinking)

$10.00

$20.00

Cached input (implicit caching only)

$0.10

$0.20

Batch input

$1.00

$2.00

Batch output

$5.00

$10.00

Batch cached input

$0.05

$0.10


........


Prices as listed on the official Gemini API pricing page on November 21, 2026.


··········


WHAT GEMINI API TOOL RETRIEVAL CAN SAVE: A WORKED COST EXAMPLE.


Because Google gives no benchmark figures, the clearest way to judge the feature is to model a workload and see which assumptions decide the outcome. The scenario below is illustrative and every input is an assumption, not a Google figure: a customer-support agent on Gemini 4 Argon with 120 tools whose schemas average 250 tokens each, so 30,000 tokens of tool definitions; a base prompt of 6,000 tokens per request, of which 2,000 tokens are new on each request; five core tools kept loaded; 25 tokens for each deferred name and short description; a 300-token `tool_search` block; two schemas loaded per conversation; and an extra search call whose prompt is assumed to be about the size of the deferred prompt itself, roughly 10,400 tokens. Output tokens are left out because the feature does not change them. The volume is 10,000 requests a day, or 300,000 a month.


With static loading, each request carries 36,000 input tokens. With tool retrieval, the main pass falls to about 10,900 tokens, and the assumed search call brings the total to about 21,350 tokens when the model searches on every request. The results below were calculated in Python at the Argon rates in the table above.


........


Argon input cost, 120-tool agent

Per 1,000 requests (Nov)

Per month (Nov)

Per 1,000 requests (from Dec 1)

Per month (from Dec 1)

Static loading, no cache hits

$72.00

$21,600

$144.00

$43,200

Tool retrieval, search every request, no cache hits

$42.70

$12,810

$85.40

$25,620

Static loading, stable prefix cached

$7.40

$2,220

$14.80

$4,440

Tool retrieval, cached, search every request

$26.69

$8,008

$53.39

$16,016

Tool retrieval, cached, search on 25% of requests

$11.05

$3,316

$22.11

$6,633


........


When the prompt prefix is not being cached, tool retrieval cuts input cost by about 41% in this scenario, and the saving grows with the library: the same arithmetic for 600 tools gives a 71% reduction, from 156,000 to about 45,350 input tokens per request. The picture reverses when implicit caching works well. A static block of tool definitions that sits unchanged at the top of the prompt is billed at the cached rate, which for Argon is 5% of the input price, while each search call is billed at the full input rate. In that case the deferred setup costs more than three times as much as static loading if it searches on every request, and the break-even point in this model is a search on roughly one request in thirteen. With 600 tools and a well-cached prefix, the two approaches come out roughly even when the agent searches on a quarter of its requests. The price change on December 1 doubles every figure but does not change which approach wins.


The practical reading is that tool retrieval pays off most for agents with very large or changing tool libraries, for workloads where caching is weak because prompts vary from the first tokens, and for agents that need a deferred tool only occasionally. Teams with a modest, stable catalogue and good cache hit rates should compare `total_tool_use_tokens` and cached-token counts from real traffic before switching, which is exactly the measurement Google recommends. Our troubleshooting guide on Gemini API implicit caching that reports zero cached tokens covers how to check whether a prefix is actually being cached.


··········


GEMINI API TOOL RETRIEVAL COMPARED WITH CLAUDE AND OPENAI TOOL SEARCH.


Google is the third major API provider to ship on-demand tool loading, and the three implementations share the same `defer_loading` vocabulary while differing in maturity, limits and documentation detail. Anthropic's tool search tool is listed as stable in the Claude Platform documentation and offers two server-side variants, one where Claude writes regular-expression patterns and one based on BM25 natural-language queries. OpenAI's tool search in the Responses API supports hosted and client-executed modes and, according to its documentation, works with `gpt-5.4` and later models.


........


Feature

Gemini API tool retrieval

Claude tool search tool

OpenAI tool search

Status

Preview (v1beta)

Stable

Available in Responses API

API surface

Interactions API only

Messages API

Responses API

Activation

`tool_search` + `defer_loading`

regex or BM25 tool + `defer_loading`

`tool_search` + `defer_loading`

Client-side search

Yes, `execution: "client"`

Yes, via `tool_reference` results

Yes, `execution: "client"`

MCP

Defer a whole `mcp_server`

Defer via `mcp_toolset` config

Defer on MCP server tool

Published tool limit

None stated

10,000 deferred tools per request

None stated

Separate fee

No

No

No pricing figures given

Models

Gemini 3.6 Flash and later, Gemini 4 family

Claude Haiku 4.5, Sonnet 4.5, Opus 4.5 and later

`gpt-5.4` and later


........


Anthropic's documentation goes further than Google's on limits and sizing. It caps a request at 10,000 deferred tools, returns up to five matching tools per search by default, and states that a typical multi-server setup spanning GitHub, Slack, Sentry, Grafana and Splunk can consume about 55,000 tokens in definitions, which tool search typically reduces by more than 85%. It also says tool selection accuracy degrades once more than 30 to 50 tools are loaded, and recommends tool search for libraries of ten or more tools or more than 10,000 tokens of definitions. OpenAI's guidance focuses on structure: it recommends grouping functions into namespaces with fewer than ten functions each, and notes that discovered tools are injected at the end of the context to preserve the cache. Google's preview has not yet published a tool limit, a size for the fixed `tool_search` block or accuracy figures, so developers moving from Claude or OpenAI should expect to establish those numbers themselves during the preview.


··········


WHAT CHANGES FOR DEVELOPERS BUILDING AGENTS ON THE GEMINI API.


For developers already on the Interactions API, adoption is a configuration change: add one `tool_search` entry, flag the long-tail functions with `defer_loading: true`, write a clear `short_description` for each deferred function, and keep the full schemas in every request. Google's best practices add three points that affect reliability more than cost. Tools whose names appear in system instructions or in other tools' outputs should not be deferred, because the model may try to call them before loading them. Similar tools should share a service prefix so that searches find them together. Agent loops should look for `function_call` steps instead of trusting the interaction status, since a failed search ends in plain text.


Developers still on `generateContent` cannot use tool retrieval without moving to the Interactions API, the same requirement that applies to other recent agent features covered in our report on Gemini API computer use reaching general availability. Because the feature is in preview, Google states that the API surface may change before general availability, which argues for keeping the deferred-tool configuration in one place in application code. For a broader view of how access tiers, free usage and limits work across Google's developer products, see our guide to Google AI Studio free plans, access tiers and usage limits, and for the underlying mechanics of function calling across models, our explainer on how function calling and tool use work in advanced AI models.


··········


GEMINI API TOOL RETRIEVAL: CONFIRMED DETAILS AND OPEN POINTS IN THE PREVIEW.


Google has documented the essentials clearly. Tool retrieval is a free preview feature of the Interactions API on six Gemini models, it supports server-side and client-side search and whole-server deferral for MCP, deferred tools cost only their name and short description until loaded, and each server-side search is an extra model call billed at the input rate and reported in `total_tool_use_tokens`. Loaded schemas cannot be unloaded, and the full tool list must travel with every request.


Three points remain open. Google has not stated how large the fixed `tool_search` block is, how many deferred tools a request can carry, or how much the feature improves tool selection accuracy, and it has published no benchmark. The worked example shows why the missing figures matter: the size of the search call and the share of requests that trigger a search decide whether the feature cuts an agent's input bill by 40% or more, or raises it compared with a well-cached static tool list. For large and changing tool catalogues the case for deferral is strong; for small, stable ones the answer has to come from production token counts.


··········


FOLLOW US FOR MORE.


·····


DATA STUDIOS


·····


datastudios.org


bottom of page