Our agent has thirteen tools. It used to bind all thirteen on every turn. The full JSON schemas for those tools take 5,268 characters. The agent paid that cost on every turn, including turns where the user only asked a question.
We changed the design. Input tokens per turn dropped from 11,209 to 7,636, a reduction of about one third. This article describes the design, because the obvious version of it has a serious flaw.
Two versions of progressive disclosure
Progressive disclosure means the model does not receive every tool definition up front. There are two ways to build it, and they differ in one important respect.
The search-based version. The model is bound to a small set of tools. One of those tools is a search tool. If the model needs a capability it does not have, it calls the search tool with a query, receives matching tool names, and those tools become available on the next round.
The index-based version. The model is bound to a small set of tools. In addition, the model always sees a compact index of every unbound tool: the name, plus one line. If the model wants one of them, the model calls it directly by name. The runtime finds the tool, binds it, and runs it.
The difference is what the model knows. In the search-based version, the model cannot see the unbound tools. It has to suspect that a capability might exist, and then guess a query that finds it.
That is the flaw. A model that does not know a capability exists will not search for it. It will do one of two things instead. It will tell the user that it cannot help. Or, worse, it will answer from its own general knowledge and produce something that looks fine.
We hit the second case in production. A user asked how to write a business plan. We have a curated guide behind a tool. The model did not call the tool. It wrote a generic ten-section outline from its own knowledge. Nobody would have reported that as a bug. I wrote about that incident separately in The index is a prompt.
So we use the index-based version. The cost is small. For our nine unbound tools, the full schemas are 3,688 characters. The index is about 1,000. We pay 1,000 characters so that the model always knows what it has.
The parts
| Part | What it does | Who owns it |
|---|---|---|
| Starter set | The tools that stay bound every turn | The agent (domain judgment) |
| Catalog | Every tool, bound or not | The framework |
| Index | One line per unbound tool, injected into the system prompt | The framework |
| Reachability fallback | Model calls an unbound tool by name, runtime binds it and runs it | The framework |
| Search tool | A second path for finding tools by keyword | The framework |
| Tool trace | Per-turn record of which tools ran and how each was reached | The framework |
The starter set is the only part that requires domain knowledge. The rule we use is: a tool belongs in the starter set if the user’s very first sentence might need it. For our registration assistant that means five tools, including “start a registration” and “check order progress”. Everything else lives in the catalog.
The starter set can also grow during a turn. When the user is in the middle of filling a form, we add the form-related tools to the starter set, because those turns almost always need them. When the user explicitly asks for a chart, we add the chart tool. In both cases the alternative is a wasted round trip, and a round trip costs more than the schema.
What an index entry must contain
This is where we got it wrong the first time, so it is worth stating precisely. An index entry has three required parts.
- view_startup_guide(kind: self_register|tax_filing|business_plan)
【BP / 商业计划书 / 自己报税 / 自己注册】：取内置长指南全文，据全文作答。The name. Obvious, and not sufficient.
The required parameters, including closed enum values. This part is not optional, and the reason is structural. The reachability fallback binds a tool after the call arrives. Therefore the model composes its first call to any unbound tool without the schema, using only the index line. If the index line does not state the parameters, the model guesses. In our case the model guessed “no arguments”:
[AI] view_startup_guide args={}
[TOOL] missing 1 required positional argument: 'kind'
[AI] view_startup_guide args={"kind": "self_register"}
[TOOL] {"guide": "..."}The tool self-corrects on the second call, because by then it is bound and the model can see the schema. But every unbound tool with a required argument wastes one round the first time it is reached. That happens in every conversation, not once.
Putting the enum values in the index requires declaring them as types. A closed set described in a docstring is a comment, and no tooling can read a comment. The same set declared as Literal["self_register", "tax_filing", "business_plan"] appears in the JSON schema, and the index can extract it.
The trigger vocabulary. The index line answers a different question than the docstring does. A docstring is read by someone who has already decided to call the tool; it explains what the tool does. An index line is read by a model that is deciding whether to call the tool; it needs to say when the tool applies.
Our first version derived the index line from the docstring, and it produced entries like this one: “retrieve the full text of a built-in guide; call it when the user asks about the corresponding topic.” Which topics? The docstring listed them on the following lines. The index took only the first line and dropped the rest.
We already had the right vocabulary elsewhere in the codebase. Each tool had a hand-maintained set of trigger words, registered for a keyword-scoring function used by the search tool. The guide tool’s set already contained 商业计划书 and BP, the exact words the user typed. Those words had never been shown to the model.
Now the index uses that same set. There is one vocabulary, and both the search tool and the index read it. Two lists would drift.
One detail: the trigger words are stored in a set, and set iteration order is not stable across processes. Rendering them directly would produce a slightly different system prompt on every process start, which defeats prompt caching. We sort them before rendering. Shorter words come first, because shorter words are closer to what a user actually types.
Extracting the one-line description
The description half of the entry comes from the docstring. Our first implementation was one line of code:
description.splitlines()[0][:40]It was wrong in two ways. Docstrings wrap at 79 columns for readability, so the first line is often half a sentence. And a hard cut at 40 characters lands mid-word; one of our entries showed the model bar/line/area/pie/don, a truncated enum value.
The replacement joins the wrapped lines back into text, splits on sentence terminators, and packs whole sentences until it reaches a budget. If even the first sentence exceeds the budget, the function logs a warning, because that is a defect in the tool’s docstring rather than something the index should paper over.
A test asserts that no index entry for our agent ends mid-sentence, and that every tool in the index has registered trigger words. A tool with no trigger words appears in the index as a bare function name, which in practice means the model will not reach for it.
Observability
Progressive disclosure fails silently, so it needs instrumentation built on purpose. We record one line per turn: which tools ran, and how each one was reached.
| Origin | Meaning |
|---|---|
bound | The tool was in the starter set |
catalog | The tool was unbound; the model called it by name and the fallback found it |
search | The search tool returned it, and it was bound for the next round |
unknown | The tool is not in the catalog at all; the model invented the name |
A turn that contains catalog is direct evidence that index-based disclosure worked. That is the whole point of the mechanism, and before we recorded it we had no way to confirm it.
The distinction between catalog and unknown matters more than it looks. Both are cases where the model called a tool that was not bound. In one case the model read the index correctly. In the other case the model made up a name, which usually means the index entry for the real tool is unclear. If the trace collapsed these two into one value, the metric would count hallucinations as successes.
We write this trace to the same table that records per-turn latency and model name. Our message bus does not record tool calls for external runtimes, so the agent has to record its own.
Results
| Measure | Before | After |
|---|---|---|
| Tool definitions in context | 5,268 characters | ~1,000 characters (index) + 1,980 (starter schemas) |
| Input tokens per turn | 11,209 | 7,636 |
| First call to an unbound tool | Failed, then retried | Succeeds |
The token reduction is about one third. It is worth being clear about what produced the rest of our cost reduction: we also moved from a large model tier to a small one, and that change was larger than this one. Progressive disclosure and model selection are separate decisions.
When not to do this
Two cases where we bind a tool directly instead.
The tool is a core capability. Our knowledge base search tool stays bound. We tested it in the catalog, and the model consistently answered policy questions from general knowledge rather than searching. For a tool the product depends on, one extra hop is enough to lose it.
The user has already asked for it. When the user says “draw me a chart”, the chart tool joins the starter set for that turn. Making the model discover a tool it was just explicitly asked to use is a waste of a round trip.
The general rule behind both cases: the index saves tokens, and a wasted round trip costs more tokens than the schema you saved. Only hide a tool when the probability of needing it in a given turn is genuinely low.