Our agent had thirteen tools and bound all of them every turn. Their schemas came to 5,268 characters of the context window, on every single turn, whether or not any of them was relevant.
So we stopped binding them all. Five starter tools stay bound; the rest live in a catalog and appear as a one-line index — name plus a short description. If the model calls one by name, a reachability fallback finds it, binds it, and runs it. Input tokens dropped from 11,209 to 7,636, about a third.
Two days later a user asked how to write a business plan.
We have a curated BP guide behind a tool. Nine chapters, opinionated: decide your reader before you write; a solution should be ten times better, not ten percent; “we have no competitors” means the market doesn’t exist; three paying customers beat thirty pages. It was written by someone who has read a lot of bad business plans.
The model didn’t call it. It answered from general knowledge — a clean ten-module outline, project overview through appendix. Perfectly reasonable. Completely generic. I diffed the reply against our guide’s ten most distinctive phrases and got zero matches.
The failure had no symptom
This is the part worth dwelling on. A tool the model cannot reach usually announces itself: the user asks for something and gets I can’t do that. That failure is visible, annoying, and reported immediately.
Here the model produced a competent answer to the question asked. Nobody would file a bug. The only evidence that anything was wrong lived in a comparison nobody would think to run — between the reply we shipped and the document we’d paid someone to write.
A capability that degrades into a plausible answer is worse than one that degrades into an error, because only one of them gets reported.
I found it because that morning I had added a per-turn record of which tools were called and how each one was reached. I did not add it to catch this. I added it to answer a colleague’s question about whether the disclosure mechanism was working at all. The instrumentation had existed for one day before it paid for itself.
What the model was actually shown
The index line was built like this:
description.splitlines()[0][:40]First line of the docstring, truncated to forty characters. Three things wrong with that, and I’d shipped all three.
A source line is not a sentence. Docstrings wrap at 79 columns for human readability, so the first line is usually half a thought. Five of our nine index entries ended mid-clause — one broke after “when the user says they don’t want to continue” and dropped the word “call” that started the next line.
Truncating at forty characters cuts mid-word. Our chart tool’s first line was 51 characters, so the model saw kind can only be bar/line/area/pie/don. It was being shown a corrupted enum value and expected to work with it.
And the entry for the guide tool said nothing. Its first line — grammatical, complete, under the limit — read: retrieve the full text of a built-in guide; call it when the user asks about the corresponding topic. Which topics? Listed on the next four lines, cut off. The model could see a tool for “topics” and had no way to know business plans were one.
None of this is subtle in hindsight. All of it was invisible in practice, because nothing renders the index for a human to read. I’d written the compression and never once looked at its output.
The vocabulary was already there
The part that stings: our codebase already contained a hand-maintained trigger vocabulary for every tool. The guide tool’s entry included 商业计划书 and BP — business plan and BP, verbatim, the exact words the user typed.
Those words were registered for a keyword-scoring function used by a search tool. They had never been shown to the model.
So there were two descriptions of every tool — one derived incidentally from a docstring written for a different audience, one hand-curated for exactly this purpose — and the index used the wrong one. Not because anyone decided that. Because the index was never designed; it was derived, and derived things inherit whatever the source happened to be for.
The fix was to stop deriving and start composing. An entry is now the name, its required parameters, its trigger vocabulary, and whole sentences packed to a budget:
- view_startup_guide(kind: self_register|tax_filing|business_plan)
【BP / 商业计划书 / 自己报税 / 自己注册 / …】：取内置长指南全文，据全文作答。The index grew from 562 characters to about 1,000. The schemas it replaces are 5,268. Spending a few hundred extra characters so that the model knows what it has is an easy trade. Tokens saved by hiding a capability from the model are not saved at all; they get spent somewhere worse.
The second bug was structural
With the index fixed, the model found the tool. Then the transcript showed this:
[AI] view_startup_guide args={}
[TOOL] missing 1 required positional argument: 'kind'
[AI] view_startup_guide args={"kind": "self_register"}
[TOOL] {"guide": "..."}Called it twice. The first call was empty.
This one isn’t a mistake in my implementation — it’s a property of the design. Reachability binds the tool after the call arrives. So the first call to any unbound tool is composed without its schema, from the index line alone. If the index doesn’t state the parameters, the model guesses, and the guess is “no arguments.”
Every unbound tool with a required argument wastes one round the first time it is reached. This happens in every conversation, for every such tool, not once overall.
The fix follows from naming it: required parameters go in the index. Which surfaced a smaller thing worth stating on its own. Our closed sets were documented in prose — kind can only be bar/line/area/pie/… — and prose is not machine-readable. To put the enum in the index I had to declare it as a type. A Literal[...] instead of a str.
A closed set written in a docstring is a comment. The same set written in the type is an interface. Everything downstream can only read the second one: the index, the validation layer, and the schema the model eventually sees.
What I’d take from this
Anything that decides what the model can see is a prompt surface. I’d been treating the tool index as plumbing — a serialization detail between the catalog and the context window. It’s a prompt. It gets the same review as a prompt: read the rendered output, ask whether a reader could act on it, notice when it says “the corresponding topic” and never names the topics.
Compression is where information disappears without a signal. The whole mechanism is a compression: 5,268 characters down to 562. I chose the budget and never inspected what survived it. Truncation defects do not raise errors. They produce shorter text that still looks like text.
A derived default inherits the wrong audience. A docstring is written for whoever reads the full schema, and that person has already decided to call the tool. The index is read by a model that is still deciding whether to call it. Different question, different text. Reaching for the docstring was convenient, and convenience is how a wrong source becomes the default.
Instrument the mechanism, not just the outcome. The tool trace that caught this cost half an hour and records one string per turn: which tools ran and how each was reached. Latency and error rates would never have shown this — nothing was slow and nothing failed. The only observable was a question the model chose not to ask.