Search for a command to run...
Here is a number that explains a lot of what is hard about building a tool-use agent over real data. When our dosage tool runs, the full result it returns to the interface — every study, every dose, every unit breakdown — is about 282 kilobytes. The version of that same result that the model actually needs to reason about is about 0.6 kilobytes. That is a 99.8 percent reduction, and the model's answers got better when we stopped pretending those two things were the same.
This is a post about the pattern that made that work, and about the subtle way it almost did not work at all.
A retrieval tool in an agent has two audiences, and they want opposite things.
The model wants the gist. To decide what to say about a supplement's dosage, the model needs the median dose, the range, maybe a couple of representative values. It does not need a hundred and forty rows of per-study dosages. Dump all of those into its context and three things happen: it gets slower, it gets more expensive, and — counterintuitively — it gets worse at reasoning, because the signal is buried under the noise of raw rows it was never going to read anyway.
The interface wants everything. The user looking at the dosage chart wants to see the full dot plot, every study point, the percentiles, the unit breakdown. Hand the interface only the summary and you have thrown away the most useful part of the tool.
So a good retrieval tool has to return two payloads. A rich one for rendering, and a lean one for reasoning. The mistake is to return only one.
Every retrieval tool in our agent returns an object with two nested payloads. There is the full output — everything, for the interface. And there is a lean payload — the aggregates and a few representative rows, for the model. The tool computes both in one pass, because they come from the same data.
The magic is in how they are routed. The interface stream takes the full output and ignores the lean one entirely. The model context takes the lean one and never sees the full one. A hook, called when the framework converts tool results into model input, performs the substitution: it reaches into the result, pulls out the lean payload, and hands that to the model in place of the whole.
This sounds simple, and the idea is. The implementation is where the traps are.
We shipped the split, checked the logs, and saw exactly what we wanted — a 99.8 percent reduction in what the model received. Then a later request came back oversized anyway. Same tool, same data, suddenly the model was getting the full 282 kilobytes again. The split was working, and then it wasn't.
The cause was a second path the tool result could take into the model. When a user asks a follow-up question, the previous tool results are already sitting in the conversation history. To send that history back to the model, the framework has to convert it — and to run the substitution hook, it has to know which tools are registered. In the follow-up path, we had not told it. No tool registry supplied, no hook run, and the framework did the safe thing: it serialized the entire stored result, full output and all, back into the model's context.
The split had worked the first time because the hook ran. It failed on follow-ups because the hook was silently skipped. The number we logged during execution was honest. The number the model actually received on the next turn was not. They were two different measurements of two different things, and we had been reading the good one.
The fix was a line: build the tool registry once, and pass it both when running the model and when converting the history. Once the hook ran on every path, the lean payload reached the model on the first turn and every turn after.
There is a general lesson here that cost us time to learn.
When you add a transformation to a pipeline — compress this, summarize that, substitute the lean version for the full one — you have to ask not just whether it works, but on which paths it runs. A tool result can reach the model through more than one door: the first call, a retry, a follow-up, a streaming resume. If your optimization only runs on the front door, the back doors will quietly undo it, and your logs will show the front-door numbers and look clean.
The pattern that finally held was: one registry, every door, same hook. The lean payload is not an optimization we apply sometimes. It is the contract under which the model receives tool results, applied uniformly, with a log line on every path so that the next back door is visible before it costs us a week.
The numbers, once it held, were worth it. The dosage tool: 282 kilobytes down to 0.6. A consensus tool, on a synthetic hundred-and-twenty-claim pair: forty kilobytes down to one and a third, keeping every aggregate and three representative direction-balanced claims while the full claim list stayed in the interface. The model reasons over the shape of the evidence. The user sees every study that shape was built from. Neither audience is served at the other's expense.
That is the real point of the split. It is not really about kilobytes or cost, though those are nice. It is about giving each audience the representation it can actually think in. The model gets a summary dense enough to reason about. The interface gets the raw records dense enough to trust. The agent between them just has to be careful never to swap the envelopes.
