Why state beats context

The usual levers — shorter system prompts, smaller models for cheap turns, prompt caching, speculative decoding, distillation — are real wins. They scale linearly with calls. Halve the prompt and you halve the per-call bill.

That works fine for a single chatbot. It breaks the moment you have N agents coordinating, because the cost of coordination itself grows faster than the work you're doing.

When agents talk in natural language, every agent's full output becomes the next agent's input. With N agents that's an O(N²) blowup in tokens — and most of those tokens are restating what other agents already said.

You're paying the model to re-parse the same paragraphs over and over so it can extract three facts it actually needs.

The fix isn't smaller prompts — it's a different wire format. Agents commit typed packets (structured findings, not prose) to a shared state. A deterministic reducer merges them. On each turn an agent reads a compact, bounded view of state instead of the full transcript.

Result: coordination cost flattens from ~O(N²) to ~O(N). Same intelligence, far less to chew on. That's the optimization that actually moves the needle as agent count grows.

• You have 4+ agents touching shared work.
• Token bills are growing faster than throughput.
• You need auditability — who decided what, and why.
• Silent disagreements between agents are causing bugs.

If you're running one agent or two agents with a clean handoff, stick with the standard playbook. The wins from state-based coordination compound with scale.