BUSINESSDevelopments · · Source: Anthropic Research
Anthropic study: AI agents sway experts without them noticing
Anthropic published an experiment with 50 data analysts on October 27, 2026: they adopted their AI agent's randomly assigned opinion on 86% of decisions. Only 15% mentioned the AI when they explained their reasoning afterwards.
How the experiment worked
Anthropic recruited 61 professionals, and 50 worked with the agent – Claude Opus 4.8 running in Claude Code. The task took about an hour: analyze data from a fictional farming cooperative and advise whether to expand or end an advisory program. It involved four methodological decisions. For each one, a hidden coin flip set which side the agent favored. The agent wasn't allowed to lie or mislead; it simply argued its side openly.
The results (Anthropic's figures)
- 86% of methodological choices matched the agent's lean, against roughly 50% by chance.
- 60% switched sides when they already had a preference and the agent disagreed, compared with 6% when it agreed.
- 77% reused the agent's arguments later, but only 15% credited it.
- 52% felt the recommendation was more their own than the agent's.
- Experience and education offered no protection.
The effect on the final recommendation was weaker: 54% versus 36%, which The Decoder reports was not statistically significant.
The caveats
The leans were deliberately strengthened, so the everyday effect is likely smaller. The study covered one task and one model, and it comes from the vendor itself. The AI helped too: 6% of analyses with the agent had serious methodological flaws, against 55% without it – though that comparison group was just 11 people, and less experienced.
What Anthropic recommends
Agents should make the strongest case for each option or ask for the person's view first. Organizations should decide which calls need independent human judgment and protect the time to make them.
What this means for you
- Write down your own view before you ask the AI. That way you'll notice when it changes your mind.
- For important decisions, ask for both sides rather than one recommendation – for example: "Give me the strongest arguments against your suggestion."
- Agree as a team which decisions a person reviews independently. According to the study, experience alone isn't enough protection.
Luigi's take
The number that strikes me here isn't 86% but 15%. Most participants took on the AI's arguments without mentioning it. I know that pattern from more than 30 years in marketing: whoever puts a well-worded position on the table first shapes the discussion. An AI agent does that in every session, patiently and persuasively.
That's no reason to avoid AI – the work done with the agent was methodologically cleaner. My advice is simple and free: for decisions that matter, write down two sentences of your own first, then ask the AI. And tell it to disagree with you. Trust is fine; your own judgment is better.
Source
Anthropic Research: How AI agents shape experts' judgment calls
More sources
- heise online – Bericht zur Studie
- The Decoder – Einordnung der Studiendetails
This article was researched and written with AI assistance. Responsible: Luigi Lugmayr.
