Real Americans changed their minds about the president between January and April 2026, and the AI copies built to stand in for them did not. That gap sits at the centre of the case against synthetic polling, the growing practice of asking AI personas the questions you would normally put to real people.
The evidence comes from a large Pew Research Center experiment published on 30 September 2026. A Syracuse University expert’s warning, republished by Phys.org on 19 October 2026, has since brought it to a wider audience.
Key takeaways
- Pew had AI “digital twins” of its own panel members answer nearly 300 survey questions. Their answers missed the real ones by about 12 percentage points on average.
- Real presidential approval fell from 37% in January 2026 to 34% in April 2026. The AI personas gave 46% both times.
- Nearly half of all questions had at least one answer choice that no AI respondent picked.
- Swapping the model changed the picture: GPT-5.1 painted a more polarized public than the real one, and Claude Opus 4.6 painted a more moderate one.
What is synthetic polling, exactly?
A synthetic poll skips the phone calls. A company builds hundreds or thousands of AI personas on top of a large language model (LLM), the same kind of system behind chatbots. Each persona gets a demographic profile, sometimes past survey answers, and is asked to respond as that person would.
The appeal is obvious: it’s fast and cheap. A good national survey reaches roughly 1,200 people, while a synthetic firm can spin up several thousand personas in minutes. According to New Statesman reporting from 5 May 2026, startups such as Aaru, Artificial Societies and Electric Twin are selling simulated audiences, YouGov bought the synthetic-response firm Yabble in 2024, and Savanta has launched “Virtual Personas”.
How did Pew test AI personas against real voters?
Pew picked the fairest setup it could. It gave an AI model the demographics and 2025 political typology answers of real members of its American Trends Panel, then had each “twin” take the same surveys those people took in January, March and April 2026.
So this wasn’t a vague guess about “Americans”. Each persona copied one specific, known person. Pew says it tested several models and that Anthropic’s Claude Opus 4.6 performed best. Even so, the average miss was about 12 points, and on roughly 28% of questions the error passed 15 points.
Why don’t AI personas change their minds?
A persona runs on fixed instructions and a model whose knowledge stops at a training cutoff, which Pew puts at May 2025 for the model it used. Real people get new information every day. As Jennifer Stromer-Galley, a professor in Syracuse University’s School of Information Studies, told Phys.org: “These personas will give you consistent answers every month.”
Pew’s analysis of timely questions shows what that means. Human approval of the president slid from 47% at the start of his term in January 2025 to 34% by April 2026. The AI sample held at 46% in both replicated waves. The biggest real shift came among Republicans, and that is where the AI missed by as much as 19 points.
Think of it like a weather station that stopped updating last spring. It can describe the climate quite well. It just can’t tell you it’s raining.
Where synthetic polling missed most
The largest errors cluster around new topics and anything that happened after the model’s cutoff. Here are some of Pew’s figures from the 2026 waves:
| Question (Pew, 2026) | Real people | AI personas | Gap |
|---|---|---|---|
| Very concerned about electricity prices | 52% | 23% | -29 |
| Heard “a lot” about data centers | 25% | 3% | -22 |
| OK for immigration officers to cover faces | 38% | 17% | -21 |
| Approve of US strikes on Iran | 36% | 48% | +12 |
| Correct answer on the First Amendment | 52% | 98% | +46 |
The last row shows a quieter problem. The model knows the facts, so its personas do too. Real people pick “not sure” about four times as often as the AI did.
Why the outliers disappear
LLMs drift toward the most typical answer for a group, so persona panels flatten people into stereotypes. In Pew’s test, 97% of Hispanic personas said they were likely to follow the World Cup, against 43% of real Hispanic adults. Some 86% of Democratic personas called billionaires bad for the country, while only 45% of real Democrats said so.
On abortion, 11% of real adults said it should be illegal in all cases. Not one AI persona chose that answer. Stromer-Galley’s example is the rural Black voter in a small town whose views don’t match the national average for either group. A persona matched to the national average simply erases that person. She calls synthetic data a “distorted mirror of us that potentially creates misunderstandings that get magnified.”
The model you pick matters too. GPT-5.1 had 100% of personas dissatisfied with the country’s direction, while 69% of real people were. Opus pushed answers toward the middle of the scale instead. Both pictures were wrong, just in opposite directions. We saw a similar pattern when cheap AI models misread the night sky: confident output is not the same thing as accurate output.
What happens next
Pew’s verdict is blunt: AI models “are not an adequate replacement for traditional polling on topics of broad public importance.” Pew says it has no plans to use AI to generate its survey results, though it does see a role for AI in tasks like coding open-ended answers.
Synthetic firms aren’t going away. The likely outcome is a split market: personas for cheap early work like testing ad copy or drafting questions, and real samples for anything published as “what the public thinks”. The thing to watch is labelling. A simulated statistic presented as real is the same trust problem we covered in fake users and inflated metrics.
My view, clearly opinion: synthetic polling works best as a way to generate hypotheses, not as a measurement tool. It can tell you what a 2025 snapshot of the internet thinks a voter would say. It can’t tell you what the voter says now.
FAQ
Is synthetic polling accurate?
Not reliably. In Pew’s 2026 test of nearly 300 questions, AI personas missed real answers by about 12 percentage points on average, and some misses went past 40 points.
Why can’t AI personas track changing opinions?
They run on fixed prompts and a model with a training cutoff, so they don’t react to news. Pew found they were especially weak on events after the model’s May 2025 cutoff.
Does the AI model used make a difference?
Yes. With the same setup, GPT-5.1 made the public look more polarized and Claude Opus 4.6 made it look more moderate. Neither matched reality.
Next time a poll lands in your feed, look for who was actually asked. If the answer is “personas”, treat the numbers as a guess, not a measurement.
Leave a comment