A synthetic respondent is a language model told to answer your questionnaire as if it were a person: a 35-year-old nurse in Ohio, a retired engineer in Leeds, a college gamer in Austin. Run it a thousand times with a thousand descriptions and you have an “AI panel.” It never speeds, never claims to drive a nuclear submarine and costs almost nothing.
What can such a panel tell you? It is useful for drafting and pretesting questions. It is worthless as a measurement of what real people think. And when it is sold without saying so, it is dishonest, because the client believes he is buying people.
I gave synthetic respondents one section in You killed market research. Since then the evidence has piled up and the industry’s codes have taken sides.
How do synthetic respondents work?
A large language model is trained on an enormous amount of text to predict which word comes next. It is then tuned, with human feedback, to give answers people rate as helpful. When you write “You are a 52-year-old Republican farmer from Iowa. How warm do you feel toward labor unions, from 0 to 100?”, the model asks nobody. It produces the number that text by and about people like that makes most plausible.
That is the whole trick. The model knows what has been written, on average, before its training ended. It knows the stereotype of a group better than any member of the group does. It knows nothing about last month unless somebody wrote about it in time.
Two details matter to anyone paying for the output. First, the spread of answers is partly a setting. Models have a “temperature” parameter that controls how random the output is, and political scientists led by James Bisbee found a strong positive association between that setting and the variance of their synthetic data. In a real survey, how much people disagree is a finding; in a synthetic one, it is a dial.
Second, newer “digital twins” feed the model a real person’s earlier answers instead of a two-line description. I come to them below.
What did the silicon sample studies find?
The optimistic case was made in Political Analysis in 2023 by a team at Brigham Young University, who coined the term “silicon sampling”. They gave a model thousands of backstories taken from real respondents to the American National Election Studies and asked how each would vote. The correlation with the real votes was 0.90 to 0.94 across the 2012, 2016 and 2020 elections.
But vote choice is about the easiest thing in American politics to guess from someone’s party, race and religion. For pure independents in 2020, the people whose vote you can’t read off their demographics, the correlation was 0.02. The model was excellent at predicting the predictable and close to useless exactly where a survey earns its fee.
The skeptical test came a year later from the same Bisbee team, and the detail beneath the headline numbers I quoted last time is worse. Where the synthetic regression coefficients differed significantly from the human ones, the sign flipped 32% of the time: the opposite conclusion. Synthetic Democrats liked liberals more and conservatives less than real Democrats did, by 10 to 20 points on a 100-point scale. In the authors’ words, the synthetic responses “would suggest that society is more politically hostile than it actually is.”
Because the synthetic answers were bunched too tightly, the authors also warned against using them for power analysis, the calculation that tells you how many real people to interview. Their closing worry is the line I would print on every synthetic panel invoice: such data risk “hard-wiring the past into the present.”
Why do AI panels flatten and stereotype people?
The problem isn’t one model or one year. At ICML 2023 a Stanford team presented OpinionQA, which compares language models’ opinions with those of 60 US demographic groups. The misalignment was “on par with the Democrat-Republican divide on climate change,” and it persisted when the models were told which group to speak for. Among the worst-represented were people aged 65 and over and widowed people.
In Nature Machine Intelligence in 2025, Angelina Wang, Jamie Morgenstern and John Dickerson compared four models with 3,200 real people across 16 identities. The models often answered the way outsiders imagine a group, not the way its members describe themselves, which is what you’d expect from a machine trained on text that rarely says who wrote it. Their answers were also less varied than the humans’.
Digital twins were supposed to fix this by modeling individuals instead of types. In September 2026 a Columbia-led team published a large test in Science Advances: 19 preregistered studies comparing real panelists with twins built from each person’s answers to more than 500 questions. In the preprint, the average correlation between twin and human was about 0.2, only slightly better than a generic persona.
As Science News reported, the twins were more uniform than the people they copied, leaned on demographic stereotypes, trusted others more, worried less about technology and behaved more rationally. Ask that panel about your new app and it will like it.
Can AI replace a poll? Pew’s 2026 answer
The most thorough test so far comes from the organization least likely to be dazzled. In September 2026 Pew Research Center ran a digital-twin copy of its American Trends Panel through three surveys fielded between January and April, nearly 300 questions in all, and compared the answers with the real panelists’. The report’s title is Can AI Stand In for Human Survey-Takers? Not Really.
On average the synthetic estimates missed the human results by 12 percentage points, and on about 28% of questions the gap exceeded 15 points. The errors had a shape.
The model estimated that 97% of Hispanic adults were at least somewhat likely to follow the World Cup; the real figure was 43%. It put the share saying abortion should be illegal in all cases at 0%, against 11% in reality. Nearly half of the questions had at least one answer option that no synthetic respondent chose. The fringe, the undecided and the merely unusual were edited out.
The synthetic Americans were also suspiciously well informed. On a question about the First Amendment, 98% of them answered correctly, against 52% of real adults, and real panelists were about four times as likely to say “not sure.”
On new topics the model was simply behind: asked about awareness of data centers, it estimated 3% where the real answer was 25%. When Pew swapped the model, the error changed direction: one model made public opinion more extreme than it is, the other more moderate. Pew says it has “no current or future plans to use AI models to generate survey results.”
Why the same question gets a different answer next quarter
A tracker is only worth something if the instrument stays still while the market moves. Synthetic panels reverse that. In 2023 researchers at Stanford and Berkeley compared two releases of the same commercial model three months apart and found, among other changes, that the newer one was less willing to answer opinion survey questions at all. A synthetic panel built on it would have changed its behavior without a single respondent changing their mind.
Models also disappear. The model behind the original silicon sampling paper was retired by its maker on January 4, 2024, so that study can no longer be rerun on the instrument that produced it.
Pew’s swap shows the same problem at scale. If changing the model changes the direction of the error, a “trend” in synthetic data may be nothing more than a software upgrade.
The respondent who knows what you want
Real respondents misread questions, get bored and guess. Synthetic ones have the opposite fault: they are too good. A study presented at ICML 2023 found a “hyper-accuracy distortion” in some newer models: asked to simulate a crowd guessing at general-knowledge questions, they made it far more accurate than any real crowd. Pew’s First Amendment result is the same fault in a poll.
Nor do they fall for what people fall for. A 2024 study in Transactions of the Association for Computational Linguistics tested nine models against the known effects of wording, order and answer scales on human answers. They generally failed to reproduce them, especially models tuned with human feedback, while reacting to wording changes that leave people unmoved. A synthetic pretest can’t tell you how a leading question will lean real people.
Then there is the hypothesis. A model reads the whole questionnaire, concept board and all, and works out what the study is about. Sean Westwood’s 2025 study in PNAS showed that AI agents can infer a researcher’s hypothesis and produce data that confirm it, even without any malicious intent. A model tuned to be helpful is a respondent who has read the brief.
The best market research evidence I know points the same way. In a Marketing Science Institute working paper, James Brand, Ayelet Israeli and Donald Ngwe found model-based estimates of willingness to pay realistic and comparable with human studies. Fine-tuning on earlier human surveys improved the answers for existing and new features, but not for new product categories or for differences between customer segments. The model works best where you have already asked people and fails where the decision is new, which is backwards for a product sold as a replacement for asking people.
Where synthetic respondents genuinely help
None of this makes the technology useless. It makes it a drafting tool. A 2024 review in Psychology & Marketing by Marko Sarstedt and colleagues found that results varied considerably from one domain to another and saw the most promise upstream, in qualitative pretesting and pilot studies. My own list is similar:
- Drafting. A first version of the questionnaire, answer lists that don’t miss obvious options, plain-English rewrites of jargon.
- Criticism. Ask it to find double-barreled, leading or ambiguous questions. It is a tireless, slightly pedantic reviewer.
- Plumbing. Push synthetic cases through the survey to test routing, then use the fake data to build tables and scripts before fieldwork.
- Hypotheses. Generate explanations worth testing with real people, as you might with a well-read colleague who has never met a customer.
What these uses share is that the model criticizes the questionnaire instead of answering it. The moment its answers become numbers on a chart, it has stopped being a tool and started being a respondent, and as a respondent it doesn’t exist.
What do industry codes say about synthetic data?
In May 2024 the UK’s Market Research Society published a Delphi Group report on using synthetic participants, and its chief executive, Jane Frost, presented it as a way to make synthetic data “another weapon in our arsenal.” Less than a year later another MRS body, the Advanced Insights and Analytics Council, called synthetic data “currently over-hyped” and pointed out that it looks backward, so it can’t show emerging trends.
The 2025 ICC/ESOMAR International Code is specific. Article 7 says the client must be told when AI is used to compile, analyze, report or interpret data, explicitly including “synthetic data and synthetic personas,” and that “the extent of human oversight must be stated.” Article 9 extends the duty to the public: published findings must disclose whether AI or synthetic data played a significant role.
The American Association for Public Opinion Research went further in its code as amended in June 2026. AI-generated cases “are not research ‘participants,’” and the words poll, survey and their cousins “should not be used to describe data created through artificial intelligence.” I think that is exactly right. A product described as a panel of 1,000 consumers, when it is one model run 1,000 times, misdescribes what it sells.
My take
I don’t object to synthetic respondents. I object to the word “respondent.” Nothing responded.
A model produced the answer its training text made most plausible for a type of person, and a type is not a person. That is a fine thing to have while you write a questionnaire, and a terrible thing to put in a deck under the heading “What customers think.”
The appeal is easy to see. Real people are expensive and harder to reach every year, and the cheap ones are often not who they claim to be. An industry that pays its human respondents very little for their time is bound to be tempted by respondents who cost nothing. But the cure for bad human sample is better human sample, not a sample with no humans in it.
So here is my rule: use the model to write the questionnaire, argue with it and test it, and then ask people. If anything synthetic reaches the client, label it on the slide where the number appears, not in an appendix. And if a supplier won’t tell you which parts of your data were generated, assume all of them were.
Questions people ask
What are synthetic respondents?
Language models told to answer a questionnaire as a described person, such as a 35-year-old nurse in Ohio. They produce the answer such a person would plausibly give, based on text the model was trained on. No person with an opinion is asked.
Can AI replace survey respondents?
Not for measurement. In Pew Research Center’s 2026 test of nearly 300 questions, AI answers missed the real results by 12 percentage points on average, and switching the model changed the direction of the error.
Do you have to disclose synthetic data in market research?
Yes. The 2025 ICC/ESOMAR Code requires researchers to tell clients when synthetic data or synthetic personas are used and how much human oversight there was. AAPOR’s 2026 code says AI-generated cases are not research participants and must be labeled.
What are synthetic respondents good for?
Drafting and criticizing a questionnaire, generating hypotheses, testing survey routing and producing dummy data for analysis scripts. They should not size markets, describe subgroups or set the sample size for a real study.