Synthetic Users, Explained

What are synthetic users?

A synthetic user (also called a synthetic respondent, synthetic participant, or AI persona) is a research participant simulated by a large language model. Instead of recruiting a human who matches your target profile, you specify the profile ("a 42-year-old plant safety manager at a mid-size food manufacturer, budget-constrained, burned by two failed software rollouts") and the model role-plays that person consistently across an interview, a survey, or a group discussion.

The term covers a spectrum of rigor. At the shallow end, it's one prompt asking a chatbot to "act like a customer." At the deep end, it's structured panels where each persona has stable goals, constraints, decision criteria, and objections, participating in multi-round facilitated methodologies with the output distilled into labeled, directional reports. The difference in usefulness between those two ends is enormous.

How synthetic users actually work

1. Panel definition. You describe the audience; the system drafts distinct persona profiles spanning it, not twelve copies of the average.
2. Grounding. Each persona gets texture that constrains its behavior: a role, a context, things it wants, things it fears, and reasons to say no.
3. Methodology. The personas are run through a structured exercise: a moderated multi-round discussion, a one-on-one depth interview, a comparison task.
4. Synthesis. The raw transcript is analyzed into themes, sentiment, tallies, and recommendations, with the verbatims available underneath.

What the research says

The academic record is genuinely mixed, and anyone quoting only one side of it is selling something. In support: Argyle and colleagues' "silicon sampling" work found that properly conditioned language models reproduce the response distributions of human survey populations with surprising fidelity, and Stanford's generative-agent studies showed LLM agents producing believable, internally consistent social behavior. Industry replications have repeatedly reported synthetic panels reaching the same directional conclusions as human panels on message and concept comparisons, though results vary by domain.

On the critical side: studies have documented flattened variance (synthetic panels are more agreeable and less weird than real humans), demographic caricature (the model plays the stereotype of a group more readily than its diversity), sycophancy toward the researcher's framing, and a systematic miss on embodied or sensory experience. Nielsen Norman Group's caution stands: treat synthetic users as a hypothesis machine, not a user-evidence machine.

When synthetic users are the right tool

• Comparing options. Relative judgments ("which of these four headlines is strongest for this audience?") are robust because the biases mostly apply to all variants equally.
• Finding flaws early. Confusion, objections, and credibility gaps surface reliably. A claim that synthetic skeptics don't believe will struggle with real skeptics too.
• Piloting research instruments. Running a discussion guide past synthetic participants before real sessions catches leading questions and dead ends for free.
• Reaching the unrecruitable. Churned customers, competitors' buyers, niche B2B roles: synthetic stand-ins beat nothing, which is the realistic alternative.
• Pre-work for real research. Sharpening hypotheses and shortlisting stimuli so real-participant studies are spent on questions that survived the cheap round.

When they're the wrong tool

• Demand measurement. "Would you pay $49?" from a synthetic panel is not willingness-to-pay data. It's a prompt for finding pricing objections.
• Anything requiring real incidence. Market sizing, prevalence claims, and statistically representative results need real sampling.
• Usability of real interfaces. Synthetic users don't have fingers, eyes, or your production environment. Watch real humans use the thing.
• Lived-experience research. Health, identity, trauma, culture: simulating these for decisions that affect real communities is both unreliable and wrong.
• Anywhere the label gets dropped. The moment synthetic findings get laundered into 'users said,' the method is being abused.

What separates good synthetic users from bad ones