From Silicon Sampling to Population Steering: When Does Survey-Grounded Persona Generation Beat Demographic Prompting?

1. Introduction

Argyle et al. demonstrated that a language model conditioned on socio-demographic backstories drawn from real survey respondents emits response distributions that track the corresponding human subpopulations, a property they call algorithmic fidelity [1]. The result seeded a fast-growing practice of silicon sampling. It also left a design question open. A demographic backstory is a one-line sketch; a person is not. If a persona were instead grounded, with its personality, moral foundations, values, ideology, and religiosity sampled jointly from the empirical distribution of real respondents conditioned on those same demographics, would the silicon sample get closer to the humans?

This paper answers the question twice, at two different levels of analysis, with opposite verdicts.

Study 1 (individual level): grounding fails, and must fail. In a preregistered, within-respondent, same-model comparison on ANES 2020, personas grounded by a multi-survey latent factor model lose to plain demographic backstories at predicting individual vote choice and party identification. In the production configuration the losses are decisive on the well-calibrated models; at the grounded arm's strongest configuration most gaps contract inside the run's precision bound, and grounding still never decisively beats a well-calibrated baseline. Its one decisive win, in either configuration, is rescuing the single badly miscalibrated model in the roster. The better vanilla baselines match a supervised reference trained directly on the ANES demographic-to-vote mapping, with no ANES training at all. The mechanism is structural: the elicitation stage obeys an injected ideology label nearly perfectly (label-to-prediction correlation 0.893), while the label itself is individually near-random (weighted correlation with true partisanship 0.041). Injecting a noisy compression of the demographics replaces the model's own strong demographic priors with noise.

Study 2 (population level): grounding wins, once the generative layer is audited and calibrated. The product claim of a grounded persona system is that K sampled personas constitute a population whose response distributions are calibrated, appropriately dispersed, and steerable. Study 1's design could not reward any of that. Re-scoping to a cell-level design first forced an audit of the generative model's directly decoded outputs, which surfaced two silent defects that had been corrupting the political fields of every generated persona. Repairing them and adding post-hoc calibration cut fusion-only total-variation distance from 0.417 to 0.130 with a dispersion ratio of 0.994. With the layer fixed, grounded K-persona simulation beats repeated vanilla prompting decisively on distributional fidelity and preserves the within-cell dispersion that vanilla prompting collapses.

Honest bounds frame both studies. Four baselines bound the population win: an arm that simply asks the base model beats every simulation arm on static readouts, for vote shares and the full seven-point distribution alike; a weighted lookup on the generator's own training data beats the generator; and a single calibrated label recovers most of the advantage on coarse outcomes. A verbalized-sampling arm, which asks each call for five distinct responses with probabilities, matches grounded simulation on static population distributions at a fifth of the calls and restores dispersion, showing the vanilla collapse is a prompting artifact. Static distribution readout, as a task, belongs to prompting. Grounded simulation earns its cost on per-persona joint structure, calibration auditable against named surveys, conditionals outside the model's world knowledge, and downstream interaction.

2. The study system

AnthroSim's persona generator keeps the American Community Survey as a calibrated population backbone (270,479 cells), then fits an eight-dimensional latent factor model over 4.87M sparse observations from mounted survey sources: CES, GSS, CPS, Big Five inventories, the Moral Foundations Questionnaire, Pol.is aggregates, Voteview, and EPA EJScreen, with demographic overlap anchors tying non-ACS rows into the shared latent space. Sampling compiles user criteria into a reweighting of the backbone and a conditioning of the latent posterior; a persona is one draw. ANES is not among the training sources, making it a genuinely held-out criterion throughout. The full specification, including the generation pseudo-code and identifiability notes, is Appendix D of the PDF.

3. Study 1: individual-level fidelity

Every test-set respondent is predicted twice, once from an Argyle-style demographic backstory and once from a grounded persona built from the same demographics, through an identical single-shot elicitor. A strict leakage rule holds: the scored targets never appear in any persona text, and the grounded ideology is the model's inference from demographics alone, not the respondent's self-report.

The pilot (N = 200, gpt-4o-mini) split cleanly along a discrimination-versus-calibration axis: the vanilla baseline won every individual-accuracy and rank-ordering metric while being a full point too Democratic on the seven-point scale; the grounded arm was nearly exactly calibrated in aggregate (bias 0.03) but individually noisier. Sixteen renderer variants with a held-out validation split showed wording moves individual fidelity only within noise, while deleting the single grounded ideology line sends aggregate bias straight back to the vanilla skew: content dominates prose.

The cross-model study, rerun on the repaired and calibrated generative artifact, reframed the pilot. Across seven base LLMs, against a supervised reference fit on the same seven demographics:

Two facts reorder the conclusions. Good vanilla baselines match or exceed the supervised reference with no ANES training, so for demographics-only individual prediction an off-the-shelf LLM with a plain backstory is at the practical state of the art. And the baseline leads on five of seven models, decisively on four; the two small reversals sit inside the run's precision bound and we decline to interpret them. A second rerun at the grounded arm's strongest configuration (mode decode, where the repaired label carries its full signal) contracts most gaps into the bound and leaves one decisive loss and one decisive, interpretable win: the single badly miscalibrated model being rescued by a calibrated label. Grounding never decisively beats a well-calibrated baseline in either configuration.

Mechanism. Three measurements assemble into a structural argument: the grounded label carries almost no individual signal (0.041 in the pilot; 0.251 at mode decode after repair; effectively zero at the production temperature, by design); the elicitor substantially obeys it (0.56 to 0.81 across models, measured); therefore arm accuracy tracks label accuracy, and an obedient reader of a weak label cannot beat the reader's own priors. No single-draw, single-shot individual-prediction design can reward population-grounded persona generation, because what it adds, correct conditional distributions, is invisible at the individual draw.

Why a single calibrated draw is a coin flip. The table above already runs on the repaired artifact, so the negative result is not an artifact of the decode defects. The repair makes the arithmetic visible: a single calibrated draw's party field matches the respondent's true vote 50.9% of the time, individually a coin flip by design, because the calibration restores the population's near-maximal within-cell heterogeneity as per-draw noise. Sampling one attribute value from a correctly calibrated conditional is the worst individual point estimate that conditional supports. The scope matters: population fidelity is opposed to sampled-draw accuracy, not to individual prediction as such. The same conditional read out as a probability rather than as a sampled character remains a perfectly good probabilistic prediction.

4. Study 2: auditing and calibrating the generative layer

Scoring the fusion model's decoded political fields directly against ground truth, a free operation the original study skipped, surfaced two silent defects: junk codes from one training source leaking into the party vocabulary, and categorical fields decoded as a scalar regression on the alphabetical category index, which sent the most-Democratic cells decoding past "democrat" into junk labels. The fix closes the vocabulary, redeclares party as ordinal, and retrains. Individual-level signal in the decoded party field rose from r = 0.041 to 0.251, at the supervised ceiling as a standalone predictor. The grounding carried real signal all along; the decode was destroying it.

Two calibration procedures then repaired the population-level residuals without retraining: per-field cutpoints on the prediction axis, fit by weighted quantile matching against the survey-weighted training marginal, and per-field idiosyncratic decode noise, fit by bisection so the sampler's within-cell dispersion matches the training population's. A third procedure, correlated decode noise across the party and ideology fields (rho = 0.862, fit the same way), restores per-persona cross-field coherence from 0.10 to 0.61 against a training-population reference of 0.62, with marginals and dispersion provably unchanged. The full mathematics, and the joint-dependence audit that motivated the third procedure, are Appendix E of the PDF.

5. Population calibration

The preregistered design scores predicted response distributions against survey-weighted ANES conditionals over 74 demographic cells. Five arms: fusion-only decoded draws (no LLM), the grounded product path (K personas, one elicitation each), vanilla-K (one cell backstory elicited K times), direct readout (one call asking for the distribution), and verbalized sampling (each call asks for five distinct voters with probabilities).

Four observations. The failure of vanilla-K is dispersion: K elicitations of one backstory are one modal answer with noise, and its error is flat in K while the grounded arm's keeps falling (Appendix F). The product pipeline preserves the generative layer's calibration: grounded vote TV brackets the fusion-only arm's 0.13, so rendering personas to text and passing them through an LLM does not destroy the distributions. The direct arm wins static readouts outright, on vote shares and on the full seven-point distribution; modern LLMs have well-calibrated conditional knowledge of American political distributions, not just their modes. And verbalized sampling ties the grounded arm on static distributions while restoring dispersion, at a fifth of the calls: the vanilla collapse is an artifact of one-character-repeated prompting, not an inherent limit of unguided LLMs.

6. Steering, and the two channels that bound it

Two preregistered knobs the fusion posterior conditions on and ANES independently measures: ideology band (five levels) and religious attendance (four levels), each swept within three demographic contexts. The grounded arm's TV is roughly half the vanilla arm's everywhere, and its dose-response is monotone on deepseek for both knobs. The vanilla arm is a caricature machine, mapping "never attends" to near-certain Democrat and "attends weekly" to certain Republican; direction right, levels absurd. The grounded arm has sane levels but an under-steered attendance gradient.

A free fusion-only audit explains the asymmetry, and the explanation is not the obvious one. Drawing K personas per steering cell with no LLM and reading the decoded politics shows the posterior barely propagates either knob: the fusion-only ideology gradient spans 0.31 to 0.52 against a true 0.03 to 0.97. Stage 2's near-perfect LLM-level ideology steering therefore cannot come from the posterior. Steering flows through two channels: the rendered label, which the elicitor obeys and which is strong exactly when the knob semantically names the outcome, and posterior coupling into the rest of the profile, which is weak for both knobs in the current artifact. The grounded arm composes the two; the vanilla arm is the label channel alone, in caricatured form. Both channels are measurable in advance, for free, and we argue any steering claim should ship with both audits.

7. What the advantage is, and what it is not

Two further comparisons locate what remains after the prompting arms take the static-readout franchise.

Same-data statistical baselines. The generator is fit on 4.87M survey observations, while the arms above are prompting arms with no access to that data. How much of its population fidelity is available from the same data with textbook machinery? Three baselines over the identical 74 cells: the pooled training marginal (identical for every cell) scores TV 0.138; a survey-weighted empirical conditional with hierarchical backoff scores 0.103 exact, 0.113 when sampled at the fusion arm's draw count; a marginalized multinomial logit scores 0.129. The fusion layer scores 0.118 to 0.133 across runs of the same instrument. One instrument detail initially misled us: the lookups return exact distributions while the fusion arm is scored from finite sampled draws, which carry noise. Scored symmetrically, the honest verdict is that a weighted lookup on the generator's own training data matches the generator at reproducing coarse-cell conditionals. The latent model neither beats the table it approximates nor loses to it; it earns nothing on this task. What the lookups cannot do is emit an individual with jointly coherent attributes, condition on anything outside the demographic table, say anything about a cell with no training support, or be rendered into a persona that enters a conversation.

Is it just a calibrated label injector? The renderer sweep found one grounded field, ideology, carries nearly all political behavior. Two ablations test whether that holds at the population level: L gives the model the vanilla cell backstory plus the draw's ideology sentence and nothing else; S renders the full persona but permutes the political stance and psychographic block independently across draws, preserving every marginal while destroying the joint.

The answer is mostly yes, and we report it as such. On vote share the label injector is statistically indistinguishable from the full persona on one model and better on the other: one calibrated ideology sentence appended to the vanilla backstory recovers essentially all of the three-category advantage, and restores dispersion from 0.49 and 0.24 to 0.85 and 1.00. On the finer seven-point distribution, adding psychographic content at all, even incoherently, improves error by 0.052 and 0.023, both separated from zero in a paired bootstrap. Making that content jointly coherent shows a further 0.015 and 0.011, but those intervals span zero on both models: coherence, at this design's resolution, is not distinguishable from no effect. The practical conclusion cuts against our own framing: a practitioner who needs a coarse subgroup distribution should inject a calibrated label rather than build a persona generator. The case for the generator rests on outcomes finer than a three-way share, on attributes for which no calibrated label exists to inject, and on uses that need an individual rather than a distribution.

8. Discussion

Match the evaluation to the level of the claim. The same system, on the same held-out survey, is refuted at one level of analysis and supported at another. Claims about simulated populations should be scored on distributions, dispersion, and steerable conditionals; claims about simulated individuals need individually measured grounding of the kind Park et al. collect [2], not population draws.

Whatever you inject will be obeyed. An LLM treats persona attributes as authoritative, overriding its own often-good priors. Injection is a bet that the injected value is at least as accurate as what it displaces, at the level of the use.

Score the generative layer; it is free. Two silent defects sat in the sampler's most product-critical field for months, invisible in rendered text and end-to-end metrics, and were exposed within hours by the first direct comparison of decoded fields against ground truth. The same discipline applied to our own run logs caught a silently failed experiment whose aggregates looked complete (Appendix G).

Dispersion is a first-class metric. Without it, both halves of the variance story are invisible: the vanilla collapse, and the fact that a different prompting pattern fully restores it.

When to simulate rather than ask. Static distribution readout belongs to prompting. Simulation earns its cost on per-individual draws whose many attributes cohere jointly, calibration auditable against named surveys rather than implicit in pretraining, conditionals outside the model's world knowledge, and above all downstream interaction: conversation, deliberation, group dynamics, where there is no single question whose answer is the deliverable. Quantifying that last setting against real deliberation corpora is the subject of the program's second paper.

9. Limitations

The pilot ran at N = 200 and cross-model runs at N = 150 (one at N = 60); population stages cover two base models. Effect directions were consistent everywhere tested and the population-stage margins are CI-separated, but small-N comparisons are directional. Backend nondeterminism bounds the resolution of all LLM-arm comparisons; we pool repeats throughout. All results concern U.S. politics, ANES 2020, self-reported voters, and two political targets; the direct-readout arm's strength should shrink where pretraining coverage is thinner, and generalization to less-surveyed populations is untested. Steering covers two knobs in three contexts. Preregistration was internal, in version-controlled documents with deviations logged. The author develops the system under test; the mitigations are locked decision rules, within-run supervised and direct-readout baselines, and the fact that the paper's first study is a self-refutation.

10. Ethics

A system that produces calibrated synthetic publics, steerable by ideology or religiosity, is useful for the same reason it is abusable. The steering that works is bounded and legible, and the per-knob audits are cheap to run and publish; we argue vendors should ship them as disclosures, since a steering claim without its audit is marketing. The failure modes concentrate harm: caricature amplification lands on the groups being simulated, and miscalibrated conditioning distorts underrepresented subgroups most. Synthetic respondents do not replace human data; every result here is disciplined by a real probability survey, and the appropriate uses are diagnostic and exploratory. All grounding sources are public-use microdata; sampled personas are draws from modeled conditional distributions, not records of real individuals.

11. Availability

The persona system under test is commercial software; its code and trained artifacts are not publicly released. Access is available to partners and clients under agreement, and we are exploring a hosted evaluation endpoint through which third parties could run the audits reported here against the deployed artifact without code access. The evaluation protocol itself carries no restriction: ground truth is a public survey, the baselines require nothing but an LLM API, and the prompts, cells, metrics, model equations, calibration procedures, and audits are specified in full in the PDF's appendices for independent reimplementation.

References

1. Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023). "Out of One, Many: Using Language Models to Simulate Human Samples." Political Analysis, 31(3), 337–351.
2. Park, J. S., Zou, C. Q., Shaw, A., et al. (2024). "Generative Agent Simulations of 1,000 People." arXiv:2411.10109.
3. Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023). "Whose Opinions Do Language Models Reflect?" ICML 2023.
4. Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., & Larson, J. M. (2024). "Synthetic Replacements for Human Survey Data? The Perils of Large Language Models." Political Analysis, 32(4).
5. Lutz, M., Sen, I., Ahnert, G., Rogers, A., & Strohmaier, M. (2025). "The Prompt Makes the Person(a)." Findings of EMNLP 2025.
6. Cheng, M., Piccardi, T., & Yang, D. (2023). "CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations." EMNLP 2023.
7. Zhang, J., Yu, S., Chong, D., et al. (2025). "Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity." arXiv:2510.01171.
8. Cao, Y., Liu, H., Arora, A., Augenstein, I., Röttger, P., & Hershcovich, D. (2025). "Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations." arXiv:2502.07068.
9. Zhao, J., Yuan, C., Luo, W., et al. (2025). "Large Language Models as Virtual Survey Respondents." arXiv:2509.06337.
10. Yu, C., Ye, J., Li, Y., et al. (2024). "A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science." arXiv:2412.15291.
11. Taday Morocho, E. E., Cima, L., Fagni, T., Avvenuti, M., & Cresci, S. (2026). "Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents." arXiv:2602.18462.
12. Li, M., & Conrad, F. G. (2026). "Persona-Based Simulation of Human Opinion at Population Scale." arXiv:2603.27056.
13. American National Election Studies (2021). ANES 2020 Time Series Study. electionstudies.org.

Part of the AnthroSim research program map.