Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning LLMs on first-token probabilities can substantially improve simulation of country-level survey response distributions, even for unseen countries, questions, and surveys.

desk verdict Useful empirical paper on fine-tuning LLMs to predict survey response distributions, but the main attribution to country-specific specialization is undercut by the missing zero-shot calibration baseline. read the letter →

arxiv 2502.07068 v2 pith:AYCRFTET submitted 2025-02-10 cs.CL

Yong Cao , Haijiang Liu , Arnav Arora , Isabelle Augenstein , Paul Röttger , Daniel Hershcovich This is my paper · ORCID
classification cs.CL
keywords surveysimulationlargelanguagemodelsfine-tuningfirst-tokenprobabilityKLdivergenceWorldValuesPewGlobalAttitudesresponsedistributions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can be specialized, rather than merely prompted, to predict how whole populations would answer survey questions. The authors propose a fine-tuning method that aligns a model's first-token probability distribution over answer options with the observed country-level response distribution from the World Values Survey, using a Kullback-Leibler divergence loss. They show that this method substantially outperforms zero-shot prompting and other baselines across seven models and three model families, and that the gains carry over to unseen countries, unseen questions, and a completely unseen survey (the Pew Global Attitudes Survey). The paper also reports that even the best fine-tuned models remain far from accurate, especially on unseen questions, and that model predictions are consistently less diverse across countries than the human data. If the method proves robust, it would offer a cheaper, faster way to approximate group-level survey results for social science and policy before investing in expensive data collection.

What carries the argument

The central mechanism is first-token probability alignment: the model's input is formatted so that the first output token must be one of the valid answer options, the logits for those options are normalized with a softmax to form a predicted distribution, and the training loss is the KL divergence between that predicted distribution and the human response distribution for the given country and question. This turns distribution simulation into a calibration task rather than a single-answer prediction task, and it is what allows a decoder-only LLM to output a full response distribution from just the first token. The format field that restricts the first-token vocabulary, together with LoRA for efficient fine-tuning, carries most of the method's effectiveness.

What would settle it

Compute country-level response distributions from the WVS microdata using respondent weights and survey design, then re-run the fine-tuned models against those weighted references; if the 1-JSD gaps between fine-tuned and zero-shot models (roughly 0.05 to 0.10 points) fall inside the surveys' margins of error, the claim of substantial improvement would not survive.

Watch

Extended reading notes

Core claim

The central claim is that specializing LLMs through first-token probability alignment fine-tuning substantially improves the simulation of country-level survey response distributions, and that these gains generalize beyond the training distribution. The authors devise a training objective that minimizes the KL divergence between the softmax distribution over the first-token logits of the valid answer options and the human response distribution for each country-question pair, using LoRA for parameter-efficient tuning. Across seven models (Vicuna1.5-7B/13B, Llama3-8B-Base/Instruct, and Distil-Qwen-7B/14B/32B), the fine-tuned models consistently beat zero-shot prompting and control baselines (shuffled countries, KNN, average-culture, and JSON prompting) on both 1-JSD and EMD metrics. The improvement holds for unseen WVS questions and countries, for the Chinese version of the WVS, and for the held-out Pew survey. At the same time, the paper's own results show systematic limitations: unseen questions remain the hardest setting, and all models, fine-tuned or not, produce less cross-country diversity than humans do.

Load-bearing premise

The paper treats the published country-level percentages from the WVS and Pew surveys as exact, noise-free human response distributions, and it uses machine-translated text for part of the Chinese training data, so any errors in those references would shrink the reported improvements.

Editorial extensions

If this is right

  • If the method is correct, LLMs specialized this way could provide low-cost, preliminary estimates of country-level opinions on new questions without fielding a survey.
  • The consistent improvement across base and instruct models of different families suggests that first-token KL fine-tuning is a robust specialization recipe, not a quirk of one model architecture.
  • Because gains transfer to a completely unseen survey, the approach may generalize beyond the World Values Survey to other global questionnaires with the same multiple-choice format.
  • The paper's finding that even the best models are weak on unseen questions and under-diverse across countries implies that such simulations should not yet replace real surveys for consequential decisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method could plausibly be extended beyond country-level aggregates to subnational groups, demographic segments, or even individual-level response distributions, since the same first-token alignment loss applies to any target distribution.
  • The persistent under-diversity across countries suggests a structural limit of likelihood-based fine-tuning: the model learns an average cultural prior and may require explicit diversity-promoting terms to match inter-country variance.
  • A natural testable extension is to compare first-token KL fine-tuning against fine-tuning that predicts the full response distribution in one step (e.g., via a softmax over all options decoded at once); the paper's ablation with JSON-style prompting only tests the zero-shot variant, leaving a fine-tuned multi-token baseline unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a fine-tuning method for large language models that predicts country-level survey response distributions. The method minimizes KL divergence between the model's first-token probabilities over answer options and the human response distribution from the World Values Survey. The authors evaluate seven models from three families on held-out questions, held-out countries, and the Pew Global Attitudes Survey, reporting consistent improvements over zero-shot prompting. They also analyze country-context sensitivity through a control condition, ablate the loss function and option ordering, and include a Chinese-language robustness study. The paper is honest about remaining limitations, especially for unseen questions.

Significance. If the central claim is fully supported, this is a useful contribution to LLM-based survey simulation: it provides a concrete fine-tuning recipe, publicly available datasets, and an honest assessment of current limits. The breadth of models tested and the held-out Pew survey are clear strengths. However, the stronger attribution to 'specialization' rather than calibration is not yet established because the zero-shot baseline is not calibrated and no uncertainty quantification is provided. These issues are addressable, so the contribution is promising but needs revision.

major comments (4)
  1. [Section 6.1, Table 4] All reported scores are single point estimates without standard deviations, confidence intervals, or significance tests. Since each test subset contains many country-question entries, the authors should report variability (for example, bootstrap over entries or per-country or per-question standard errors) to support the claims of 'substantially outperforms' and 'consistent across seven models.' This is particularly important for comparisons with small absolute differences, such as several FT versus FT[ctrl] rows in Table 4.
  2. [Section 6.2, Table 4] The sentence reporting 'a 16.7% avg. (1−JSD) drop between FT[ctrl] and FT across seven models, compared to 3.7% for ZS[ctrl] and ZS' is not reproducible from Table 4. Averaging the listed averages gives FT ≈ 0.816 versus FT[ctrl] ≈ 0.789, a relative drop of about 3.2%, and ZS ≈ 0.707 versus ZS[ctrl] ≈ 0.702, a drop of about 0.7%. Please correct these numbers and re-examine the conclusion that fine-tuned models are substantially more sensitive to country context.
  3. [Section 6.1, Tables 4 and 11] The zero-shot baseline is never calibrated. LLM first-token distributions are typically overconfident, and the control condition FT[ctrl] already yields large gains over ZS[ctrl] without using the correct country labels (for example, Llama3-8B-Instruct: 0.599 to 0.786), so part of the improvement may be due to output smoothing rather than learned country-specific structure. To support the paper's 'specialization' attribution, add a calibrated zero-shot baseline (for example, temperature scaling or label smoothing of the zero-shot logits) or explicitly reframe the contribution as distribution calibration. This is the main correctness-risk concern for the paper's central claim.
  4. [Section 3.1, Table 4] The reference distributions are treated as noise-free ground truth. The WVS aggregate percentages come with sampling error (typical country samples are around 1,000 to 1,500 respondents), and the reported improvements are on the order of 0.05 to 0.10 in 1−JSD. The authors should quantify the sensitivity of the main comparison to plausible noise in the reference distributions, for example by perturbing the targets with multinomial sampling error and recomputing the metrics.
minor comments (6)
  1. [Abstract and Section 6.3] The abstract states that results hold for a completely unseen survey, but Table 6 reports Pew results for only two models, not all seven. Please qualify the claim accordingly.
  2. [Section 3.3, Table 2] C2 and C3 overlap (Nigeria and Morocco appear in both). Please state explicitly that the test subsets are disjoint from the training countries but not from each other, and justify the overlap.
  3. [Section 5.2] The control condition replaces countries 'randomly selected from the full set of countries.' If random replacement can select the original country, the control is not a pure no-country condition. Please clarify or constrain the sampling.
  4. [Appendix F and Section 6.1] There are several typos: 'Comparsion' should be 'Comparison' in Appendix headings, 'V olkswagenStiftung' in the acknowledgments should be 'VolkswagenStiftung,' and 'further revealing' in Section 6.1 should be 'further reveal.'
  5. [Figure 3 caption] The description of the blue-shaded area is unclear; please define the lower and upper boundaries more precisely.
  6. [Section 6.4, Table 7] In Table 7, please clarify that the 'Shuffled' row evaluates a model on a test set with shuffled option order, not a shuffled training order.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported fine-tuning gains are measured on held-out countries, questions, and an unseen survey, so the specialization claim is an empirical finding rather than a construction.

full rationale

The paper's central claim is that first-token KL-divergence fine-tuning improves simulation of country-level survey response distributions, and it evaluates this against held-out WVS country/question splits and a completely unseen Pew survey. The training objective directly minimizes divergence from human survey distributions, but the evaluation uses data not seen during training (Tables 4 and 6), so the improvements are not guaranteed by construction. No parameter is fitted to the test distributions and then renamed as a prediction; the model is trained on WVS Q1/C1 entries and tested on Q3, C2, C3, and Pew. The authors' earlier cultural-alignment papers are cited only as related work or evaluation context, not as load-bearing justification for the method or as a uniqueness theorem. The control condition FT[ctrl], which randomly replaces country labels, is presented as an analysis of prior-distribution effects rather than as the main claim, and it does not reduce the headline comparison to a tautology. The skeptical concern about zero-shot calibration is a correctness risk, not a circularity, because the observed generalization could have failed and is assessed against external human survey data. The derivation chain is therefore self-contained and empirically falsifiable.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim is empirical, so the ledger records the training hyperparameters and split choices that shape the result, plus the domain assumptions about survey ground truth, country conditioning, first-token softmax validity, and machine translation quality. No new theoretical entities are introduced.

free parameters (7)
  • LoRA rank = 8
    Rank of low-rank adaptation chosen by hand; affects how much cultural distribution knowledge can be stored during fine-tuning (Appendix C).
  • LoRA alpha = 32
    Scaling factor for LoRA updates, chosen by hand (Appendix C).
  • LoRA dropout = 0.05
    Dropout rate for LoRA layers, chosen by hand (Appendix C).
  • learning rate = 1e-4
    AdamW learning rate for all fine-tuning runs, chosen by hand (Appendix C).
  • batch size = 16 for Llama3/Vicuna-7B, 4 for Vicuna-13B
    Batch size chosen by hand; affects optimization dynamics and reproducibility (Appendix C).
  • country inclusion threshold = more than 1,000 respondents
    WVS countries with fewer than 1,000 respondents are excluded to ensure balanced representation; this preprocessing choice shapes the task distribution (Section 3.1).
  • question and country split boundaries = Q1 questions 1-163, Q2 164-198, Q3 199-259; C1/C2/C3 country groupings
    The splits are designed by hand to create challenging unseen sets; results depend on which questions and countries fall into each split (Section 3.3).
assumptions (5)
  • domain assumption WVS and Pew country-level distributions accurately represent population opinions.
    Aggregate survey percentages are used as training targets and evaluation references without modeling sampling weights or margins of error (Section 3.1, Section 3.4).
  • domain assumption Country label is a sufficient condition for a population's response distribution.
    The input includes only the country name and question; within-country demographic variation is ignored, a limitation the paper acknowledges in its Limitations section.
  • domain assumption The restricted first-token format captures the full answer-choice distribution.
    The prompt format forces the first token to be an option letter, so the softmax over option logits is treated as the predicted response distribution (Section 3.2, Section 4.2).
  • domain assumption LoRA fine-tuning with KL loss can internalize generalizable cultural distributions.
    The method assumes that a low-rank update on 6,841 training entries transfers to unseen questions, countries, and surveys; the paper tests this empirically but does not justify it theoretically (Section 4.2, Section 6).
  • ad hoc to paper GLM-4 machine translation preserves survey question meaning in Chinese.
    Missing Chinese WVS questions are translated with GLM-4 rather than taken from official human translations, so the cross-linguistic robustness result depends on translation quality (footnote 3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations." pith.science (2026). https://pith.science/paper/AYCRFTET

@misc{pith2026250207068,
  author       = {Pith},
  title        = {Pith review of: Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYCRFTET}},
  note         = {Machine review of arXiv:2502.07068}
}
read the original abstract

Large-scale surveys are essential tools for informing social science research and policy, but running surveys is costly and time-intensive. If we could accurately simulate group-level survey results, this would therefore be very valuable to social science research. Prior work has explored the use of large language models (LLMs) for simulating human behaviors, mostly through prompting. In this paper, we are the first to specialize LLMs for the task of simulating survey response distributions. As a testbed, we use country-level results from two global cultural surveys. We devise a fine-tuning method based on first-token probabilities to minimize divergence between predicted and actual response distributions for a given question. Then, we show that this method substantially outperforms other methods and zero-shot classifiers, even on unseen questions, countries, and a completely unseen survey. While even our best models struggle with the task, especially on unseen questions, our results demonstrate the benefits of specialization for simulation, which may accelerate progress towards sufficiently accurate simulation in the future.

Figures

Figures reproduced from arXiv: 2502.07068 by the authors.

Figure 1
Figure 1. Overview of our proposed survey response distribution simulation framework (right panel) ver￾sus direct answer prediction (left panel), highlighting a novel perspective on cultural simulation with LLMs. stereotypical, or overconfident answers, especially in culturally diverse contexts (Yang et al., 2024), which limits their usefulness for survey simulations. Prior work has at most tried to improve simulation accurac… view at source ↗
Figure 2
Figure 2. Option prediction accuracy for cultural ques [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Model Diversity and Country Accuracy Analysis. (a)-(d) denotes the comparison of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Sample distribution of the World Values Survey across countries, including only countries with more than [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Visualization of country and cultural dimension divisions of WVS. Countries are categorized into three [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 7
Figure 7. Figure 7: Training loss comparsion of three mod￾els: Qwen-7B, Llama-8B-Instruct, Distill-Qwen￾32B(Deepseek) effectively align the Base and Instruct models. E More Baseline Comparsion To further assess the effectiveness of our proposed approach, we compare it against several othe…
Figure 6
Figure 6. Figure 6: Llama3 Accuracy of options on African coun [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 8
Figure 8. Figure 8: Distribution of 1−JSD Global Scores for the Model on Unseen Cultural Questions (C1-Q3). The Instruct model exhibits a more distinct improvement compared to the Base model. Methods C1-Q3 C2-Q1 C2-Q3 C3-Q1 C3-Q3 Avg. KNN 0.381 0.518 0.371 0.541 0.384 0.439 Avg_Culture 0.…
Figure 9
Figure 9. Figure 9: Comparsion of different languages in 1-JSD [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models

    cs.CL 2026-07 conditional novelty 7.0 of 10

    LLM probability estimates violate the law of total probability across partitions, and subgroup-aggregated estimates often beat direct population-level estimates (the macro fallacy).

  2. Tailored untruths: How personalisation challenges LLM safeguards

    cs.CL 2025-10 conditional novelty 7.0 of 10

    A 1.6-million-text study of eight LLMs in four languages finds that adding demographic personae to disinformation prompts raises jailbreak rates from 78% to 82%.

Reference graph

Works this paper leans on

33 extracted references · 12 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337--371. PMLR

    work page 2023

  2. [2]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  3. [3]

    Badr AlKhamissi, Muhammad ElNokrashy, Mai Alkhamissi, and Mona Diab. 2024. https://doi.org/10.18653/v1/2024.acl-long.671 Investigating cultural alignment of large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12404--12422. Association for Computational Linguistics

    doi:10.18653/v1/2024.acl-long.671 2024

  4. [4]

    Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337--351

    2023

  5. [5]

    Arnav Arora, Lucie-aim \'e e Kaffee, and Isabelle Augenstein. 2023. https://doi.org/10.18653/v1/2023.c3nlp-1.12 Probing pre-trained language models for cross-cultural differences in values . In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 114--130. Association for Computational Linguistics

    doi:10.18653/v1/2023.c3nlp-1.12 2023

  6. [6]

    Joris Baan, Wilker Aziz, Barbara Plank, and Raquel Fernandez. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.124 Stop measuring calibration when humans disagree . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1892--1915. Association for Computational Linguistics

    doi:10.18653/v1/2022.emnlp-main.124 2022

  7. [7]

    Joris Baan, Raquel Fern \'a ndez, Barbara Plank, and Wilker Aziz. 2024. https://aclanthology.org/2024.eacl-short.24 Interpreting predictive probabilities: Model confidence or human label variation? In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 268--277. Associ...

    work page 2024

  8. [8]

    Christopher A Bail. 2024. Can generative ai improve social science? Proceedings of the National Academy of Sciences, 121(21):e2314021121

    work page 2024

Show all 33 references
  1. [9]

    Noam Benkler, Drisana Mosaphir, Scott Friedman, Andrew Smart, and Sonja Schmer-Galunder. 2023. Assessing LLM s for moral value pluralism. arXiv preprint arXiv:2312.10075

    2023 arXiv

  2. [10]

    James Bisbee, Joshua D Clinton, Cassy Dorff, Brenton Kenkel, and Jennifer M Larson. 2023. Synthetic replacements for human survey data? the perils of large language models. Political Analysis, pages 1--16

    2023

  3. [11]

    Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. https://doi.org/10.18653/v1/2023.c3nlp-1.7 Assessing cross-cultural alignment between C hat GPT and human societies: An empirical study . In Proceedings of the First Workshop on Cross-Cultur...

    2023 doi

  4. [12]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing GPT -4 with 90\

    2023

  5. [13]

    Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-D \"u nner. 2023. Questioning the survey responses of large language models. arXiv preprint arXiv:2306.07951

    2023 arXiv

  6. [14]

    Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXi...

    2023 arXiv

  7. [15]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  8. [16]

    Haerpfer, R

    C. Haerpfer, R. Inglehart, A. Moreno, C. Welzel, K. Kizilova, J. Diez-Medrano, M. Lagos, P. Norris, E. Ponarin, and B. Puranen. 2022. https://doi.org/10.14281/18241.18 World values survey wave 7 (2017-2022) cross-national data-set

    2022 doi

  9. [17]

    Guande He, Peng Cui, Jianfei Chen, Wenbo Hu, and Jun Zhu. 2024. https://openreview.net/forum?id=pVKEFtGkM6 Investigating uncertainty calibration of aligned language models under the multiple-choice setting

    2024

  10. [18]

    Luke Hewitt, Ashwini Ashokkumar, Isaias Ghezae, and Robb Willer. 2024. https://samim.io/dl/Predicting

    2024

  11. [19]

    John J Horton. 2023. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research

    2023

  12. [20]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  13. [21]

    Austin Kozlowski and James Evans. 2024. Simulating subjects: The promise and peril of ai stand-ins for social agents and interactions

    2024

  14. [22]

    Louis Kwok, Michal Bravansky, and Lewis D Griffin. 2024. Evaluating cultural adaptability of a large language model via simulation of synthetic personas. arXiv preprint arXiv:2408.06929

    2024 arXiv

  15. [23]

    Wangyue Li, Liangzhi Li, Tong Xiang, Xiao Liu, Wei Deng, and Noa Garcia. 2024. https://aclanthology.org/2024.lrec-main.251 Can multiple-choice questions really be useful in detecting the abilities of LLM s? In Proceedings of the 2024 Joint International Conference on Computati...

    2024

  16. [24]

    Benjamin S Manning, Kehang Zhu, and John J Horton. 2024. Automated social science: Language models as scientist and subjects. Technical report, National Bureau of Economic Research

    2024

  17. [25]

    Peter S Park, Philipp Schoenegger, and Chongyang Zhu. 2024. Diminished diversity-of-thought in a standard large language model. Behavior Research Methods, pages 1--17

    2024

  18. [26]

    Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 1998. A metric for distributions with applications to image databases. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271), pages 59--66. IEEE

    1998

  19. [27]

    Jansen, and Jang Hyun Kim

    Seungjong Sun, Eungu Lee, Dongyan Nan, Xiangying Zhao, Wonbyung Lee, Bernard J. Jansen, and Jang Hyun Kim. 2024. https://arxiv.org/abs/2402.18144 Random silicon sampling: Simulating human sub-population opinion using a large language model based on group-level demographic info...

    2024 arXiv

  20. [28]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  21. [29]

    Dustin Wright, Arnav Arora, Nadav Borenstein, Srishti Yadav, Serge Belongie, and Isabelle Augenstein. 2024. Revealing fine-grained values and opinions in large language models

    2024

  22. [30]

    Yahan Yang, Soham Dan, Dan Roth, and Insup Lee. 2024. https://arxiv.org/abs/2311.08669 On the calibration of multilingual question answering llms . Preprint, arXiv:2311.08669

    2024 arXiv

  23. [31]

    Wenlong Zhao, Debanjan Mondal, Niket Tandon, Danica Dillion, Kurt Gray, and Yuling Gu. 2024. Worldvaluesbench: A large-scale benchmark dataset for multi-cultural value awareness of language models. In Proceedings of the 2024 Joint International Conference on Computational Ling...

    2024

  24. [32]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  25. [33]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.