Thinking beyond the anthropomorphic paradigm benefits LLM research

Lujain Ibrahim; Myra Cheng

REVIEW 4 major objections 5 minor 2 cited by

Thinking beyond the anthropomorphic paradigm benefits LLM research

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM research increasingly describes models in human terms, and this paper argues those assumptions narrow the questions researchers ask.

desk verdict A useful position paper with a real framework and trend update, but the empirical motivation has a small numeric inconsistency and the metric's interpretation deserves more scrutiny. read the letter →

arxiv 2502.09192 v2 pith:J3YJX5E6 submitted 2025-02-13 cs.CL

Lujain Ibrahim , Myra Cheng This is my paper · ORCID

classification cs.CL

keywords anthropomorphismlargelanguagemodelsLLMresearchframingassumptionsevaluationbenchmarkschain-of-thoughtreasoninghuman-AIinteraction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that anthropomorphism — treating large language models as human-like thinkers, learners, and interlocutors — is not just a habit of everyday speech but a structural feature of LLM research that has grown more prevalent over time. The authors measure this growth across more than 250,000 research abstracts and report that the share of abstracts containing at least one anthropomorphic sentence rose roughly 150% between 2007 and 2022, with LLM papers showing the highest and fastest-growing rates. They then argue that five common anthropomorphic assumptions — about training, alignment, evaluation, interpreting model behavior, and user interaction — each constrain the questions researchers ask and the methods they try. For each assumption they point to a non-anthropomorphic alternative that already shows promise but remains under-explored, and they recommend a both-and approach rather than a ban on human-centered language. If the argument is right, deliberately diversifying the metaphors and assumptions of the field would open new research pathways.

What carries the argument

Two objects carry the argument. The first is AnthroScore, a measure of implicit anthropomorphism that uses a masked language model to compare the probability that an entity is replaced by the pronouns 'he' or 'she' against the probability it is replaced by 'it'; the paper's binary version flags a text as anthropomorphic if any sentence in it scores positive. This metric produces the prevalence numbers and the temporal trend. The second is a five-stage lifecycle framework — training, alignment, evaluation, understanding model behavior, and user interaction — which turns the metric's signal into a structural claim: each stage rests on an identifiable anthropomorphic assumption, and the assumption is the true constraint.

What would settle it

Have human annotators classify a random sample of high- and low-scoring abstracts for the presence of the paper's five anthropomorphic assumptions, and test whether the binary AnthroScore predicts assumption presence; if it does not, the metric fails as a proxy. A second check: recompute the temporal trend after removing verbs with technical meanings in machine learning (such as 'learn', 'understand', and 'train'); if the measured increase depends on those verbs, the trend reflects vocabulary rather than changing assumptions.

Watch

Extended reading notes

Core claim

The paper's central claim is that the unit of analysis for critiquing anthropomorphism should be implicit assumptions, not just terminology, because assumptions give rise to the methods and words that other critiques attack. The authors identify five assumptions that organize the LLM development lifecycle: human-like approaches are optimal for training, models should explicitly reason about and implement human values, model capabilities should be measured in human-like ways, human-like normative judgments should be assigned to model behaviors, and human-LLM interaction mirrors human-human interaction. For each, they show both the usefulness of the assumption and a body of empirical work that challenges it — byte-level tokenization, latent-space reasoning, normative specifications without moral language, dynamic and model-specific evaluation, role-play reframings of deception, and structured interfaces. The paper's conclusion is not that anthropomorphism should be eliminated, but that the field has leaned on it disproportionately and would benefit from a both-and stance.

Load-bearing premise

The quantitative case depends on the assumption that a language model's pronoun-choice score is a valid proxy for anthropomorphic thinking in an abstract, and that the terms researchers use reliably reflect the assumptions that shape their research.

Editorial extensions

If this is right

If anthropomorphic framing shapes research questions, then the subfields with the highest measured rates — interpretability, ethics, and dialogue systems — are the places where non-anthropomorphic alternatives are most likely to surface new methods.
If byte-level tokenization and latent-space reasoning keep matching or beating human-centric methods, the default assumption that human-understandable units are optimal for models loses its force.
If static, human-designed benchmarks miss model-specific failure modes, then evaluation practice should move toward dynamic and model-specific tests as a complement to standard exams.
If 'hallucination' and 'sycophancy' are reader-side normative labels rather than model properties, interventions should target training data and post-training procedures instead of asking the model to be more honest.
If conversational prompting is not the optimal interaction paradigm, structured interfaces that expose system capabilities deserve to be tested as first-class alternatives rather than afterthoughts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same metric could be applied to model-generated text, product documentation, or policy reports to trace where anthropomorphic assumptions propagate outside research.
The framework suggests a testable prediction: research groups that deliberately adopt non-anthropomorphic framing will produce evaluations that predict out-of-distribution behavior better than standard human-benchmark scores, because they target the model's actual training objective.
The five assumptions are coupled; for example, if human-style evaluation loses its default status, the evidence base for alignment changes too, so shifting one stage should ripple through the others.
A concrete extension would be a 'de-anthropomorphized' benchmark that rewrites standard evaluation prompts in structured, non-conversational formats and measures whether user performance changes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should read this one. It's a position paper arguing that anthropomorphic assumptions constrain LLM research, and it actually backs the argument with a new empirical slice plus a reusable framework. What's genuinely new: the authors binarize AnthroScore, apply it to 2023–2024 arXiv data, and show the upward trend continues (34% to 40% of CS abstracts, 48% of LLM abstracts). They also organize the field's scattered critiques into five lifecycle assumptions—training, alignment, evaluation, understanding behavior, user interaction—and pair each with concrete non-anthropomorphic alternatives like byte-level tokenization, latent-space reasoning, and control-theoretic alignment. That framework is the real contribution; it sharpens the debate and gives future work a unit of analysis beyond individual word choices.

The paper is well written and honest. It acknowledges anthropomorphism can be useful, includes an alternative-views section, and doesn't overclaim. The literature coverage is solid, and the self-citation to AnthroScore is legitimate—it's a published metric, not a fitted parameter.

Soft spots, in order of seriousness. First, the numeric inconsistency: the intro says a ~150% increase since 2007, but the ACL Anthology numbers go from 5% to 11%, which is a 120% relative increase. That needs a fix. Second, no code or data are shipped, so the empirical figures can't be audited. For a paper that leans on a quantitative trend, that's a real omission for a conference submission. Third, the stress-test concern has teeth: the Fightin' Words analysis shows high-AnthroScore sentences are rich in standard ML verbs like 'learn,' 'train,' 'understand,' 'memorize.' It's plausible the metric is partly tracking technical jargon rather than anthropomorphic thinking. The authors acknowledge terminology is an imperfect proxy and cite the language-thought link, but they don't validate the binary metric against human-labeled abstracts. That's the weakest link in the empirical chain. I wouldn't call it fatal—they are careful to talk about 'anthropomorphic terminology' and then reason about assumptions—but it's load-bearing for the paper's claim that anthropomorphic thinking itself is increasing.

Who's this for? NLP/ML researchers, AI alignment and evaluation folks, and HCI. It deserves a serious referee: the framework is useful and the trend update is worth publishing, provided the authors fix the numbers and release the artifacts. I'd send it to review.

Referee Report

4 major / 5 minor

Summary. The paper is a position piece arguing that anthropomorphic framing of large language models is prevalent and increasing, and that this framing, while sometimes useful, may constrain research directions. The empirical basis is a large-scale analysis of arXiv and ACL Anthology abstracts using AnthroScore, a masked-language-model-based measure, with a new binary threshold variant Abin. The paper then proposes a framework of five anthropomorphic assumptions spanning the LLM lifecycle (training, alignment, evaluation, understanding behavior, user interaction) and surveys non-anthropomorphic alternatives for each. It concludes with recommendations and an acknowledgment of alternative views.

Significance. If the empirical trend is trustworthy, the paper provides a timely and useful reframing of a widespread linguistic habit in LLM research. Its main strengths are the breadth of the synthesized examples, the constructive both-and stance rather than a blanket rejection of anthropomorphism, and the explicit link between terminology and research assumptions. The framework in Table 1 is a plausible organizing device, and the paper names concrete falsifiable alternatives (e.g., byte-level tokenization, latent-space reasoning, non-human-centric benchmarks) that could be pursued empirically. However, the quantitative motivation is less solid than the framing suggests: the metric's validity on this corpus is not independently demonstrated, the reported trend awaits robustness checks, and the paper does not ship code or data, which limits the extent to which the empirical claims can be audited.

major comments (4)

[§1 and §3] The abstract and introduction claim a "~150% increase since 2007," but the ACL Anthology numbers reported in Section 3 imply an increase from 5% to 11%, which is a 120% relative increase (or a 6-percentage-point absolute increase). The paper should correct this discrepancy and ensure that all quantitative statements are mutually consistent.
[§3, Score sensitivity] The AnthroScore-based evidence is not validated against human judgments on this corpus. The Fightin' Words analysis lists verbs such as "learn," "train," "understand," and "memorize" as characterizing high-AnthroScore sentences, and these are standard machine learning terms rather than obvious attributions of human mental states. The paper asserts a language-thought link via citations, but it does not provide evidence that Abin behaves differently from a keyword detector for ML jargon. At minimum, the authors should report a validation of Abin against human-labeled abstracts, or an analysis showing that the temporal increase is not driven by generic ML vocabulary growth.
[§3, Abin definition] The binary threshold Abin(S) = 1 if any sentence in the abstract has A(sx) > 0 makes the prevalence measure sensitive to abstract length and to the number of entity mentions per abstract. If recent arXiv abstracts are systematically longer or mention models more frequently, the observed increase from 34% to 40% could be an artifact. The paper should control for abstract length, entity count, and sentence count, or report the robustness of the trend to these covariates.
[§3, data and code] No code, data, or analysis scripts are provided, despite the empirical claims being central to the paper's motivation. The paper relies on a specific metric, a thresholding variant, dataset versions, and filtering criteria; without these artifacts, reviewers and readers cannot reproduce the central quantitative results or check the subfield analysis.

minor comments (5)

[§3] There is a typo in "an entitiy x" in the definition of Abin.
[Figure 1] The left panel caption says "Jan 2023 – Oct 2024" while the text says "January 2023 – December 2024," and the x-axis ticks show 2023-01, 2023-08, 2024-03, 2024-10; please align the reported time window with the plotted data.
[§4.1] The "Using words for tokenization" example characterizes subword tokenization as reflecting a human-like assumption; however, byte-level tokenization is itself a design choice that may be motivated by engineering constraints rather than by an explicit non-anthropomorphic stance. Clarifying this distinction would avoid overloading the concept of anthropomorphism.
[Table 1] The Training row lists chain-of-thought reasoning as an example of anthropomorphism's usefulness, but chain-of-thought is an inference-time prompting technique rather than a training-phase method; consider renaming the stage or moving the example.
[References] Reference [63] is a social media post; replacing it with a peer-reviewed or preprint source would strengthen the scholarly framing of the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical trend is measured with a published external metric, and the framework is a synthesis, not a derivation.

full rationale

The paper's central empirical claim is an observational measurement of anthropomorphic terminology using AnthroScore, a metric published in prior work (Cheng et al., 2024) by one of the present authors. The present paper's modification, Abin, is a deterministic thresholding of the original sentence-level score into a binary abstract-level indicator; it introduces no fitted parameter and does not presuppose the trend it reports. The metric itself is publicly specified and code-reproduced, and the paper reproduces prior ACL Anthology findings with it, so this self-citation is independent support rather than load-bearing circularity. The qualitative framework in Section 4 is a synthesis of existing results and proposals from the literature, each cited to its original source; it does not derive the framework from the measurement, nor does any prediction reduce to an input by construction. The paper explicitly acknowledges that terminology is an imperfect proxy for anthropomorphic thinking (Section 3, 'Significance'), which is a stated limitation about validity, not a circular step. There is no fitted-parameter-renamed-as-prediction, no importation of a uniqueness theorem, and no ansatz smuggled in via citation. The self-citations that are present are contextual or methodological, and the central argument retains independent content. Therefore the paper exhibits no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper contributes no free parameters or invented entities. Its argument rests on the validity of AnthroScore as a measurement tool and on the premise that linguistic framing shapes research directions; both are stated assumptions, not derived results.

assumptions (4)

domain assumption AnthroScore via RoBERTa is a valid proxy for anthropomorphic framing.
The paper relies on this metric to measure prevalence and admits it is an imperfect proxy in Section 3.
domain assumption Terminology reflects underlying assumptions (language structures thinking).
The paper argues 'language fundamentally structures our thinking' in Section 3, citing Lakoff and Johnson, to link measured terminology to assumptions.
ad hoc to paper The five lifecycle stages are the natural and most relevant decomposition.
The authors select five stages without derivation; the framework is a conceptual proposal presented in Section 4.
domain assumption The cited non-anthropomorphic alternatives are indeed under-explored.
The claim that these paths are 'under-explored' is based on the authors' assessment of the literature, not a quantitative analysis.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Thinking beyond the anthropomorphic paradigm benefits LLM research." pith.science (2026). https://pith.science/paper/J3YJX5E6

@misc{pith2026250209192,
  author       = {Pith},
  title        = {Pith review of: Thinking beyond the anthropomorphic paradigm benefits LLM research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J3YJX5E6}},
  note         = {Machine review of arXiv:2502.09192}
}

read the original abstract

Anthropomorphism, or the attribution of human traits to technology, is an automatic and unconscious response that occurs even in those with advanced technical expertise. In this position paper, we analyze hundreds of thousands of research articles to present empirical evidence of the prevalence and growth of anthropomorphic terminology in research on large language models (LLMs). We argue for challenging the deeper assumptions reflected in this terminology -- which, though often useful, may inadvertently constrain LLM development -- and broadening beyond them to open new pathways for understanding and improving LLMs. Specifically, we identify and examine five anthropomorphic assumptions that shape research across the LLM development lifecycle. For each assumption (e.g., that LLMs must use natural language for reasoning, or that they should be evaluated on benchmarks originally meant for humans), we demonstrate empirical, non-anthropomorphic alternatives that remain under-explored yet offer promising directions for LLM research and development.

Figures

Figures reproduced from arXiv: 2502.09192 by the authors.

**Figure 1.** Temporal increase in % of abstracts with > 1 anthropomorphic sentence in CS arXiv papers from Jan 2023 – Oct 2024 (left) and ACL anthology papers from 2007 – 2022 (middle). Anthropomorphism is prevalent and is steadily increasing, especially in LLM and NLP papers. Rates of anthropomorphic abstracts by ACL anthology topics (right). “Interpretability", “ethics", and “dialogue" have the highest rates of anthropomorphis… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
cs.CL 2025-07 conditional novelty 6.0 of 10

A minimal-prior pipeline with automated data curation and verifier-driven RL lets small LLMs generate verifiable Dafny specifications and beat larger proprietary models on a synthetic compositional benchmark.
A validity-guided workflow for robust large language model research in psychology
cs.HC 2025-07 conditional novelty 4.0 of 10

A six-stage workflow scales validity requirements to research ambition so that LLM-based psychological claims rest on demonstrated measurement quality rather than prompt artifacts.

Reference graph

Works this paper leans on

93 extracted references · 40 canonical work pages · cited by 2 Pith papers

[1]

Abercrombie, G., Cercas Curry, A., Dinkar, T., Rieser, V ., and Talat, Z. Mirages. on anthropo- morphism in dialogue systems. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of 9 the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4776–4790, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18...

doi:10.18653/v1/2023 2023
[2]

R., Smith, N

Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., and Tsvetkov, Y . Do all languages cost the same? tokenization in the era of commercial language models. arXiv preprint arXiv:2305.13707, 2023

arXiv 2023
[3]

All Too Human? Map- ping and Mitigating the Risk from Anthropomorphic AI

Akbulut, C., Weidinger, L., Manzini, A., Gabriel, I., and Rieser, V . All Too Human? Map- ping and Mitigating the Risk from Anthropomorphic AI. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 7, pp. 13–26, 2024

2024
[4]

Language models as agent models

Andreas, J. Language models as agent models. arXiv preprint arXiv:2212.01681, 2022

arXiv 2022
[5]

Claude’s character, Jun 2024

Anthropic. Claude’s character, Jun 2024. URL https://www.anthropic.com/research/ claude-character

2024
[6]

alignment faking in large language models

Anthropic. External reviews of “alignment faking in large language models”, Dec 2024. URL https://assets.anthropic.com/m/24c8d0a3a7d0a1f1/original/ Alignment-Faking-in-Large-Language-Models-reviews.pdf

2024
[7]

arxiv dataset, 2024

arXiv.org submitters. arxiv dataset, 2024. URL https://www.kaggle.com/dsv/7548853

arXiv 2024
[8]

Training a helpful and harmless assistant with reinforcement learning from human feedback

Bai, Y ., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

arXiv 2022

Show all 93 references

[9]

Constitutional ai: Harmlessness from ai feedback

Bai, Y ., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022

2022 arXiv
[10]

Feedback control of flexible systems

Balas, M. Feedback control of flexible systems. IEEE Transactions on Automatic Control, 23 (4):673–679, 1978

1978
[11]

Engineering ai systems: Architecture and devops essentials, 2025

Bass, L., Lu, Q., Weber, I., and Zhu, L. Engineering ai systems: Architecture and devops essentials, 2025

2025
[12]

Bender, E. M. Resisting dehumanization in the age of “AI”. Curr. Dir. Psychol. Sci., 33(2): 114–120, April 2024

2024
[13]

and Gavves, E

Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety–a review. arXiv preprint arXiv:2404.14082, 2024

2024 arXiv
[14]

B., Skjuve, M., and Følstad, A

Brandtzaeg, P. B., Skjuve, M., and Følstad, A. My AI friend: How users of a social chatbot understand their human–AI friendship. Human Communication Research, 48(3):404–429, 2022

2022
[15]

C., Burgers, C., and Steen, G

Brugman, B. C., Burgers, C., and Steen, G. J. Recategorizing political frames: a systematic review of metaphorical framing in experiments on political communication. Annals of the International Communication Association, 41(2):181–197, 2017

2017
[16]

The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence

Brynjolfsson, E. The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence. In Augmented education in the global age, pp. 103–116. Routledge, 2023

2023
[17]

K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al

Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217, 2023

2023 arXiv
[18]

Chang, S., Anderson, A., and Hofman, J. M. Chatbench: From static benchmarks to human-ai evaluation. arXiv preprint arXiv:2504.07114, 2025

2025 arXiv
[19]

A survey on evaluation of large language models.ACM Transactions on Intelligent Systems and Technology, 15(3):1–45, 2024

Chang, Y ., Wang, X., Wang, J., Wu, Y ., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y ., et al. A survey on evaluation of large language models.ACM Transactions on Intelligent Systems and Technology, 15(3):1–45, 2024. 10

2024
[20]

AnthroScore: A computational linguistic measure of anthropomorphism

Cheng, M., Gligoric, K., Piccardi, T., and Jurafsky, D. AnthroScore: A computational linguistic measure of anthropomorphism. In Graham, Y . and Purver, M. (eds.),Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume...

2024
[21]

Y ., Rapuano, K., Niederhoffer, K., Liebscher, A., and Hancock, J

Cheng, M., Lee, A. Y ., Rapuano, K., Niederhoffer, K., Liebscher, A., and Hancock, J. From tools to thieves: Measuring and understanding public perceptions of ai through crowdsourced metaphors. arXiv preprint arXiv:2501.18045, 2025

2025 arXiv
[22]

O., Moran, J

Cohn, M., Pushkarna, M., Olanubi, G. O., Moran, J. M., Padgett, D., Mengesha, Z., and Heldreth, C. Believing anthropomorphism: Examining the role of anthropomorphic cues on trust in large language models. In Extended Abstracts of the CHI Conference on Human Factors in Computin...

2024
[23]

Anthropomorphism as cognitive bias

Dacey, M. Anthropomorphism as cognitive bias. Philosophy of Science, 84(5):1152–1164, 2017

2017
[24]

Beyond personhood: Agency, accountability, and the limits of anthropomorphic ethical analysis

Dai, J. Beyond personhood: Agency, accountability, and the limits of anthropomorphic ethical analysis. arXiv preprint arXiv:2404.13861, 2024

2024 arXiv
[25]

and Khakzar, A

Davies, A. and Khakzar, A. The cognitive revolution in interpretability: From explaining behavior to interpreting representations and algorithms. arXiv preprint arXiv:2408.05859, 2024

2024 arXiv
[26]

DeVrio, A., Cheng, M., Egede, L., Olteanu, A., and Blodgett, S. L. A taxonomy of linguistic expressions that contribute to anthropomorphism of language technologies. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25. Association for Compu...

2025
[27]

Dijkstra, E. W. On anthropomorphism in science. EWD936, Sept, 1985

1985
[28]

and So, R

Duede, E. and So, R. J. The humanistic case for ai optimism. Poetics Today, 45(2):215–222, 2024

2024
[29]

A mind like mine: The exceptionally ordinary underpinnings of anthropomorphism

Epley, N. A mind like mine: The exceptionally ordinary underpinnings of anthropomorphism. Journal of the Association for Consumer Research, 3(4):591–598, 2018

2018
[30]

M., and Dereventsov, A

Fagbohun, O., Harrison, R. M., and Dereventsov, A. An empirical categorization of prompting techniques for large language models: A practitioner’s guide. arXiv preprint arXiv:2402.14837, 2024

2024 arXiv
[31]

S., Zhang, A

Feng, K., Pu, K., Latzke, M., August, T., Siangliulue, P., Bragg, J., Weld, D. S., Zhang, A. X., and Chang, J. C. Cocoa: Co-planning and co-execution with ai agents. arXiv preprint arXiv:2412.10999, 2024

2024
[32]

and Nobre, A

Floridi, L. and Nobre, A. C. Anthropomorphising machines and computerising minds: the crosswiring of languages between artificial intelligence and brain & cognitive sciences. Minds and Machines, 34(1):1–9, 2024

2024
[33]

Gallagher, K. M. and Updegraff, J. A. Health message framing effects on attitudes, intentions, and behavior: a meta-analytic review. Annals of behavioral medicine, 43(1):101–116, 2012

2012
[34]

Improving alignment of dialogue agents via targeted human judgements

Glaese, A., McAleese, N., Trkebacz, M., Aslanides, J., Firoiu, V ., Ewalds, T., Rauh, M., Weidinger, L., Chadwick, M., Thacker, P., et al. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375, 2022

2022 arXiv
[35]

Alignment faking in large language models

Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., et al. Alignment faking in large language models. arXiv preprint arXiv:2412.14093, 2024

2024 arXiv
[36]

The rua-robot dataset: Helping avoid chatbot deception by detecting user questions about human or non-human identity

Gros, D., Li, Y ., and Yu, Z. The rua-robot dataset: Helping avoid chatbot deception by detecting user questions about human or non-human identity. arXiv preprint arXiv:2106.02692, 2021. 11

2021 arXiv
[37]

Training large language models to reason in a continuous latent space

Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., and Tian, Y . Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024

2024 arXiv
[38]

F., Liang, P., and Manning, C

Hewitt, J., Liu, N. F., Liang, P., and Manning, C. D. Instruction following without instruction tuning. arXiv preprint arXiv:2409.14254, 2024

2024 arXiv
[39]

We need a science of evals, Apr 2024

Hobbhahn, M. We need a science of evals, Apr 2024. URL https://www.apolloresearch. ai/blog/we-need-a-science-of-evals

2024
[40]

Characterizing and modeling harms from interactions with design patterns in AI interfaces

Ibrahim, L., Rocher, L., and Valdivia, A. Characterizing and modeling harms from interactions with design patterns in AI interfaces. arXiv preprint arXiv:2404.11370, 2024

2024 arXiv
[41]

R., McKee, K

Ibrahim, L., Akbulut, C., Elasmar, R., Rastogi, C., Kahng, M., Morris, M. R., McKee, K. R., Rieser, V ., Shanahan, M., and Weidinger, L. Multi-turn evaluation of anthropomorphic be- haviours in large language models. arXiv preprint arXiv:2502.07077, 2025

2025
[42]

Inie, N., Druga, S., Zukerman, P., and Bender, E. M. From AI to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency , FAccT ’24, pp. 2...

2024
[43]

Jensen, T., Theofanos, M., Greene, K., Williams, O., Goad, K., and Fofang, J. B. Reflection of its creators: Qualitative analysis of general public and expert perceptions of artificial intelligence. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 7...

2024
[44]

Jones, C. R. and Bergen, B. K. Lies, damned lies, and distributional language statistics: Persuasion and deception with large language models. arXiv preprint arXiv:2412.17128, 2024

2024 arXiv
[45]

D., Potts, C., and Csordás, R

Kallini, J., Murty, S., Manning, C. D., Potts, C., and Csordás, R. Mrt5: Dynamic token merging for efficient byte-level language models. arXiv preprint arXiv:2410.20771, 2024

2024 arXiv
[46]

and Gabriel, I

Kasirzadeh, A. and Gabriel, I. In conversation with artificial intelligence: aligning language models with human values. Philosophy & Technology, 36(2):27, 2023

2023
[47]

and Mahowald, K

Kaushal, A. and Mahowald, K. What do tokens know about their characters and how do they know it? arXiv preprint arXiv:2206.02608, 2022

2022 arXiv
[48]

T., and Bernstein, M

Khadpe, P., Krishna, R., Fei-Fei, L., Hancock, J. T., and Bernstein, M. S. Conceptual metaphors impact perceptions of human-ai collaboration. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2):1–26, 2020

2020
[49]

T., Moazam, H., et al

Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., et al. Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714, 2023

arXiv
[50]

Dynabench: Rethinking benchmarking in nlp

Kiela, D., Bartolo, M., Nie, Y ., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. Dynabench: Rethinking benchmarking in nlp. arXiv preprint arXiv:2104.14337, 2021

2021 arXiv
[51]

Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing

Kudo, T. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint arXiv:1808.06226, 2018

2018 arXiv
[52]

and Johnson, M

Lakoff, G. and Johnson, M. Metaphors we live by. University of Chicago press, 2008

2008
[53]

Crafting papers on machine learning

Langley, P. Crafting papers on machine learning. In Langley, P. (ed.),Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207–1216, Stanford, CA,

2000
[54]

Z., Tamkin, A., Goodman, N., and Andreas, J

Li, B. Z., Tamkin, A., Goodman, N., and Andreas, J. Eliciting human preferences with language models. arXiv preprint arXiv:2310.11589, 2023. 12

2023 arXiv
[55]

Warmth, competence, and the determinants of trust in artificial intelligence: A cross-sectional survey from china.International Journal of Human–Computer Interaction, pp

Li, Y ., Wu, B., Huang, Y ., Liu, J., Wu, J., and Luan, S. Warmth, competence, and the determinants of trust in artificial intelligence: A cross-sectional survey from china.International Journal of Human–Computer Interaction, pp. 1–15, 2024

2024
[56]

R., Robinson, K., Nagpal, C., and D’Amour, A

Lum, K., Anthis, J. R., Robinson, K., Nagpal, C., and D’Amour, A. Bias in language models: Beyond trick tests and toward ruted evaluation. arXiv preprint arXiv:2402.12649, 2024

2024 arXiv
[57]

Dartmouth workshop

McCarthy, J., Rochester, N., and Shannon, C. Dartmouth workshop. 1956

1956
[58]

L., Rochester, N., and Shannon, C

McCarthy, J., Minsky, M. L., Rochester, N., and Shannon, C. E. A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955. AI magazine, 27(4):12–12, 2006

1955
[59]

T., Yao, S., Friedman, D., Hardy, M., and Griffiths, T

McCoy, R. T., Yao, S., Friedman, D., Hardy, M., and Griffiths, T. L. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638, 2023

2023 arXiv
[60]

T., Yao, S., Friedman, D., Hardy, M

McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., and Griffiths, T. L. When a language model is optimized for reasoning, does it still show embers of autoregression? an analysis of openai o1. arXiv preprint arXiv:2410.01792, 2024

arXiv
[61]

Medin, D. L. Concepts and conceptual structure. American psychologist, 44(12):1469, 1989

1989
[62]

State of what art? a call for multi-prompt llm evaluation

Mizrahi, M., Kaplan, G., Malkin, D., Dror, R., Shahaf, D., and Stanovsky, G. State of what art? a call for multi-prompt llm evaluation. Transactions of the Association for Computational Linguistics, 12:933–949, 2024

2024
[63]

Sometimes our anthropocentric assumptions about how intelligence "should" work (like using language for reasoning) may be holding AI work back., March 2024

Mollick, E. Sometimes our anthropocentric assumptions about how intelligence "should" work (like using language for reasoning) may be holding AI work back., March 2024. URL https://x.com/emollick/status/1866497607792525419

2024
[64]

L., Colaresi, M

Monroe, B. L., Colaresi, M. P., and Quinn, K. M. Fightin’words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis, 16(4):372–403, 2008

2008
[65]

Morris, M. R. Prompting considered harmful. Communications of the ACM, 67(12):28–30, 2024

2024
[66]

Topics, authors, and institutions in large language model research: Trends from 17k arxiv papers

Movva, R., Balachandar, S., Peng, K., Agostini, G., Garg, N., and Pierson, E. Topics, authors, and institutions in large language model research: Trends from 17k arxiv papers. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computation...

2024
[67]

H., and Hammerschmidt, M

Mozafari, N., Weiger, W. H., and Hammerschmidt, M. The chatbot disclosure dilemma: Desirable and undesirable effects of disclosing the non-human identity of chatbots. In ICIS, pp. 1–18, 2020

2020
[68]

The big book of concepts

Murphy, G. The big book of concepts. MIT press, 2004

2004
[69]

Nass, C., Steuer, J., and Tauber, E. R. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 72–78, 1994

1994
[70]

Are people polite to computers? responses to computer-based interviewing systems 1

Nass, C., Moon, Y ., and Carney, P. Are people polite to computers? responses to computer-based interviewing systems 1. Journal of applied social psychology, 29(5):1093–1109, 1999

1999
[71]

Mapping global dynamics of benchmark creation and saturation in artificial intelligence

Ott, S., Barbosa-Silva, A., Blagec, K., Brauner, J., and Samwald, M. Mapping global dynamics of benchmark creation and saturation in artificial intelligence. Nature Communications, 13(1): 6793, 2022

2022
[72]

Training language models to follow instructions with human feedback

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022

2022
[73]

Feedback loops with language models drive in-context reward hacking

Pan, A., Jones, E., Jagadeesan, M., and Steinhardt, J. Feedback loops with language models drive in-context reward hacking. arXiv preprint arXiv:2402.06627, 2024. 13

2024 arXiv
[74]

Discovering language model behaviors with model-written evaluations

Perez, E., Ringer, S., Lukoši ¯ut˙e, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. Discovering language model behaviors with model-written evaluations. arXiv preprint arXiv:2212.09251, 2022

2022 arXiv
[75]

Anthropomorphism and ai: Turing’s much misunderstood imitation game

Proudfoot, D. Anthropomorphism and ai: Turing’s much misunderstood imitation game. Artificial Intelligence, 175(5-6):950–957, 2011

2011
[76]

D., Bender, E

Raji, I. D., Bender, E. M., Paullada, A., Denton, E., and Hanna, A. Ai and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366, 2021

2021 arXiv
[77]

The impact of anthropomorphic and functional chatbot design features in enterprise collaboration systems on user acceptance

Rietz, T., Benke, I., and Maedche, A. The impact of anthropomorphic and functional chatbot design features in enterprise collaboration systems on user acceptance. 2019

2019
[78]

Talking about large language models

Shanahan, M. Talking about large language models. Communications of the ACM, 67(2):68–79, 2024

2024
[79]

Role play with large language models

Shanahan, M., McDonell, K., and Reynolds, L. Role play with large language models. Nature, 623(7987):493–498, 2023

2023
[80]

R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S

Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., et al. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548, 2023

arXiv
[81]

and Luximon, Y

Song, Y . and Luximon, Y . Trust in ai agent: A systematic review of facial anthropomorphic trustworthiness for social robot design. Sensors, 20(18):5087, 2020

2020
[82]

Bridging the gulf of envisioning: Cognitive challenges in prompt based interactions with llms

Subramonyam, H., Pea, R., Pondoc, C., Agrawala, M., and Seifert, C. Bridging the gulf of envisioning: Cognitive challenges in prompt based interactions with llms. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1–19, 2024

2024
[83]

Sui, P., Duede, E., Wu, S., and So, R. J. Confabulation: The surprising value of large language model hallucinations. arXiv preprint arXiv:2406.04175, 2024

2024 arXiv
[84]

Turing, A. M. Computing machinery and intelligence. Mind, 59(236):433–460, 1950

1950
[85]

There is a nuanced but important difference between chain-of-thought before and after o1

Wei, J. There is a nuanced but important difference between chain-of-thought before and after o1. ..., November 2024. URL https://x.com/_jasonwei/status/1855417833775309171

2024
[86]

V ., Zhou, D., et al

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V ., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

2022
[87]

Eliza—a computer program for the study of natural language communication between man and machine

Weizenbaum, J. Eliza—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1):36–45, 1966

1966
[88]

Assessing and addressing ethical risk from anthropomorphism and deception in socially assistive robots

Winkle, K., Caleb-Solly, P., Leonards, U., Turton, A., and Bremner, P. Assessing and addressing ethical risk from anthropomorphism and deception in socially assistive robots. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, pp. 101–109, 2021

2021
[89]

Llm lies: Hallucinations are not bugs, but features as adversarial examples

Yao, J.-Y ., Ning, K.-P., Liu, Z.-H., Ning, M.-N., Liu, Y .-Y ., and Yuan, L. Llm lies: Hallucinations are not bugs, but features as adversarial examples. arXiv preprint arXiv:2310.01469, 2023

2023 arXiv
[90]

Robustness of demonstration-based learning under limited data scenario

Zhang, H., Zhang, Y ., Zhang, R., and Yang, D. Robustness of demonstration-based learning under limited data scenario. arXiv preprint arXiv:2210.10693, 2022

2022 arXiv
[91]

Beyond preferences in AI alignment

Zhi-Xuan, T., Carroll, M., Franklin, M., and Ashton, H. Beyond preferences in AI alignment. Philosophical Studies, pp. 1–51, 2024

2024
[92]

Least-to-most prompting enables complex reasoning in large language models

Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625, 2022

2022 arXiv
[93]

D., Ren, X., Dziri, N., Jurafsky, D., and Sap, M

Zhou, K., Hwang, J. D., Ren, X., Dziri, N., Jurafsky, D., and Sap, M. Rel-ai: An interaction- centered approach to measuring human-lm reliance. arXiv preprint arXiv:2407.07950, 2024. 14

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Abercrombie, G., Cercas Curry, A., Dinkar, T., Rieser, V ., and Talat, Z. Mirages. on anthropo- morphism in dialogue systems. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of 9 the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4776–4790, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18...

doi:10.18653/v1/2023 2023

[2] [2]

R., Smith, N

Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., and Tsvetkov, Y . Do all languages cost the same? tokenization in the era of commercial language models. arXiv preprint arXiv:2305.13707, 2023

arXiv 2023

[3] [3]

All Too Human? Map- ping and Mitigating the Risk from Anthropomorphic AI

Akbulut, C., Weidinger, L., Manzini, A., Gabriel, I., and Rieser, V . All Too Human? Map- ping and Mitigating the Risk from Anthropomorphic AI. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 7, pp. 13–26, 2024

2024

[4] [4]

Language models as agent models

Andreas, J. Language models as agent models. arXiv preprint arXiv:2212.01681, 2022

arXiv 2022

[5] [5]

Claude’s character, Jun 2024

Anthropic. Claude’s character, Jun 2024. URL https://www.anthropic.com/research/ claude-character

2024

[6] [6]

alignment faking in large language models

Anthropic. External reviews of “alignment faking in large language models”, Dec 2024. URL https://assets.anthropic.com/m/24c8d0a3a7d0a1f1/original/ Alignment-Faking-in-Large-Language-Models-reviews.pdf

2024

[7] [7]

arxiv dataset, 2024

arXiv.org submitters. arxiv dataset, 2024. URL https://www.kaggle.com/dsv/7548853

arXiv 2024

[8] [8]

Training a helpful and harmless assistant with reinforcement learning from human feedback

Bai, Y ., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

arXiv 2022