Pith. sign in

REVIEW 4 major objections 6 minor 40 references

From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that prompting an LLM as a Dutch, Chinese, or English speaker changes both the psycholinguistic judgments it produces and how its internal layers encode sound-symbolic and valence information, so these models are not…

desk verdict Real deterministic prompt effects, but the 'language identity' framing is undercut by a design that never isolates identity from prompt or stimulus language. read the letter →

arxiv 2508.02502 v1 pith:MVIULKTL submitted 2025-08-04 cs.CL

Shuzhou Yuan , Zhan Qu , Mario Tawfelis , Michael Färber This is my paper
classification cs.CL
keywords psycholinguisticssoundsymbolismwordvalencelanguageconditioningmultilingualLLMsBouba-Kikieffectprobinganalysiscross-linguisticcognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models behave like language-specific minds when told to adopt a linguistic identity. Using two psycholinguistic tasks, sound symbolism (round versus spiky pseudowords) and word valence (positive versus negative real words), the authors prompt Llama-3.3-70B and Qwen2.5-72B as monolingual or bilingual speakers of English, Dutch, or Chinese. The central claim is that language identity conditions both output behavior and internal representations: the same model changes its judgments and its layer-by-layer decodability depending on the prompted language. A reader should care because it bears on whether LLMs can serve as models of cross-linguistic human cognition, and because it shows that assumptions of language-neutral LLM behavior are false.

What carries the argument

The central mechanism is prompt-based language conditioning: a system prompt assigns the model a persona such as 'You are a native speaker of Dutch and do not speak any other language' or 'bilingual speaker of English and Mandarin Chinese', with the user message written in the matching language. Two psycholinguistic tasks carry the evaluation: 648 pseudowords with round/spiky shape labels (the Bouba–Kiki effect) and 1,034 real words translated from the ANEW valence norms into Dutch and Chinese, with Pinyin provided for Chinese. Two discrepancy metrics, $D_m$ and $D_l$, turn raw accuracy into measures of how much the prompted language moves behavior. For internal evidence, the paper trains a frozen MLP probe on hidden states at eight layers of the 80-layer Llama model and compares how decodable sound-symbolic and valence labels are across language conditions.

What would settle it

Collect native-speaker valence and sound-shape norms from Dutch and Mandarin participants for the same 648 pseudowords and 1,034 translated words, and compare the models' Dutch- and Chinese-prompted judgments against those native norms. If Dutch-prompted outputs align better with English norms than with Dutch human ratings, the claim that language identity changes psycholinguistic cognition in the model would be weakened.

Watch

Extended reading notes

Core claim

On the paper's own terms, LLMs are not language-neutral. In the sound symbolism task, both models classify pseudowords differently when prompted as Dutch, Chinese, or English speakers, and in the word valence task they produce divergent positive/negative judgments for the same translated words. The behavioral differences are quantified by two discrepancy metrics: $D_m$ compares bilingual versus monolingual prompting, and $D_l$ compares Chinese versus Dutch prompts. Layer-wise probing of Llama-3.3-70B on hidden states from every tenth layer shows that psycholinguistic signals become more linearly decodable in deeper layers, that bilingual prompts can delay this emergence, and that Chinese prompts yield stronger, more stable valence representations than Dutch prompts. The authors read these results as evidence that prompt-based language conditioning modulates both output and internal encoding, not merely surface response style.

Load-bearing premise

The load-bearing assumption is that the English-derived labels are a valid reference for the Dutch and Chinese conditions; if native Dutch or Chinese speakers would rate these pseudowords and translated words differently than English speakers do, the reported accuracy scores measure deviation from English norms rather than alignment with the prompted language.

Editorial extensions

If this is right

  • Prompt language should be treated as a controlled variable in psycholinguistic LLM studies, because a single English-only accuracy score does not describe the model's behavior under other linguistic identities.
  • Monolingual prompts produce earlier and more stable sound-symbolism representations than bilingual prompts in Llama, so bilingual conditioning can delay or diffuse phonological signal in the network.
  • Chinese prompts yield stronger and more stable valence probing accuracy than Dutch prompts, so valence encoding is language-dependent even when the ground-truth labels are English-derived.
  • The English-centric Llama is more stable across languages on the behavioral tasks, while the multilingual Qwen shows larger and sometimes reversed language effects, so training-language coverage is a plausible driver of psycholinguistic sensitivity.
  • Bilingual prompting can either improve or degrade alignment with English labels depending on model, language, and task, as seen in Qwen's large Task-2 gain for Dutch and its loss for Chinese.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that any cross-lingual LLM evaluation fixing one language's human norms as ground truth confounds language conditioning with norm divergence.
  • A natural testable extension is to collect native Dutch and Mandarin norms and check whether models prompted in those languages track those norms better than they track English norms.
  • The layer-wise curves predict that languages whose phonology and script differ sharply from English should alter early-layer decodability trajectories more strongly, a pattern that could be checked in Arabic, Hindi, or Japanese.
  • Because the paper notes that exact-match scoring penalized synonymous answers such as 'joyful' for 'positive', part of Qwen's Chinese-bilingual instability may be an evaluation artifact; a soft-match rerun would isolate the representational effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates whether LLMs exhibit human-like psycholinguistic responses under different linguistic identities by evaluating Llama-3.3-70B-Instruct and Qwen2.5-72B-Instruct on two tasks: sound symbolism (round/spiky judgments on pseudowords) and word valence (positive/negative judgments on translated ANEW words). The authors compare monolingual (Dutch, Chinese, English) and bilingual (Dutch-English, Chinese-English) prompting conditions, reporting behavioral accuracy relative to English-derived ground truth and layer-wise probing accuracy on the hidden states of Llama (and Qwen in the appendix). The central claim is that 'language identity conditions both output behavior and internal representations in LLMs,' with Qwen showing stronger language sensitivity and Chinese prompts yielding more robust valence representations than Dutch.

Significance. If the central claim were established, the paper would contribute useful evidence about how LLMs simulate cross-linguistic psycholinguistic behavior and whether prompt-based persona conditioning affects not just outputs but also internal encodings. Strengths include deterministic generation at temperature 0, the use of two model families with contrasting multilingual training profiles, a layer-wise probing design with hyperparameters in Appendix C, and an explicit acknowledgment in the Table 2 note that English-based labels are a reference rather than an absolute standard. However, the design confounds language identity with the surface language of the prompt and the stimulus, so the observed differences cannot currently be attributed to the assigned linguistic identity. The significance is accordingly contingent on additional control conditions or a substantially weakened interpretation.

major comments (4)
  1. [§4.2, Appendix A] 'Language identity' is never manipulated independently of the surface language of the prompt and the stimulus. In the monolingual condition both system and user prompts are written in the target language, and in the bilingual condition the system prompt is English while the user prompt is in the target language; the valence task additionally presents pinyin words to the Dutch persona and Dutch words to the Chinese persona. Consequently Tables 2–4 and Figures 3–4 cannot attribute the observed differences to the assigned linguistic identity as opposed to the language of the instruction text or the language of the stimulus word. The central abstract claim requires at least one condition that varies the persona while holding the task text and stimulus language fixed, or a full crossing of persona language and stimulus language.
  2. [§3.2, Appendix A] In Task 2 the persona language and the stimulus language are perfectly confounded. A Dutch persona always judges a romanized Chinese word (e.g., niao) and a Chinese persona always judges a Dutch word (e.g., vogel), so the large Dl values in Table 4 (e.g., +30.50 for Llama monolingual, −41.76 for Qwen bilingual) may reflect stimulus-language effects rather than identity effects. Without crossing persona language with stimulus language, the qualitative claims in Section 4.4 about language conditioning are underdetermined.
  3. [§4.3, Tables 2–4] All behavioral accuracy scores are computed against English-derived ground truth (ANEW norms and Alper and Averbuch-Elor pseudoword labels). The paper acknowledges in the Table 2 note and Section 4.3 that these scores 'should not be interpreted as absolute accuracy,' but Dm and Dl are still interpreted as alignment shifts. If native Dutch or Chinese human judgments differ from the English labels, these discrepancies measure divergence from an English reference, not psycholinguistic alignment. The paper should report raw response distributions (e.g., proportion 'positive' or 'round' per condition) or use language-specific human norms, and it should at least quantify how many of the observed differences survive when the evaluation is rerun with an alternative reference.
  4. [§4.3, Tables 3–4] The discrepancies Dm and Dl are reported without any measure of uncertainty. Although generation is deterministic at temperature 0, the estimates are computed over 648 and 1034 items, so binomial confidence intervals or bootstrap intervals are needed before claims such as 'Qwen's behavior reverses' (Section 4.3.2) or '+46.5%' (Table 3) can be evaluated. The probing results in Figures 3–6 likewise lack error bars or significance tests across random restarts of the probe.
minor comments (6)
  1. [§4.3.1] The paper notes that synonymous responses such as 'joyful' instead of 'positive' are penalized, but it does not quantify how many responses are affected or provide an alternative evaluation (e.g., semantic equivalence matching). This could systematically lower accuracy in specific conditions and should be reported.
  2. [Figure 2] The English translation in the valence example reads 'The word is bird,' which does not correspond to the Dutch/Chinese words shown; this appears to be a typo and should be corrected to reflect the actual stimulus (e.g., 'niao' or 'vogel').
  3. [§4.4, Table 5] The qualitative claim that nasal-initial words are perceived more positively in Chinese and more negatively in Dutch is not consistently supported by Table 5; for example, 'fear' (hai pa) is labeled negative in Chinese and 'problem' (ma fan) is labeled negative in Chinese despite the nasal onset. The text should either present a quantitative test of the nasal-onset hypothesis or temper the generalization.
  4. [§1] The statement that translations were 'manually verified and cross-checked' is not accompanied by any procedure or inter-annotator detail; a brief description of the verification process would strengthen reproducibility.
  5. [Appendix C] The probe hyperparameters are listed, but there is no mention of how many random seeds were used or whether the reported probing accuracies are averaged over seeds; this is needed because probing accuracy can vary with initialization.
  6. [Limitations] The limitations listed (three languages, two models, narrow task set) do not include the English-based ground truth or the prompt-language confound; these should be explicitly acknowledged, or the claims in the abstract should be weakened accordingly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the behavioral and probing results are evaluated against external human benchmarks, and no central claim reduces to fitted inputs or self-citation.

full rationale

The paper's derivation chain is empirical rather than formal: behavioral accuracy is scored against two external English-derived human benchmarks, ANEW norms for valence (Bradley and Lang, 1999) and Alper and Averbuch-Elor's pseudoword shape labels for sound symbolism. The discrepancy measures Dm and Dl are arithmetic differences of these externally anchored accuracies and are explicitly framed as shifts in alignment relative to a fixed reference, not as predictions derived from the model's own outputs. The probing analysis trains an MLP on the same labels, but this is a standard supervised probing procedure; the representational claim rests on layer-wise generalization to held-out test data and on cross-condition comparisons, not on the training labels being treated as a predicted outcome. The paper's citation of Yuan et al. (2025) for persona prompting is not load-bearing: the prompt templates are fully specified in the paper and the manipulation is transparent, and no uniqueness theorem or ansatz is imported from that work. The acknowledged limitations, including the English-derived ground truth and the confound between prompt language and stimulus word language in Task 2, are validity or scope concerns rather than circularity: they do not make any result true by definition or by fitting. No equation reduces to an input by construction, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claims are empirical measurements, not derivations. No fitted parameters are used to produce the reported accuracies. The domain assumptions listed above are load-bearing: the English ground truth, the interpretation of probing accuracy as evidence of internal encoding, and the equivalence between persona prompts and simulated linguistic identity.

free parameters (1)
  • probe architecture and training schedule = MLP 512->256->2, ReLU, Adam, 200 epochs, 80/20 split
    Chosen by hand for probing; standard and not fitted to the central claim, but the choice can affect decodability estimates.
assumptions (3)
  • domain assumption English-based labels from ANEW and the sound symbolism dataset are valid ground truth for evaluating responses in Dutch and Chinese.
    All accuracy metrics in Tables 2-4 compare model outputs to English human judgments. The paper acknowledges this but does not provide Dutch/Chinese human norms.
  • domain assumption High probing accuracy implies that psycholinguistic information is encoded in model representations.
    The paper interprets layer-wise probe accuracy as evidence of internal encoding without control tasks such as random-label baselines.
  • domain assumption Prompting the model with a 'native speaker' persona simulates a linguistic identity.
    The paper assumes persona prompts activate language-specific behavior; prior work (Yuan et al. 2025) supports this but it remains an assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks." pith.science (2026). https://pith.science/paper/MVIULKTL

@misc{pith2026250802502,
  author       = {Pith},
  title        = {Pith review of: From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MVIULKTL}},
  note         = {Machine review of arXiv:2508.02502}
}
read the original abstract

Large Language Models (LLMs) exhibit strong linguistic capabilities, but little is known about how they encode psycholinguistic knowledge across languages. We investigate whether and how LLMs exhibit human-like psycholinguistic responses under different linguistic identities using two tasks: sound symbolism and word valence. We evaluate two models, Llama-3.3-70B-Instruct and Qwen2.5-72B-Instruct, under monolingual and bilingual prompting in English, Dutch, and Chinese. Behaviorally, both models adjust their outputs based on prompted language identity, with Qwen showing greater sensitivity and sharper distinctions between Dutch and Chinese. Probing analysis reveals that psycholinguistic signals become more decodable in deeper layers, with Chinese prompts yielding stronger and more stable valence representations than Dutch. Our results demonstrate that language identity conditions both output behavior and internal representations in LLMs, providing new insights into their application as models of cross-linguistic cognition.

Figures

Figures reproduced from arXiv: 2508.02502 by the authors.

Figure 1
Figure 1. The Bouba-Kiki effect illustrates that people [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of two psycholinguistic tasks. (a) [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Llama Layer-wise probing accuracy for task [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Llama Layer-wise probing accuracy for task [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Qwen Layer-wise probing accuracy for task [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 5
Figure 5. Figure 5: Qwen Layer-wise probing accuracy for task [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 21 canonical work pages

  1. [1]

    Ekaterina Abramova and Raquel Fern \'a ndez. 2016. https://doi.org/10.18653/v1/N16-1038 Questioning arbitrariness in language: a data-driven study of conventional iconicity . In Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 343--352, San Diego, Califo...

    work page doi:10.18653/v1/n16-1038 2016

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    Adelman, Zachary Estes, and Martina Cossu

    James S. Adelman, Zachary Estes, and Martina Cossu. 2018. https://doi.org/10.1016/j.cognition.2018.02.007 Emotional sound symbolism: Languages rapidly signal valence via phonemes . Cognition, 175:122--130

    work page doi:10.1016/j.cognition.2018.02.007 2018

  4. [4]

    Morris Alper and Hadar Averbuch-Elor. 2023. Kiki or bouba? sound symbolism in vision-and-language models. Advances in Neural Information Processing Systems, 36:78347--78359

    work page 2023

  5. [5]

    Samuel Amouyal, Aya Meltzer-Asscher, and Jonathan Berant. 2024. https://aclanthology.org/2024.findings-eacl.12/ Large language models for psycholinguistic plausibility pretesting . In Findings of the Association for Computational Linguistics: EACL 2024, pages 166--181, St. Julian ' s, Malta. Association for Computational Linguistics

    work page 2024

  6. [6]

    Arnav Arora, Lucie-aim \'e e Kaffee, and Isabelle Augenstein. 2023. https://doi.org/10.18653/v1/2023.c3nlp-1.12 Probing pre-trained language models for cross-cultural differences in values . In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), pages 114--130, Dubrovnik, Croatia. Association for Computational Linguistics

    doi:10.18653/v1/2023.c3nlp-1.12 2023

  7. [7]

    Maxim Bazhukov, Ekaterina Voloshina, Sergey Pletenev, Arseny Anisimov, Oleg Serikov, and Svetlana Toldova. 2024. https://doi.org/10.18653/v1/2024.conll-1.22 Of models and men: Probing neural networks for agreement attraction with psycholinguistic data . In Proceedings of the 28th Conference on Computational Natural Language Learning, pages 280--290, Miami...

    doi:10.18653/v1/2024.conll-1.22 2024

  8. [8]

    Dami \'a n E Blasi, S ren Wichmann, Harald Hammarstr \"o m, Peter F Stadler, and Morten H Christiansen. 2016. Sound--meaning association biases evidenced across thousands of languages. Proceedings of the National Academy of Sciences, 113(39):10818--10823

    work page 2016

Show all 40 references
  1. [9]

    Roberto Bottini, Marco Barilari, and Olivier Collignon. 2019. https://doi.org/10.1016/j.cognition.2019.01.006 Sound symbolism in sighted and blind. the role of vision and orthography in sound-shape correspondences . Cognition, 185:62--70

    2019 doi

  2. [10]

    Margaret M Bradley and Peter J Lang. 1999. Affective norms for english words (anew): Instruction manual and affective ratings. Technical report, Technical report C-1, the center for research in psychophysiology …

    1999

  3. [11]

    Javier Conde, Miguel Gonz \'a lez, Mar \' a Grandury, Gonzalo Mart \' nez, Pedro Reviriego, and Mar Brysbaert. 2025. Psycholinguistic word features: a new approach for the evaluation of llms alignment with humans. arXiv preprint arXiv:2506.22439

    2025 arXiv

  4. [12]

    Yan Cong. 2022. https://doi.org/10.18653/v1/2022.csrr-1.3 Psycholinguistic diagnosis of language models' commonsense reasoning . In Proceedings of the First Workshop on Commonsense Representation and Reasoning (CSRR 2022), pages 17--22, Dublin, Ireland. Association for Computa...

    2022 doi

  5. [13]

    Mark Dingemanse, Will Schuerman, Eva Reinisch, Sylvia Tufvesson, and Holger Mitterer. 2016. What sound symbolism can and cannot do: Testing the iconicity of ideophones from five languages. Language, 92(2):e117--e133

    2016

  6. [14]

    Xufeng Duan, Xinyu Zhou, Bei Xiao, and Zhenguang Cai. 2025. https://aclanthology.org/2025.coling-main.677/ Unveiling language competence neurons: A psycholinguistic approach to model interpretability . In Proceedings of the 31st International Conference on Computational Lingui...

    2025

  7. [15]

    Mathilde Fort and Jean-Luc Schwartz. 2022. Resolving the bouba-kiki effect enigma by rooting iconic sound symbolism in physical properties of round and spiky objects. Scientific reports, 12(1):19172

    2022

  8. [16]

    Mario Giulianelli, Luca Malagutti, Juan Luis Gastaldi, Brian DuSell, Tim Vieira, and Ryan Cotterell. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1032 On the proper treatment of tokenization in psycholinguistics . In Proceedings of the 2024 Conference on Empirical Methods...

    2024 doi

  9. [17]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  10. [18]

    Thilo Hagendorff, Ishita Dasgupta, Marcel Binz, Stephanie CY Chan, Andrew Lampinen, Jane X Wang, Zeynep Akata, and Eric Schulz. 2023. Machine psychology. arXiv preprint arXiv:2303.13988

    2023 arXiv

  11. [19]

    Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021. https://doi.org/10.18653/v1/2021.naacl-main.301 Discourse probing of pretrained language models . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan...

    2021 doi

  12. [20]

    Eun-Kyoung Rosa Lee, Sathvik Nair, and Naomi Feldman. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.186 A psycholinguistic evaluation of language models' sensitivity to argument roles . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 326...

    2024 doi

  13. [21]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

    2024 arXiv

  14. [22]

    Zizhou Liu, Ziwei Gong, Lin Ai, Zheng Hui, Run Chen, Colin Wayne Leach, Michelle R Greene, and Julia Hirschberg. 2025. The mind in the machine: A survey of incorporating psychological theories in llms. arXiv preprint arXiv:2505.00003

    2025

  15. [23]

    Tyler Loakman, Yucheng Li, and Chenghua Lin. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.167 With ears to see and eyes to hear: Sound symbolism experiments with multimodal large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Langu...

    2024 doi

  16. [24]

    Max Louwerse and Zhan Qu. 2017. Estimating valence from the sound of a word: Computational, experimental, and cross-linguistic evidence. Psychonomic Bulletin & Review, 24:849--855

    2017

  17. [25]

    Max M Louwerse. 2008. Embodied relations are encoded in language. Psychonomic Bulletin & Review, 15(4):838--844

    2008

  18. [26]

    Sridhar Moorthy, Ruth Pogacar, Samin Khan, and Yang Xu. 2018. https://doi.org/10.18653/v1/D18-1142 Is N ike female? exploring the role of sound symbolism in predicting brand name gender . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing...

    2018 doi

  19. [27]

    a rber, Frauke Kreuter, and Hinrich Sch \

    Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid, Michael F \"a rber, Frauke Kreuter, and Hinrich Sch \"u tze. 2024. Decomposed prompting: Unveiling multilingual linguistic structure knowledge in english-centric large language models. arXiv preprint arXiv:2402.18397

    2024

  20. [28]

    Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung. 2021. https://doi.org/10.18653/v1/2021.acl-long.329 Probing toxic content in large pre-trained language models . In Proceedings of the 59th Annual Meeting of the Association for Computational Lingui...

    2021 doi

  21. [29]

    Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020. https://doi.org/10.18653/v1/2020.acl-main.420 Information-theoretic probing for linguistic structure . In Proceedings of the 58th Annual Meeting of the Association for Co...

    2020 doi

  22. [30]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. https://arxiv.org/abs/2412.15115 Qwe...

    2025 arXiv

  23. [31]

    Sarthak Roy, Ashish Harshvardhan, Animesh Mukherjee, and Punyajoy Saha. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.407 Probing LLM s for hate speech detection: strengths and vulnerabilities . In Findings of the Association for Computational Linguistics: EMNLP 2023, ...

    2023 doi

  24. [32]

    Namrata Shivagunde, Vladislav Lialin, and Anna Rumshisky. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.130 Larger probes tell a different story: Extending psycholinguistic datasets via in-context learning . In Proceedings of the 2023 Conference on Empirical Methods in Nat...

    2023 doi

  25. [33]

    Sean Trott. 2024. Can large language models help augment english psycholinguistic datasets? Behavior Research Methods, 56(6):6082--6100

    2024

  26. [34]

    Tessa Verhoef, Kiana Shahrasbi, and Tom Kouwenhoven. 2024. https://doi.org/10.18653/v1/2024.cmcl-1.17 What does kiki look like? cross-modal associations between speech sounds and visual shapes in vision-and-language models . In Proceedings of the Workshop on Cognitive Modeling...

    2024 doi

  27. [35]

    Ivan Vuli \'c , Edoardo Maria Ponti, Robert Litschko, Goran Glava s , and Anna Korhonen. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.586 Probing pretrained language models for lexical semantics . In Proceedings of the 2020 Conference on Empirical Methods in Natural Langu...

    2020 doi

  28. [36]

    Hetong Wang, Pasquale Minervini, and Edoardo Ponti. 2024. https://doi.org/10.18653/v1/2024.findings-acl.724 Probing the emergence of cross-lingual alignment during LLM training . In Findings of the Association for Computational Linguistics: ACL 2024, pages 12159--12173, Bangko...

    2024 doi

  29. [37]

    o ksal, and Hinrich Sch \

    Leonie Weissweiler, Valentin Hofmann, Abdullatif K \"o ksal, and Hinrich Sch \"u tze. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.746 The better your syntax, the better your semantics? probing pretrained language models for the E nglish comparative correlative . In Proce...

    2022 doi

  30. [38]

    u tze, and Michael F \

    Shuzhou Yuan, Ercong Nie, Mario Tawfelis, Helmut Schmid, Hinrich Sch \"u tze, and Michael F \"a rber. 2025. Hateful person or hateful model? investigating the role of personas in hate speech detection by large language models. arXiv preprint arXiv:2506.08593

    2025 arXiv

  31. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.