REVIEW 4 major objections 4 minor 1 cited by
Localizing Persona Representations in LLMs
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Persona-related information in pre-trained decoder-only LLMs is encoded almost entirely in the final third of the model's layers, and in the last layer ethical perspectives share many activations while political ideologies occupy distinct…
desk verdict Genuinely new persona-localization results, but the polysemy claim rests on an unbaselined overlap statistic and an in-sample validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is Deep Scan's salient activation set: a subset $S^* = X_S^* \times O_S^*$ of sentences and embedding positions that maximizes a non-parametric scan statistic $F(S)$ measuring how much the observed $p$-values of test activations deviate from the null expectation built from NOTMATCHINGBEHAVIOR statements. The output $O_S^*$ — the minimal set of embedding dimensions that carries the persona signal — is what the paper uses to quantify overlap between personas and to show that a downstream classifier needs no more than these positions to detect matching behavior. Layer selection (Q1) is carried by PCA plus cluster-separation scores (Silhouette, Calinski-Harabasz, Davies-Bouldin, and centroid distance) computed on the convex hulls of the two groups.
What would settle it
Take the same fourteen personas but use statements written and labeled by humans (or independently validated against human judgments), run the same PCA and Deep Scan pipeline, and check whether the final-third separation and the ethics-overlap/politics-distinct pattern reappear; if separation drops or overlap patterns change, the conclusions are tied to the model-generated dataset. A second, sharper test: train the classifier on the detected salient activations, then probe it on out-of-distribution persona statements from a different generator — chance-level performance would show the localizations do not generalize.
Extended reading notes
Core claim
The paper's central claim is that persona representations in large language models are concentrated in the last third of decoder layers, and that at the finest granularity different persona types are encoded differently within the last layer. Using principal-component separation metrics (Silhouette, Calinski-Harabasz, Davies-Bouldin, and centroid Euclidean distance), the authors show that matching vs. non-matching persona statements become separable only in layers roughly 20 through 31, with Llama3-8B-Instruct's layer 31 giving the strongest separation. Then, applying Deep Scan — a non-parametric scan statistic that searches for subsets of activation positions where a persona's statements are anomalous relative to a null distribution — they extract the small set of 'salient activations' for each persona. In Llama3's last layer these sets overlap far more for ethics personas (719 of 4096 positions shared) than for politics personas (386 shared, with 84 to 227 unique per persona), which the authors interpret as polysemy for ethical concepts and relatively dedicated locations for political ideologies. They validate the subsets by showing a classifier trained only on the salient activations matches the accuracy of one using the full embedding.
Load-bearing premise
The whole analysis assumes the model-generated persona statements and their MATCHINGBEHAVIOR/NOTMATCHINGBEHAVIOR labels actually capture the target human traits and beliefs; if those labels reflect lexical patterns or stereotypes of the generating model rather than genuine persona content, the measured layer separation and activation overlap describe the dataset, not how LLMs encode human values.
Editorial extensions
If this is right
- Interventions that edit a persona (e.g., shifting a model's political stance) need only target the final third of layers, not the whole network, because that is where the separable signal lives.
- For political personas, the largely unique salient activations mean fine-grained steering of one ideology is possible without strongly disturbing the others.
- For ethical personas, the heavy activation overlap implies that changing or removing one ethical perspective will likely bleed into neighboring ethical perspectives sharing those positions.
- The salient activation subset is enough to detect matching persona behavior with the same accuracy as the full 4096-dimensional embedding, so downstream monitoring or control can operate on far fewer dimensions.
- Because the pattern appears across Llama3, Granite, and Mistral, the late-layer concentration is probably a shared architectural tendency of instruction-tuned decoder-only transformers, not a quirk of one model.
Reading between the lines
- If the polysemanticity finding is real, attempts to debias or 'unlearn' a specific ethical stance (e.g., moral nihilism) may accidentally suppress utilitarianism or virtue-ethics content that shares those activations; a direct test is to ablate the shared positions and measure behavioral change on all ethics personas.
- The paper locates correlations, not causes: the salient activations could be byproducts of lexical regularities in the model-written statements (e.g., words like 'immigration' clustering together) rather than a semantic encoding of the underlying ideology; this could be tested by controlling for n-gram overlap between persona datasets.
- The method invites a causal follow-up: edit or clamp the values of the identified salient activations at layer 31 and measure whether persona-steered outputs actually change — that would turn a localization claim into an intervention claim.
- The politics-versus-ethics asymmetry could also be an artifact of dataset construction: political statements may be more lexically stereotyped than ethical statements, inflating the apparent uniqueness of political activations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies how and where "personas" (personality traits, ethical theories, political views) are encoded in the internal representations of decoder-only LLMs. Using the model-generated persona dataset of Perez et al. [3], the authors extract last-token activation vectors from Llama3-8B-Instruct, Granite-7B-Instruct, and Mistral-7B-Instruct. For Q1, they apply PCA and four clustering/separation metrics per layer to compare MATCHINGBEHAVIOR and NOTMATCHINGBEHAVIOR statements, concluding that persona-related separability is strongest in the final third of layers and, for Llama3, in the final layer. For Q2, they apply the Deep Scan subset-scan method to Llama3's last layer to identify a small set of "salient" activations for each persona, validate these sets with precision/recall on sentence detection, and then report overlap statistics among personas. They interpret the large shared activation set among ethics personas (17.55% of 4096 activations) as polysemy, and the larger unique sets among political personas (2.05--5.54% unique) as evidence of more distinct localization. The paper closes with limitations, an impact statement, and ethical considerations.
Significance. If the results were fully supported, the paper would make a useful interpretability contribution by localizing persona information to specific layers and specific activation subsets, with potential implications for targeted representation editing and for understanding polysemanticity in LLMs. The study has concrete strengths: it uses an external, public persona dataset; it covers three model families for the layer-level analysis; it uses multiple complementary separation metrics; and it validates the selected activation subsets by showing that they support high-precision sentence classification (Table 2, Table 8). The main quantitative claims about activation overlap, however, are not currently supported because the overlap statistics are never compared to a null model, the cardinalities of the salient sets are incomplete and internally inconsistent between tables, and the headline "polysemy vs. distinct localization" conclusion therefore rests on an unvalidated comparison. With additional null-model analysis and corrected reporting, the core approach could support a defensible version of the paper's claims.
major comments (4)
- [Section 5 (Q2), Figures 1c and 3] The overlap and unique-activation statistics are never compared to a null baseline. Deep Scan selects a set O_S* per persona using that persona's own NOTMATCHINGBEHAVIOR statements as H0, so the sets are not independent random subsets, and the intersection size depends critically on the sizes |O_S*|, which are not reported for the Ethics and Politics personas (Table 4 reports only three Personality personas, and Table 8 reports only five). For two independent random subsets of size s of a 4096-dimensional vector, the expected intersection is s^2/4096; for five sets of size roughly 0.5*4096, the expected all-five intersection is about 128. The observed Personality intersection of 312 (7.62%) is only about 2.4 times this crude expectation, and the Ethics figure of 719 (17.55%) cannot be assessed without the null distribution. In addition, the "unique activation" counts are complements of the union, so they shrink as set sizes grow; the Politics "distinctness" could simply reflect smaller salient sets. Please provide a null model (e.g., random subsets matched in cardinality, or a permutation of persona labels) and report |O_S*| for all 14 personas.
- [Appendix Table 4 vs. Table 8] The reported cardinalities of the Deep Scan salient sets for Llama3 layer 31 are internally inconsistent. Table 4 reports AGREE |O_S*| = 1167, CONSC = 1210, and OPEN = 1177, while Table 8 reports AGREE = 2210, CONSC = 2692, and OPEN = 2494 (additionally listing EXTRA = 1721 and NEURO = 2038). Since |O_S*| is the key quantity for interpreting the overlap percentages in Q2, this inconsistency must be resolved before the overlap analysis can be evaluated. The reader currently cannot reconstruct the Ethics and Politics overlap numbers, and the discrepancy also undermines confidence in the reported precision/recall values, even though those values are otherwise identical across the two tables for the overlapping rows.
- [Tables 5--7 and Section 4.1] Several entries report exactly zero standard deviation across supposedly independent random subsamples. For example, Table 5 lists Llama3 layer 31 AGREE as SH = 0.792 ± 0.0000, CH = 3264.5 ± 0.002, ED = 27.57 ± 0.000, and CONSC as SH = 0.813 ± 0.0000, CH = 4150.4 ± 0.003, DB = 0.285 ± 0.000. The text states that each of five runs uses q+ = q- = 100 randomly sampled data points, so exact zero variance in continuous metrics is implausible unless the random state is fixed or the metrics are computed on the full data rather than on the subsamples. The authors should clarify the subsampling protocol, report the actual variance, or remove the claim of five independent runs. As written, the variance information in Q1 is not reliable.
- [Section 3.2] The study treats the model-generated MATCHINGBEHAVIOR and NOTMATCHINGBEHAVIOR labels from [3] as ground truth for persona content without independent validation. These labels were produced by an LLM, so the localization results may reflect lexical regularities, prompt artifacts, or the generating model's stereotypes about the persona rather than how LLMs encode genuine human values, beliefs, and traits. The assumption is explicitly acknowledged but not tested. Since this assumption is load-bearing for the interpretation of both Q1 and Q2, I ask for at least one robustness check, such as a human-annotated subset, comparison against an independently constructed persona dataset, or a label-shuffling baseline, or, failing that, a clear restatement of the conclusions as being about the model-generated persona statements rather than about human value representations.
minor comments (4)
- [Section 4.2 vs. Table 2] The level labels are reversed between the text and Table 2. Section 4.2 defines Level 1 as intra-topic (persona vs. other personas within the same topic) and Level 0 as inter-topic (topic vs. other topics), but Table 2 labels Level 1 as "Inter-Topic" and Level 0 as "Intra-Topic." Please make the labels consistent.
- [Section 5 (Q1)] The text states that the first layer is denoted 0 and the last layer is 31, but Table 1 and several tables use "layer 1" for the early layer. Please reconcile the indexing (e.g., write "layer 1 (index 0)" or use a single convention throughout).
- [Abstract] The abstract says "across multiple pre-trained decoder-only LLMs" immediately before the overlapping-activations observation, but the Q2 analysis is performed only on Llama3's last layer. Please qualify the abstract's wording so that the layer-overlap and polysemy claims are not read as established across all three model families.
- [Section 5 (Q2), Table 2] Several Level 1 entries have very large standard deviations relative to their means (e.g., Level 0 Politics precision 0.885 ± 0.207 and Level 1 UTILI recall 0.4997 ± 0.1916). Given that these are averaged over 100 Deep Scan runs, the high variance should be discussed or the affected conclusions should be qualified.
Circularity Check
No significant circularity; the central empirical measurements are not fitted inputs renamed as predictions.
full rationale
The paper's central claims are descriptive measurements on an external persona dataset (Perez et al. [3]) and three external model families. The layer-localization result (Q1) is a direct report of PCA separation metrics computed across layers, not a parameter fitted to a target and then reused to predict that same target. The activation-overlap result (Q2) is computed by taking Deep Scan's per-persona salient activation sets OS* and intersecting them; Deep Scan is prior work by some of the present authors, but the paper validates the usefulness of OS* with precision/recall on random test samples (Table 2) and compares Deep Scan against LOF, Isolation Forest, and KMeans (Table 8), so the method does not rest solely on self-citation. No equation in the paper reduces a claimed prediction to an input by construction, and no fitted constant is renamed as a finding. The absence of a null baseline for interpreting the overlap percentages is a statistical-correctness concern about whether the numbers support the polysemy/localization interpretation, not a circularity in the derivation chain. The LLM-generated labels are an external data limitation and do not make the paper's own analysis circular, since the paper does not fit anything to the labels and then predict the same labels.
Assumptions & free parameters
free parameters (4)
- Per-persona data inclusion thresholds =
label confidence >= 0.85; count >= 300 per direction; 600 examples per dataset
- Q1 random subsample size =
q+ = q- = 100 per run; 5 runs
- Number of principal components =
not fully specified (first 3 used in plots; metrics likely computed in PC space)
- Deep Scan NPSS hyperparameters =
not specified
assumptions (4)
- domain assumption Persona statements and labels from model-written evaluations [3] accurately represent the target human values, beliefs, and traits.
- domain assumption The last-token hidden state summarizes the whole sentence for persona information.
- domain assumption Deep Scan NPSS identifies the most persona-relevant activation subsets.
- domain assumption PCA-based cluster separation metrics reflect genuine encoding of persona differences.
Cite this review
Pith. "Pith review of Localizing Persona Representations in LLMs." pith.science (2026). https://pith.science/paper/VKHORXF4
@misc{pith2026250524539,
author = {Pith},
title = {Pith review of: Localizing Persona Representations in LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/VKHORXF4}},
note = {Machine review of arXiv:2505.24539}
}
read the original abstract
We present a study on how and where personas -- defined by distinct sets of human characteristics, values, and beliefs -- are encoded in the representation space of large language models (LLMs). Using a range of dimension reduction and pattern recognition methods, we first identify the model layers that show the greatest divergence in encoding these representations. We then analyze the activations within a selected layer to examine how specific personas are encoded relative to others, including their shared and distinct embedding spaces. We find that, across multiple pre-trained decoder-only LLMs, the analyzed personas show large differences in representation space only within the final third of the decoder layers. We observe overlapping activations for specific ethical perspectives -- such as moral nihilism and utilitarianism -- suggesting a degree of polysemy. In contrast, political ideologies like conservatism and liberalism appear to be represented in more distinct regions. These findings help to improve our understanding of how LLMs internally represent information and can inform future efforts in refining the modulation of specific human traits in LLM outputs. Warning: This paper includes potentially offensive sample statements.
Figures
Figures from the paper (7 more)
Forward citations
Cited by 1 Pith paper
-
A Scalable Approach to Evaluating Moral Sensitivity in LLMs
Under morally irrelevant noise, eight LLMs preserve the semantic content of identified moral features above calibrated floors, despite significant changes in feature counts.
Reference graph
Works this paper leans on
-
[3]
Discovering language model behaviors with model-written evaluations
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. InFindings of the Association for Computational Linguistics: ACL 2023, pages 13387–13434, 2023
2023
-
[1]
Unsolved problems in ml safety.arXiv preprint arXiv:2109.13916, 2021
Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety.arXiv preprint arXiv:2109.13916, 2021
arXiv 2021
-
[2]
Interpretability in the wild: a circuit for indirect object identification in gpt-2 small
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. InICLR, 2023
2023
-
[4]
Lamp: When large language models meet personalization.arXiv preprint arXiv:2304.11406, 2023
Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization.arXiv preprint arXiv:2304.11406, 2023
arXiv 2023
-
[5]
Daly, Kush R
Erik Miehling, Michael Desmond, Karthikeyan Natesan Ramamurthy, Elizabeth M. Daly, Kush R. Varshney, Eitan Farchi, Pierre Dognin, Jesus Rios, Djallel Bouneffouf, Miao Liu, and Prasanna Sattigeri. Evaluating the prompt steerability of large language models. InConference of the North American Chapter of the Association for Computational Linguistics, 2025
2025
-
[6]
Scaling synthetic data creation with 1,000,000,000 personas.arXiv preprint arXiv:2406.20094, 2024
Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. Scaling synthetic data creation with 1,000,000,000 personas.arXiv preprint arXiv:2406.20094, 2024
arXiv 2024
-
[7]
Can llm be a personalized judge?arXiv preprint arXiv:2406.11657, 2024
Yijiang River Dong, Tiancheng Hu, and Nigel Collier. Can llm be a personalized judge?arXiv preprint arXiv:2406.11657, 2024
arXiv 2024
-
[8]
Can ai language models replace human participants? Trends in Cognitive Sciences, 27(7):597–600, 2023
Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can ai language models replace human participants? Trends in Cognitive Sciences, 27(7):597–600, 2023
2023
Show all 102 references
-
[9]
More human than human: measuring chatgpt political bias.Public Choice, 198(1):3–23, 2024
Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. More human than human: measuring chatgpt political bias.Public Choice, 198(1):3–23, 2024
2024
-
[10]
Perils and opportunities in using large language models in psychological research.PNAS nexus, 3(7):pgae245, 2024
Suhaib Abdurahman, Mohammad Atari, Farzan Karimi-Malekabadi, Mona J Xue, Jackson Trager, Peter S Park, Preni Golazizian, Ali Omrani, and Morteza Dehghani. Perils and opportunities in using large language models in psychological research.PNAS nexus, 3(7):pgae245, 2024
2024
-
[11]
Questioning the survey responses of large language models.Advances in Neural Information Processing Systems, 37:45850–45878, 2024
Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. Questioning the survey responses of large language models.Advances in Neural Information Processing Systems, 37:45850–45878, 2024
2024
-
[12]
Personas and user-centered design: How can personas benefit product design processes?Design studies, 32(5):417–430, 2011
Tomasz Miaskiewicz and Kenneth A Kozar. Personas and user-centered design: How can personas benefit product design processes?Design studies, 32(5):417–430, 2011
2011
-
[13]
Use cases for design personas: A systematic review and new frontiers
Joni Salminen, Kathleen Wenyun Guan, Soon-Gyo Jung, and Bernard Jansen. Use cases for design personas: A systematic review and new frontiers. InCHI Conference on Human Factors in Computing Systems, pages 1–21, 2022
2022
-
[14]
Me, myself, and ai: The situational awareness dataset (sad) for llms
Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Mikita Balesni, Jérémy Scheurer, Marius Hobb- hahn, Alexander Meinke, and Owain Evans. Me, myself, and ai: The situational awareness dataset (sad) for llms. InNeurIPS Systems Datasets and Benchmarks, 2024
2024
-
[15]
Bias runs deep: Implicit reasoning biases in persona-assigned LLMs
Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. Bias runs deep: Implicit reasoning biases in persona-assigned LLMs. InICLR, 2024. URL https://openreview.net/forum?id=kGteeZ18Ir
2024
-
[16]
Divide-and-conquer attack: Harnessing the power of llm to bypass the censorship of text-to-image generation model.arXiv preprint arXiv:2312.07130, 2023
Yimo Deng and Huangxun Chen. Divide-and-conquer attack: Harnessing the power of llm to bypass the censorship of text-to-image generation model.arXiv preprint arXiv:2312.07130, 2023
2023 arXiv
-
[17]
Equality of opportunity in supervised learning.Advances in neural information processing systems, 29, 2016
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning.Advances in neural information processing systems, 29, 2016. 11 Localizing Persona Representations in LLMs
2016
-
[18]
Algorithmic decision making and the cost of fairness
Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. InProceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pages 797–806, 2017
2017
-
[19]
A review of the challenges with massive web-mined corpora used in large language models pre-training
Michał Perełkiewicz and Rafał Po´swiata. A review of the challenges with massive web-mined corpora used in large language models pre-training. InInternational Conference on Artificial Intelligence and Soft Computing, pages 153–163. Springer, 2024
2024
-
[20]
Datasets for large language models: A comprehensive survey.arXiv preprint arXiv:2402.18041, 2024
Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. Datasets for large language models: A comprehensive survey.arXiv preprint arXiv:2402.18041, 2024
2024 arXiv
-
[21]
Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, and Arsalan Shahid. The ultimate guide to fine-tuning llms from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research challenges and opportunities.arXiv preprint...
2024 arXiv
-
[22]
Instruction tuning for large language models: A survey.arXiv preprint arXiv:2308.10792, 2023
Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey.arXiv preprint arXiv:2308.10792, 2023
2023
-
[23]
The self- perception and political biases of chatgpt.Human Behavior and Emerging Technologies, 2024(1):7115633, 2024
Jérôme Rutinowski, Sven Franke, Jan Endendyk, Ina Dormuth, Moritz Roidl, and Markus Pauly. The self- perception and political biases of chatgpt.Human Behavior and Emerging Technologies, 2024(1):7115633, 2024
2024
-
[24]
Whose opinions do language models reflect? InInternational Conference on Machine Learning, pages 29971–30004
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? InInternational Conference on Machine Learning, pages 29971–30004. PMLR, 2023
2023
-
[25]
Trustworthy, responsible, and safe ai: A comprehensive architectural framework for ai safety with challenges and mitigations.arXiv preprint arXiv:2408.12935, 2024
Chen Chen, Xueluan Gong, Ziyao Liu, Weifeng Jiang, Si Qi Goh, and Kwok-Yan Lam. Trustworthy, responsible, and safe ai: A comprehensive architectural framework for ai safety with challenges and mitigations.arXiv preprint arXiv:2408.12935, 2024
2024 arXiv
-
[26]
Taking stock of concept inventories in computing education: A systematic literature review
Murtaza Ali, Sourojit Ghosh, Prerna Rao, Raveena Dhegaskar, Sophia Jawort, Alix Medler, Mengqi Shi, and Sayamindu Dasgupta. Taking stock of concept inventories in computing education: A systematic literature review. InProceedings of the 2023 ACM Conference on International Com...
2023
-
[27]
Out of one, many: Using language models to simulate human samples.Political Analysis, 31(3):337–351, 2023
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples.Political Analysis, 31(3):337–351, 2023
2023
-
[28]
Towards trustworthy ai: A review of ethical and robust large language models.arXiv preprint arXiv:2407.13934, 2024
Md Meftahul Ferdaus, Mahdi Abdelguerfi, Elias Ioup, Kendall N Niles, Ken Pathak, and Steven Sloan. Towards trustworthy ai: A review of ethical and robust large language models.arXiv preprint arXiv:2407.13934, 2024
2024 arXiv
-
[29]
Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023
-
[30]
Semantics derived automati- cally from language corpora contain human-like moral choices
Sophie Jentzsch, Patrick Schramowski, Constantin Rothkopf, and Kristian Kersting. Semantics derived automati- cally from language corpora contain human-like moral choices. InAIES, pages 37–44, 2019
2019
-
[31]
A primer on the inner workings of transformer-based language models.arXiv preprint arXiv:2405.00208, 2024
Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R Costa-jussà. A primer on the inner workings of transformer-based language models.arXiv preprint arXiv:2405.00208, 2024
2024 arXiv
-
[32]
Probing then editing response personality of large language models.arXiv preprint arXiv:2504.10227, 2025
Tianjie Ju, Zhenyu Shao, Bowen Wang, Yujia Chen, Zhuosheng Zhang, Hao Fei, Mong-Li Lee, Wynne Hsu, Sufeng Duan, and Gongshen Liu. Probing then editing response personality of large language models.arXiv preprint arXiv:2504.10227, 2025
2025 arXiv
-
[33]
Aligning llms with individual preferences via interaction
Shujin Wu, Yi R Fung, Cheng Qian, Jeonghwan Kim, Dilek Hakkani-Tur, and Heng Ji. Aligning llms with individual preferences via interaction. InInternational Conference on Computational Linguistics, pages 7648– 7662, 2025
2025
-
[34]
The big five personality factors and personal values.Personality and social psychology bulletin, 28(6):789–801, 2002
Sonia Roccas, Lilach Sagiv, Shalom H Schwartz, and Ariel Knafo. The big five personality factors and personal values.Personality and social psychology bulletin, 28(6):789–801, 2002
2002
-
[35]
A very brief measure of the big-five personality domains.Journal of Research in Personality, 37(6):504–528, 2003
Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. A very brief measure of the big-five personality domains.Journal of Research in Personality, 37(6):504–528, 2003
2003
-
[36]
Evaluating large language model biases in persona-steered generation
Andy Liu, Mona Diab, and Daniel Fried. Evaluating large language model biases in persona-steered generation. arXiv preprint arXiv:2405.20253, 2024
2024 arXiv
-
[37]
Helpful assistant or fruitful facilitator? investigating how personas affect language model behavior.arXiv preprint arXiv:2407.02099, 2024
Pedro Henrique Luz de Araujo and Benjamin Roth. Helpful assistant or fruitful facilitator? investigating how personas affect language model behavior.arXiv preprint arXiv:2407.02099, 2024. 12 Localizing Persona Representations in LLMs
2024 arXiv
-
[38]
Personallm: Investigating the ability of large language models to express personality traits.arXiv preprint arXiv:2305.02547, 2023
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. Personallm: Investigating the ability of large language models to express personality traits.arXiv preprint arXiv:2305.02547, 2023
2023 arXiv
-
[39]
Marked personas: Using natural language prompts to measure stereotypes in language models.arXiv preprint arXiv:2305.18189, 2023
Myra Cheng, Esin Durmus, and Dan Jurafsky. Marked personas: Using natural language prompts to measure stereotypes in language models.arXiv preprint arXiv:2305.18189, 2023
2023 arXiv
-
[40]
Toxicity in chatgpt: Analyzing persona-assigned language models
Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. Toxicity in chatgpt: Analyzing persona-assigned language models. InEMNLP, pages 1236–1270, 2023
2023
-
[41]
Revealing persona biases in dialogue systems.arXiv preprint arXiv:2104.08728, 2021
Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, and Nanyun Peng. Revealing persona biases in dialogue systems.arXiv preprint arXiv:2104.08728, 2021
2021 arXiv
-
[42]
they are uncultured
Preetam Prabhu Srikar Dammu, Hayoung Jung, Anjali Singh, Monojit Choudhury, and Tanushree Mitra. "they are uncultured": Unveiling covert harms and social threats in llm generated conversations.arXiv preprint arXiv:2405.05378, 2024
2024 arXiv
-
[43]
The ethics of interaction: Mitigating security threats in llms.arXiv preprint arXiv:2401.12273, 2024
Ashutosh Kumar, Shiv Vignesh Murthy, Sagarika Singh, and Swathy Ragupathy. The ethics of interaction: Mitigating security threats in llms.arXiv preprint arXiv:2401.12273, 2024
2024 arXiv
-
[44]
A survey on responsible llms: Inherent risk, malicious use, and mitigation strategy
Huandong Wang, Wenjie Fu, Yingzhou Tang, Zhilong Chen, Yuxi Huang, Jinghua Piao, Chen Gao, Fengli Xu, Tao Jiang, and Yong Li. A survey on responsible llms: Inherent risk, malicious use, and mitigation strategy. arXiv preprint arXiv:2501.09431, 2025
2025 arXiv
-
[45]
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InnaacL-HLT, volume 1, 2019
2019
-
[46]
Bert has a moral compass: Improvements of ethical and moral values of machines.arXiv preprint arXiv:1912.05238, 2019
Patrick Schramowski, Cigdem Turan, Sophie Jentzsch, Constantin Rothkopf, and Kristian Kersting. Bert has a moral compass: Improvements of ethical and moral values of machines.arXiv preprint arXiv:1912.05238, 2019
1912 arXiv
-
[47]
Patchscopes: A unify- ing framework for inspecting hidden representations of language models
Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unify- ing framework for inspecting hidden representations of language models. InICML, 2024. URL https: //openreview.net/forum?id=5uwBzcn885
2024
-
[48]
Rethinking interpretability in the era of large language models.arXiv preprint arXiv:2402.01761, 2024
Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking interpretability in the era of large language models.arXiv preprint arXiv:2402.01761, 2024
2024 arXiv
-
[49]
Polysemanticity and capacity in neural networks.arXiv preprint arXiv:2210.01892, 2022
Adam Scherlis, Kshitij Sachan, Adam S Jermyn, Joe Benton, and Buck Shlegeris. Polysemanticity and capacity in neural networks.arXiv preprint arXiv:2210.01892, 2022
2022 arXiv
-
[50]
Linear algebraic structure of word senses, with applications to polysemy.Transactions of the Association for Computational Linguistics, 6:483–495, 2018
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy.Transactions of the Association for Computational Linguistics, 6:483–495, 2018
2018
-
[51]
Weakly supervised detection of hallucinations in llm activations.arXiv preprint arXiv:2312.02798, 2023
Miriam Rateike, Celia Cintas, John Wamburu, Tanya Akumu, and Skyler Speakman. Weakly supervised detection of hallucinations in llm activations.arXiv preprint arXiv:2312.02798, 2023
2023 arXiv
-
[52]
Identifying audio adversarial examples via anomalous pattern detection.Workshop on Adversarial Learning Methods for ML and DM, KDD, 2020
Victor Akinwande, Celia Cintas, Skyler Speakman, and Srihari Sridharan. Identifying audio adversarial examples via anomalous pattern detection.Workshop on Adversarial Learning Methods for ML and DM, KDD, 2020
2020
-
[53]
Pattern detection in the activation space for identifying synthesized content.Pattern Recognition Letters, 153:207–213, 2022
Celia Cintas, Skyler Speakman, Girmaw Abebe Tadesse, Victor Akinwande, Edward McFowland III, and Komminist Weldemariam. Pattern detection in the activation space for identifying synthesized content.Pattern Recognition Letters, 153:207–213, 2022
2022
-
[54]
description of personality
Lewis R Goldberg. An alternative “description of personality”: The big-five factor structure. InPersonality and personality disorders, pages 34–47. Routledge, 2013
2013
-
[55]
Paradigm shift to the integrative big five trait taxonomy
Oliver P John, Laura P Naumann, and Christopher J Soto. Paradigm shift to the integrative big five trait taxonomy. Handbook of personality: Theory and research, 3(2):114–158, 2008
2008
-
[56]
The stability of personality: Observations and evaluations.Current Directions in Psychological Science, 3(6):173–175, 1994
Robert R McCrae and Paul T Costa. The stability of personality: Observations and evaluations.Current Directions in Psychological Science, 3(6):173–175, 1994
1994
-
[57]
The role of personality in leadership: Five factor personality traits and ethical leadership
Gönül Kaya Özba˘g. The role of personality in leadership: Five factor personality traits and ethical leadership. Procedia-Social and Behavioral Sciences, 235:235–242, 2016
2016
-
[58]
Timothy A Judge and Cindy P Zapata. The person–situation debate revisited: Effect of situation strength and trait activation on the validity of the big five personality traits in predicting job performance.Academy of Management Journal, 58(4):1149–1179, 2015
2015
-
[59]
The relationship between the big five personality traits and authentic leadership
Bronti Baptiste. The relationship between the big five personality traits and authentic leadership. 2018
2018
-
[60]
Big five personality traits and risky decision-making: A study of behavioural tasks among college students.Passer Journal of Basic and Applied Sciences, 5(2):298–303, 2023
Zana Hasan Babakr and Nabi Fatahi. Big five personality traits and risky decision-making: A study of behavioural tasks among college students.Passer Journal of Basic and Applied Sciences, 5(2):298–303, 2023. 13 Localizing Persona Representations in LLMs
2023
-
[61]
The big five personality traits and leadership: A comprehensive analysis
ANSHIKA Grover and A Amit. The big five personality traits and leadership: A comprehensive analysis. International Journal For Multidisciplinary Research, 6(1), 2024
2024
-
[62]
Wiktoria Mieleszczenko-Kowszewicz, Dawid Płudowski, Filip Kołodziejczyk, Jakub´Swistak, Julian Sienkiewicz, and Przemysław Biecek. The dark patterns of personalized persuasion in large language models: Exposing persuasive linguistic features for big five personality traits in ...
2024 arXiv
-
[63]
Predicting the big five personality traits in chinese counselling dialogues using large language models.arXiv preprint arXiv:2406.17287, 2024
Yang Yan, Lizhi Ma, Anqi Li, Jingsong Ma, and Zhenzhong Lan. Predicting the big five personality traits in chinese counselling dialogues using large language models.arXiv preprint arXiv:2406.17287, 2024
2024 arXiv
-
[64]
The moral turing test: Evaluating human-llm alignment in moral decision-making.arXiv preprint arXiv:2410.07304, 2024
Basile Garcia, Crystal Qian, and Stefano Palminteri. The moral turing test: Evaluating human-llm alignment in moral decision-making.arXiv preprint arXiv:2410.07304, 2024
2024 arXiv
-
[65]
Exploring persona-dependent llm alignment for the moral machine experiment.arXiv preprint arXiv:2504.10886, 2025
Jiseon Kim, Jea Kwon, Luiz Felipe Vecchietti, Alice Oh, and Meeyoung Cha. Exploring persona-dependent llm alignment for the moral machine experiment.arXiv preprint arXiv:2504.10886, 2025
2025 arXiv
-
[66]
Moralbench: Moral evaluation of llms.arXiv preprint arXiv:2406.04428, 2024
Jianchao Ji, Yutong Chen, Mingyu Jin, Wujiang Xu, Wenyue Hua, and Yongfeng Zhang. Moralbench: Moral evaluation of llms.arXiv preprint arXiv:2406.04428, 2024
2024 arXiv
-
[67]
Fairmindsim: Alignment of behavior, emotion, and belief in humans and llm agents amid ethical dilemmas
Yu Lei, Hao Liu, Chengxing Xie, Songjia Liu, Zhiyu Yin, Canyu Chen, Guohao Li, Philip Torr, and Zhen Wu. Fairmindsim: Alignment of behavior, emotion, and belief in humans and llm agents amid ethical dilemmas. arXiv preprint arXiv:2410.10398, 2024
-
[68]
Moral alignment for llm agents.arXiv preprint arXiv:2410.01639, 2024
Elizaveta Tennant, Stephen Hailes, and Mirco Musolesi. Moral alignment for llm agents.arXiv preprint arXiv:2410.01639, 2024
2024 arXiv
-
[69]
Kaiping Chen, Anqi Shao, Jirayu Burapacheep, and Yixuan Li. A critical appraisal of equity in conversational ai: Evidence from auditing gpt-3’s dialogues with different publics on climate change and black lives matter.ArXiv, abs/2209.13627, 2022. URLhttps://api.semanticscholar...
2022 arXiv
-
[70]
Aligning large language models with diverse political viewpoints.arXiv preprint arXiv:2406.14155, 2024
Dominik Stammbach, Philine Widmer, Eunjung Cho, Caglar Gulcehre, and Elliott Ash. Aligning large language models with diverse political viewpoints.arXiv preprint arXiv:2406.14155, 2024
2024 arXiv
-
[71]
Uncovering intersectional stereotypes in humans and large language models
Falaah Arif Khan, Nivedha Sivakumar, Yinong Oliver Wang, Katherine Metcalf, Cezanne Camacho, Barry-John Theobald, Luca Zappella, and Nicholas Apostoloff. Uncovering intersectional stereotypes in humans and large language models. 2025
2025
-
[72]
The effects of demographic instructions on llm personas
J Shane Culpepper, Alistair Moffat, Sachin Pathiyan Cherumanal, Falk Scholer, and Johanne Trippas. The effects of demographic instructions on llm personas. 2025
2025
-
[73]
Concerns on bias in large language models when creating synthetic personae.arXiv preprint arXiv:2405.05080, 2024
Helena A Haxvig. Concerns on bias in large language models when creating synthetic personae.arXiv preprint arXiv:2405.05080, 2024
2024 arXiv
-
[74]
Llama3 model card
AI@Meta. Llama3 model card. https://github.com/meta-llama/llama3, 2024. Accessed: 2024-10-24
2024
-
[75]
Granite: A new framework for language models
IBM Granite Team. Granite: A new framework for language models. https://github.com/ibm-granite/granite-3.0- language-models/blob/main/paper.pdf, 2023. Accessed: 2024-10-24
2023
-
[76]
Mistral 7b (2023).arXiv preprint arXiv:2310.06825, 2023
AQ Jiang, A Sablayrolles, A Mensch, C Bamford, DS Chaplot, D de las Casas, F Bressand, G Lengyel, G Lample, L Saulnier, et al. Mistral 7b (2023).arXiv preprint arXiv:2310.06825, 2023
2023 arXiv
-
[77]
Instruction pre-training: Language models are supervised multitask learners.arXiv preprint arXiv:2406.14491, 2024
Daixuan Cheng, Yuxian Gu, Shaohan Huang, Junyu Bi, Minlie Huang, and Furu Wei. Instruction pre-training: Language models are supervised multitask learners.arXiv preprint arXiv:2406.14491, 2024
2024 arXiv
-
[78]
How to fine-tune bert for text classification? InChinese Computational Linguistics, pages 194–206
Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. How to fine-tune bert for text classification? InChinese Computational Linguistics, pages 194–206. Springer, 2019
2019
-
[79]
The semantic relations in llms: An information-theoretic compression approach
Yu-Hsiang Tseng, Pin-Er Chen, Da-Chen Lian, and Shu-Kai Hsieh. The semantic relations in llms: An information-theoretic compression approach. InWorkshop: Bridging Neurons and Symbols for NLP and KGR @ LREC-COLING, pages 8–21, 2024
2024
-
[80]
Pedregosa, G
F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python.JMLR, 12:2825–2830, 2011
2011
-
[81]
A dendrite method for cluster analysis.Communications Statistics–Theory and Methods, 3(1):1–27, 1974
Tadeusz Cali´nski and Jerzy Harabasz. A dendrite method for cluster analysis.Communications Statistics–Theory and Methods, 3(1):1–27, 1974
1974
-
[82]
Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.Journal of Computational and Applied Mathematics, 20:53–65, 1987
Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.Journal of Computational and Applied Mathematics, 20:53–65, 1987
1987
-
[83]
A cluster separation measure.IEEE Transactions Pattern Analysis & Machine Intelligence, (2):224–227, 1979
David L Davies and Donald W Bouldin. A cluster separation measure.IEEE Transactions Pattern Analysis & Machine Intelligence, (2):224–227, 1979. 14 Localizing Persona Representations in LLMs
1979
-
[84]
Fast generalized subset scan for anomalous pattern detection.JMLR, 14(1), 2013
Edward McFowland, Skyler Speakman, and Daniel B Neill. Fast generalized subset scan for anomalous pattern detection.JMLR, 14(1), 2013
2013
-
[85]
How large language models encode context knowledge? a layer-wise probing study
Tianjie Ju, Weiwei Sun, Wei Du, Xinwei Yuan, Zhaochun Ren, and Gongshen Liu. How large language models encode context knowledge? a layer-wise probing study. InJoint International Conference on Computational Linguistics, Language Resources and Evaluation, pages 8235–8246, 2024
2024
-
[86]
Student evaluations of teaching: effects of the big five personality traits, grades and the validity hypothesis.Assessment & Evaluation in Higher Education, 36(2):239–249, 2011
Carol Lynn Patrick. Student evaluations of teaching: effects of the big five personality traits, grades and the validity hypothesis.Assessment & Evaluation in Higher Education, 36(2):239–249, 2011. doi: 10.1080/ 02602930903308258
2011
-
[87]
McCrae and Oliver P
Robert R. McCrae and Oliver P. John. An introduction to the five-factor model and its applications.Journal of Personality, 60(2):175–215, 1992. doi: https://doi.org/10.1111/j.1467-6494.1992.tb00970.x. URL https: //onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-6494.1992.tb00970.x
1992
-
[88]
Chuchai Smithikrai. Moderating effect of situational strength on the relationship between personality traits and counterproductive work behaviour.Asian Journal of Social Psychology, 11(4):253–263, 2008. doi: https://doi.org/10.1111/j.1467-839X.2008.00265.x. URL https://onlinel...
2008 arXiv
-
[89]
Grehan, Rosemary Flanagan, and Robert G
Patrick M. Grehan, Rosemary Flanagan, and Robert G. Malgady. Successful graduate students: The roles of personality traits and emotional intelligence.Psychology in the Schools, 48(4):317–331, 2011. doi: https://doi. org/10.1002/pits.20556. URLhttps://onlinelibrary.wiley.com/do...
2011 doi
-
[90]
Virtue Ethics
Rosalind Hursthouse and Glen Pettigrove. Virtue Ethics. In Edward N. Zalta and Uri Nodelman, editors,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Fall 2023 edition, 2023
2023
-
[91]
Adam Carter
Maria Baghramian and J. Adam Carter. Relativism. In Edward N. Zalta and Uri Nodelman, editors,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Spring 2025 edition, 2025
2025
-
[92]
Deontological Ethics
Larry Alexander and Michael Moore. Deontological Ethics. In Edward N. Zalta and Uri Nodelman, editors,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2024 edition, 2024
2024
-
[93]
Consequentialism
Walter Sinnott-Armstrong. Consequentialism. In Edward N. Zalta and Uri Nodelman, editors,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2023 edition, 2023
2023
-
[94]
Moral Skepticism
Walter Sinnott-Armstrong. Moral Skepticism. In Edward N. Zalta and Uri Nodelman, editors,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Fall 2024 edition, 2024
2024
-
[95]
Conservatism
Andy Hamilton. Conservatism. In Edward N. Zalta, editor,The Stanford Encyclopedia of Philosophy. Meta- physics Research Lab, Stanford University, Spring 2020 edition, 2020
2020
-
[96]
Courtland, Gerald Gaus, and David Schmidtz
Shane D. Courtland, Gerald Gaus, and David Schmidtz. Liberalism. In Edward N. Zalta, editor,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Spring 2022 edition, 2022
2022
-
[97]
Explaining the trump vote: The effect of racist resentment and anti- immigrant sentiments.PS: Political Science & Politics, 51(3):528–534, 2018
Marc Hooghe and Ruth Dassonneville. Explaining the trump vote: The effect of racist resentment and anti- immigrant sentiments.PS: Political Science & Politics, 51(3):528–534, 2018
2018
-
[98]
Queer rights talk: The rhetoric of equality rights for lgbtq+ peoples
Daniel Del Gobbo and Rebecca J Cook. Queer rights talk: The rhetoric of equality rights for lgbtq+ peoples. Frontiers of gender equality. Transnational legal perspectives, pages 68–87, 2023
2023
-
[99]
Didem Unal. Political homophobia as a tool of creating crisis narratives and ontological insecurities in illiberal populist contexts: lessons from the 2023 elections in turkey.New Perspectives on Turkey, 71:143–164, 2024. doi: 10.1017/npt.2024.4
2023 doi
-
[100]
Reassessing the relationship between homophobia and political participation.European Journal of Political Research, 61, 02 2022
Douglas Page, Phillip Ayoub, Catharine Arranz, Matthew Montes, and Taylor Paulin. Reassessing the relationship between homophobia and political participation.European Journal of Political Research, 61, 02 2022. doi: 10.1111/1475-6765.12519
2022
-
[101]
Lab: Large-scale alignment for chatbots.arXiv preprint arXiv:2403.01081, 2024
Shivchander Sudalairaj, Abhishek Bhandwaldar, Aldo Pareja, Kai Xu, David D Cox, and Akash Srivastava. Lab: Large-scale alignment for chatbots.arXiv preprint arXiv:2403.01081, 2024
2024 arXiv
-
[102]
political homophobia
Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. LOF: Identifying Density-Based Local Outliers. InACM SIGMOD International Conference on Management of Data, pages 93–104, 2000. 15 Localizing Persona Representations in LLMs A Appendix A.1 Dataset In Table ...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.