{"id":"2f2f3541-9259-4ad3-a511-1cf39c2e380f","arxiv_id":"2412.16882","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PsychAdapter adds lightweight per-layer projections to GPT-2, Gemma, and Llama so that continuous psychological scores directly shape generated text, with expert raters identifying intended levels in most cases.","lead":"This paper introduces PsychAdapter, a small set of learned layers that plug into large language models and let them generate text with a chosen personality, mood, or demographic profile. The authors show that human experts can usually tell which intended trait level the generated text was aimed at, which could help build personalized chatbots or clinical training tools.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Message-level teacher labels are the load-bearing risk: if the participant-level trait models are not valid at message level, PsychAdapter learns the teacher's stereotypes, making the human-rater accuracy an artifact of stereotype matching.","rationale":"Reader identified the same weakest assumption; I agree. The paper has real strengths: lightweight per-layer projection, cross-model generalization to GPT-2 and Llama-3, and blinded expert ratings. But those strengths do not address the source of the trait signal. Because the labels are generated by the same kinds of lexical/topic models whose output the adapters are then asked to reproduce, the evaluation is at risk of being circular: the 'ground truth' is a text-based stereotype model, and the human raters are judging the same stereotypes. This does not make the method useless—stylized personality-specific generation is a valid application—but it does invalidate the stronger reading that PsychAdapter reflects actual mental health characteristics. The proposed test would settle it: if message-level teacher predictions correlate with external self-report/clinical measures, the concern is answered. Since this concern is already the basis of the reader's CONDITIONAL verdict, I do not change the verdict.","tokens_in":20302,"tokens_out":7330,"duration_ms":71799,"concrete_test":"Obtain a held-out corpus with self-report or clinical scores (e.g., IPIP Big Five, PHQ-9/CES-D, SWLS) and natural language messages from the same individuals. Apply the teacher model to each message to get psi_m, then test whether psi_m predicts the individual's score (mixed-effects model with random participant intercept, or within-participant rank correlation/AUC). If message-level predictions show no significant association with self-report after controlling for sentiment/valence, the training labels are invalid and PsychAdapter's human-eval accuracy likely reflects stereotype matching; if the association is strong, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—prompt-free, continuous control of personality and mental health expression—rests on the training labels psi_m = W X_m assigned by participant-level text assessment models (Methods, 'Obtaining estimated psychological scores for messages'). These models (Schwartz 2013; Park 2015; depression/well-being models; Sap 2014 for age) were trained on aggregated participant text with LDA topic features; a single tweet (~12 words) or 30-word blog excerpt is far sparser and noisier. The teacher's message-level predictions need not track the intended psychological construct; they may only encode topical/lexical stereotypes (e.g., high neuroticism ↔ negative-emotion words). Because the adapter is trained to reconstruct text from these labels, it can learn those stereotypes rather than genuine trait-language patterns. The expert evaluation cannot break this loop: raters also identify traits through the same stereotypical cues, so 87.3%/96.7% accuracy shows the generated text is stereotyped, not that it reflects self-reported or clinical traits. The paper's own Limitation section acknowledges demographic bias but does not validate the message-level teacher labels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PsychAdapter, a parameter-efficient modification of autoregressive transformer language models that injects continuous psychological trait vectors into every transformer layer via learned projection matrices and LoRA fine-tuning. Training uses message-level 'estimated' trait scores produced by participant-level language-based assessment models applied to blog and tweet corpora, with a text-reconstruction objective. The authors report that the resulting adapters generate text distinguishable by expert raters at 87.3% average accuracy for the Big Five, 96.7% for depression and life satisfaction, and that fine-grained levels, demographics, multi-trait combinations, and different base models (GPT-2, Gemma, Llama 3) can be controlled, with Claude 3.5 Sonnet used as an additional annotator.","tokens_in":20552,"tokens_out":3327,"duration_ms":36087,"significance":"If the central claim holds, PsychAdapter is a useful and lightweight contribution: it offers prompt-free, continuous control over trait expression in LLMs with under 0.1% added parameters, and the paper ships open code and data plus human expert evaluations. The architecture idea of per-layer trait projections is clean and clearly described. However, the validity of the entire pipeline rests on the unvalidated message-level teacher labels, and the headline accuracy figures are point estimates from a small sample without significance tests or confidence intervals. These are fixable concerns, so the contribution is potentially publishable, but the current evidence does not yet establish the strong psychological claim.","major_comments":[{"comment":"The training signal for every PsychAdapter is the message-level estimate psi_m = W X_m, obtained by applying participant-level assessment models to individual messages. No validation is reported that these participant-level models, trained on aggregated text with LDA topic features, produce valid trait estimates at the message level for 30-word blog excerpts or 12-word tweets. If these labels capture only topical or lexical stereotypes, the adapter learns the teacher's stereotypes rather than genuine trait-language patterns, and the human-rater results would then measure stereotype matching rather than psychological construct expression. This is load-bearing for the paper's central claim: please add a validation study comparing message-level teacher estimates against human trait judgments (or against participant-level ground truth on held-out authors), and report per-construct correlations, agreement, and examples where teacher estimates diverge from human judgment.","section":"Methods — 'Obtaining estimated psychological scores for messages'"},{"comment":"The headline accuracies (87.3% and 96.7%) are point estimates based on 10 generation seeds, two raters, and a nonstandard scoring rule (one point only if all three levels are correctly matched; 1/3 point if exactly one is correct). No confidence intervals, significance tests, or per-dimension inter-rater reliabilities are reported, so the statement that PsychAdapter 'significantly outperformed' the 33.3% baseline is unsupported. Please report exact binomial or permutation tests against chance, CIs for each dimension and condition, and the raw trial-level rater classifications; also clarify whether raters were forced to assign one group to each level or could make other assignments.","section":"Results — 'Evaluations with human raters and Claude'"},{"comment":"The five-level fine-grained evaluation, which is the main evidence for continuous control, is conducted entirely with Claude 3.5 Sonnet as the annotator, with no human validation of the five-level distinctions. The earlier human–Claude agreement is established only at the coarse three-level task. Since the fine-grained claim is central to the advantage over discrete prompting, either add human ratings for the five-level setup or explicitly reframe the Claude-based results as a stereotype-matching measure rather than evidence of psychological validity.","section":"Results — 'Generating text at fine-grained levels of personality' and Figures 4–5"}],"minor_comments":[{"comment":"The abstract contains two typos: 'Lamma' for 'Llama' and 'PsychAdapter — es empirically derived trait-language patterns' which appears to be missing the word 'uses'.","section":"Abstract"},{"comment":"The section heading 'Obtaining estimated psychological for messages' is missing the word 'scores'; please correct it.","section":"Methods"},{"comment":"The Limitations paragraph contains 'the propagation of bias' and 'a the propagation', which should be edited; more importantly, the paragraph does not mention the message-level teacher-label validity concern, which is the main methodological risk.","section":"Discussion — Limitations"},{"comment":"The caption contains 'avaraged' for 'averaged' and would benefit from reporting the number of trials used for the reported accuracies.","section":"Figure 5 caption"},{"comment":"The description of transformation matrices says 'each transformation matrix will have the shape of [latent_size, num_key_value_heads × head_dim]', but the text earlier refers to a 5 × 2048 matrix for Gemma-2B; please make the shape notation and the example consistent.","section":"Methods — 'Modifying transformer language models...'"}],"recommendation":"major_revision","confidential_remarks":"The work is potentially valuable and the public release of code and data is commendable. My main concern is that the central validity claim depends on message-level teacher labels that are not validated anywhere in the manuscript; no amount of human rater accuracy on generated text can break that loop without a direct check of the teacher labels. The statistical presentation also needs strengthening. I would not reject, but the revision must address these two points substantively rather than rhetorically."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the architecture is genuinely new: per-layer projection of a continuous trait vector into the key/value hidden states of an autoregressive transformer, trained with LoRA, adding under 0.1% parameters. They show it works across GPT-2 Large, Gemma-2B, and Llama-3-8B, with human experts matching intended levels at 87.3% average accuracy for Big Five and 96.7% for depression/life satisfaction. Second, the training labels are the weak point. They take participant-level text-based assessment models (Schwartz 2013, Park 2015, etc.) and apply them to single messages via psi_m = W X_m. No one has validated that these models produce meaningful trait scores at message level; a 12-word tweet or 30-word blog excerpt is far sparser than the aggregated participant text these models were built on. If the teacher is mostly encoding lexical stereotypes, the adapter learns stereotypes, and the expert raters—who also use the same cues—confirm the stereotypes. So the high accuracy is real evidence of controllability, but it's evidence of stereotype fidelity, not necessarily of authentic trait-language reflection.\n\nWhat the paper does well: the method is simple, efficient, and plausibly generalizes. The human evaluation uses trained raters with acceptable inter-rater agreement (weighted kappa 0.76). They test fine-grained levels, multi-trait combos via the interpersonal circumplex, demographic conditioning, and domain transfer (tweets vs blogs). The examples in Tables 1-4 look consistent with trait expectations. This is a useful tool for any application needing prompt-free trait-conditioned generation.\n\nBeyond the label issue, three smaller problems. First, no confidence intervals or significance tests; 'significantly outperformed' is unsupported—10 trials, two raters, and point estimates only. Second, no head-to-head comparison against prompt-based trait control, which is the main alternative they claim to beat. Third, the preprint says code and data are open-sourced but doesn't actually provide links.\n\nWho it's for: NLP researchers working on controllable generation, computational social scientists who want to generate synthetic text from psychological profiles, and people building chatbots with personality. The paper deserves a serious referee. The central method is sound; the label validity question is real but addressable in revision—either validate the teacher at message level or reframe claims as 'matches the teacher's trait-language patterns.' I'd recommend conditional acceptance with requests for uncertainty estimates, a prompting baseline, and explicit discussion of the teacher-label limitation.","headline":"Genuinely novel adapter for continuous trait control in LLMs, but the message-level teacher labels are the load-bearing risk; worth serious review.","tokens_in":21095,"tokens_out":3809,"would_cite":true,"duration_ms":40617,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Injecting continuous personality scores into every transformer layer makes LLMs produce text that expert raters match to intended traits at 87 to 97 percent accuracy.","keywords":["PsychAdapter","personality-conditioned text generation","Big Five personality","mental health language","parameter-efficient adaptation","transformer language models","text-based psychological assessment","computational psychology"],"falsifier":"Train a PsychAdapter on message-level self-reported personality scores rather than teacher-model estimates and run the same expert-rater matching protocol: if accuracy falls to chance, the reported effect comes from the teacher distillation rather than from the per-layer conditioning architecture itself.","tokens_in":20108,"feed_emoji":"🧠","tokens_out":5897,"duration_ms":52464,"temperature":0.7,"pith_summary":"The paper claims that a lightweight add-on to an ordinary transformer language model can make it generate text reflecting specified personality, demographic, and mental-health profiles, with no prompting required. The key move is to accept a vector of continuous psychological scores and project it, via learned per-layer matrices, into the model's hidden states so the scores influence every layer during training and generation. Trained on social-media and blog text labeled by text-based assessment models, the resulting PsychAdapter produced language that expert raters matched to the intended Big Five levels 87.3% of the time and to intended depression and life-satisfaction levels 96.7% of the time. If the approach works as described, it offers prompt-free, continuous control of trait expression across transformer LLMs and opens a way to study psychological constructs through generated language.","feed_headline":"Tiny adapters let LLMs write like extraverts or depressed people","feed_subtitle":"Trait scores go into every layer, so generated text matches intended personalities without prompt engineering.","key_machinery":"The load-bearing mechanism is a per-layer linear projection of the trait vector into the hidden-state key and value dimensions of an autoregressive transformer, with a separate learned matrix for each layer except the last. The trait vector is fed as the hidden state of a dummy begin-of-sentence token, so the psychological scores condition every layer during the standard next-word reconstruction objective. Training labels come from a teacher-student pipeline: participant-level text-based assessment models, built on topic or lexical features, are applied to each message to produce estimated Big Five, depression, life-satisfaction, or age scores, and the adapter learns to reconstruct the original message from that score vector. The added parameters are tiny relative to the base model, roughly 0.002% to 0.07% depending on the model, which is what allows PsychAdapters to be distributed and plugged into different base LLMs.","core_discovery":"The paper's central claim is that a transformer language model can be made to express a specified psychological profile by conditioning not on prompt words but on a continuous vector of trait scores, projected into the model's hidden states at every layer. PsychAdapter, the proposed modification, learns separate projection matrices for the key and value streams of each transformer layer so that a five-dimensional Big Five vector, or a depression, life-satisfaction, or age score, influences the representation at every layer. After training on messages labeled with scores from participant-level text-based assessment models, the adapter generates text that expert raters matched to the intended low, neutral, or high level with 87.3% average accuracy across Big Five traits and 96.7% for depression and life satisfaction; with age also conditioned, two expert raters reached 100% accuracy. The paper further reports that the same method transfers to GPT-2, Gemma, and Llama3 and produces text at five fine-grained trait levels, and that rotating extraversion and agreeableness inputs traces the interpersonal circumplex.","pith_inferences":["One testable consequence is that the trait signal is bounded by the teacher estimates used to label messages; training the same architecture on message-level self-reported scores, where available, could produce measurably different linguistic patterns than training on participant-level model estimates projected onto messages.","Because the architecture only needs a continuous variable and a text-based teacher model for it, the same adapter design could in principle condition generation on other between-person dimensions such as political orientation, loneliness, or stress, not just the traits demonstrated in the paper.","If PsychAdapters are used to build 'digital participant' cohorts for psychological experiments, the generated cohort's trait-language distribution would inherit the teacher models' biases, so the validity of those cohorts as human proxies should be measured directly before relying on them.","A direct extension would be to interpolate continuously between trait levels and test whether human perceivers recover graded differences in single messages, rather than only the Low, Neutral, and High categories used in the reported evaluation."],"forward_implications":["PsychAdapters make personality and mental-health traits a controllable input dimension of autoregressive LLMs, independent of prompt tokens, so trait expression no longer competes with the prompt for context-window space.","Chatbots and virtual agents could be instantiated with a specified Big Five profile, mental-health state, and age, enabling more human-like interaction and audience-matched content.","Clinical training tools could simulate conversation partners expressing depression or low life satisfaction without risk to patients.","Researchers can generate large, domain-focused text samples across the full continuous range of trait scores, supporting the study of how psychological constructs surface in language.","The same pipeline transfers to different base models, including GPT-2, Gemma, and Llama3, and across text domains such as tweets and blogs, suggesting the conditioning mechanism is model-agnostic."],"supporting_citations":[{"why":"Supplies the open-vocabulary topic features and the participant-level personality model that the pipeline applies to message-level text to label training data.","marker":"[11]"},{"why":"Provides the lexical-based Big Five assessment approach used to produce the estimated personality vectors that PsychAdapter is trained to reconstruct.","marker":"[16]"},{"why":"Provides the text-based depression model used to assign estimated depression scores to training messages.","marker":"[13]"},{"why":"Provides the text-based life-satisfaction model used to label training messages for the well-being condition.","marker":"[56]"},{"why":"Supplies the age-prediction text model used to add demographic conditioning to the mental-health adapters.","marker":"[31]"},{"why":"Provides the blog dataset from 19,320 authors that forms one of the two training text corpora.","marker":"[54]"},{"why":"Provides the large Twitter corpus from which 500,000 posts were sampled as the second training text corpus.","marker":"[55]"}],"fun_headline_variants":["PsychAdapter steers LLM text to match personality traits","Trait vectors at every layer shape LLM writing style","LLMs adopt personality with 87% accuracy via PsychAdapter","Trait-conditioned LLMs hit 96.7% on depression mimicry"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the participant-level text-based assessment models, applied to individual messages to produce the trait labels the adapter trains on, give valid estimates of the intended psychological constructs at message level; if those estimates are wrong, the adapter learns the teacher models' linguistic stereotypes rather than genuine trait-language patterns.","fun_headline_variants_meta":{"raw":{"variants":["PsychAdapter steers LLM text to match personality traits","Trait vectors at every layer shape LLM writing style","LLMs adopt personality with 87% accuracy via PsychAdapter","Trait-conditioned LLMs hit 96.7% on depression mimicry"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000604,"raw_usage":{"total_tokens":2839,"prompt_tokens":987,"completion_tokens":1852,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":1779}},"tokens_in":603,"tokens_out":1852,"duration_ms":12139,"temperature":1.0,"reasoning_tokens":1779,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T06:00:57.757784+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a PsychAdapter on message-level self-reported personality scores rather than teacher-model estimates and run the same expert-rater matching protocol: if accuracy falls to chance, the reported effect comes from the teacher distillation rather than from the per-layer conditioning architecture itself.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the open-vocabulary topic features and the participant-level personality model that the pipeline applies to message-level text to label training data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the lexical-based Big Five assessment approach used to produce the estimated personality vectors that PsychAdapter is trained to reconstruct."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the text-based life-satisfaction model used to label training messages for the well-being condition."},{"cited_title":"& Pennebaker, J","cited_arxiv_id":null,"evidence_quote":"Provides the blog dataset from 19,320 authors that forms one of the two training text corpora."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the large Twitter corpus from which 500,000 posts were sampled as the second training text corpus."}],"review_version":1}