Pith. sign in

REVIEW 6 major objections 6 minor 62 references

Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models

T0 review · 6 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read E2P claims that injecting a pre-computed user embedding into a frozen large language model through a single learned soft prefix token personalizes generation as well as token-heavy prompts, at a fraction of the cost.

desk verdict The core idea is clean and practical, but the paper's own reproduction scripts write zero vectors as user embeddings for Persona-Chat and PENS, so the public-data evidence for personalization does not hold together. read the letter →

arxiv 2505.17051 v1 pith:FTHTTXYY submitted 2025-05-16 cs.CL cs.AI

Bernd Huber , Ghazal Fazelnia , Andreas Damianou , Sebastian Peleato , Max Lefarov , Praveen Ravichandran , Marco De Nadai , Mounia Lalmas-Roellke
show 1 more author
Paul N. Bennett
This is my paper · ORCID
classification cs.CLcs.AI
keywords embedding-to-prefixparameter-efficientpersonalizationsoftprefixtuningfrozenlanguagemodelsuserembeddingsdialoguegenerativerecommendation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

E2P claims that a frozen large language model can be personalized by converting a pre-computed user embedding into a single soft prefix token through a learned two-layer projection, then prepending that token to the input. The paper reports that this one-token intervention beats the no-context baseline on all four evaluated tasks—dialogue (Persona-Chat), headline generation (PENS), and music and podcast recommendation—while training only the projection and leaving the base model untouched. If correct, the method turns existing user-embedding infrastructure into a cheap, low-latency personalization channel for generative models, with no per-user fine-tuning. The claim is that the user signal itself, not the added prefix structure, drives the gains.

What carries the argument

The load-bearing object is the single soft prefix token $p = \phi(c)$, a continuous vector in $\mathbb{R}^{d_h}$ produced by a two-layer MLP with ReLU and LayerNorm, prepended to the input embedding sequence before the frozen transformer processes it. Because the prefix is continuous and generated from the user embedding, it can generalize to unseen users and preserve similarity structure from the user-embedding space. The paper's training objectives are the standard next-token language-modeling loss for generation and a Kahneman–Tversky objective for recommendation engagement, both optimizing only the projection parameters.

What would settle it

Run the paper's published preprocessing (Appendix G) on Persona-Chat, train E2P with the resulting all-zero user vectors, and measure test perplexity; if it matches the reported 24.67, the user embedding carried no signal and the personalization gain is an artifact of the prefix structure. Alternatively, train E2P with real persona embeddings and with zero embeddings under identical settings and compare to E2P-Random.

Watch

Extended reading notes

Core claim

The central claim is that there exists a mapping from user embeddings to the LLM's hidden space that works with a single soft token. Specifically, E2P defines $\phi(c) = \mathrm{LayerNorm}(\mathrm{ReLU}(W_1 c)) W_2 + b$ and inserts $\phi(c)$ as the first token of the embedding sequence, so the modified input is $[\phi(c); E(x)]$. With the backbone frozen, only $\phi$ is trained, via a language-modeling loss for generation tasks and Kahneman–Tversky optimization for engagement prediction. Across the four tasks, E2P reports perplexity 24.67 versus 45.40 for no context on Persona-Chat, ROUGE-L 16.75 versus 14.66 on PENS, +12.9% predicted engagement on music, and +2.2% hit-rate@30 on podcasts, with statistical significance against the no-context baseline.

Load-bearing premise

The load-bearing premise is that the reported experiments actually conditioned on distinct, meaningful user embeddings — the published preprocessing code for Persona-Chat and PENS writes a constant zero vector for every user, and the proprietary recommendation results assume the behavioral embeddings and the engagement-classifier proxy faithfully represent user preference.

Editorial extensions

If this is right

  • Personalization becomes parameter-efficient: only the projection module is trained, so adapting to new users costs a few million parameters rather than a fine-tuned copy of the base model.
  • The single-token prefix adds negligible decoding latency, making the approach deployable in real-time recommendation and dialogue systems.
  • Because user embeddings can be updated continuously, E2P can reflect evolving preferences without retraining the projection unless the embedding distribution shifts.
  • Dense user embeddings can complement textual prompts: the paper reports that E2P plus a textual prompt beats either alone on podcast recommendation.
  • The method generalizes across generative and retrieval-style tasks, from open-ended dialogue to semantic-ID-based item ranking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The published preprocessing scripts write a constant zero vector as the user embedding for every row on Persona-Chat and PENS; if those scripts were what actually trained the model, the reported gains over E2P-Random would not be explainable by user-specific information, and the two conditions should collapse. This means the paper's headline public-dataset results may rest on a code-vs-description
  • A direct way to test the method's core premise is to train E2P on Persona-Chat with real persona embeddings and compare against the zero-vector condition; if perplexity does not differ, the single-token prefix is not carrying user identity.
  • The engagement-classifier proxy on music recommendation is the only evidence for user satisfaction; an online A/B test with actual save/listen behaviour would settle whether the +12.9% translates to real engagement.
  • The single-token design suggests a broader principle: that a frozen LLM's conditioning can be a learned continuous function of arbitrary auxiliary embeddings, which could extend to other dense contextual signals (time, location, session state) beyond user profiles.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper proposes Embedding-to-Prefix (E2P), a parameter-efficient personalization method that conditions a frozen LLM on a precomputed user embedding c by projecting c through a two-layer MLP with LayerNorm (Eq. 2) into a single soft prefix token prepended to the input embedding sequence (Eq. 3). The projection is trained with a language-modeling objective (Eq. 4) for text tasks and with Kahneman-Tversky Optimization (Eq. 5) for engagement tasks, while all base-model weights stay frozen. E2P is evaluated on Persona-Chat (perplexity), PENS (ROUGE), and two proprietary Spotify tasks: music playlist generation (predicted-engagement proxy) and podcast next-item recommendation (hit-rate@30). Table 1 reports gains over a no-context baseline on all four tasks (24.67 vs. 45.40 perplexity; ROUGE-L 16.75 vs. 14.66; +12.9% predicted engagement; +2.2% hit-rate@30), with an E2P-Random control intended to separate prefix-structure effects from genuine personalization. The appendices describe the splits, hyperparameters, significance tests, and preprocessing scripts, which Appendix F presents as the code that reproduces the data splits.

Significance. The central idea is appealing, and if the reported results held, E2P would be a practical contribution: injecting existing dense user embeddings through a single learned soft prefix keeps the backbone frozen, adds negligible inference overhead, and can generalize to new users through embedding similarity. The random-embedding control is a sound design choice, the paper addresses a real gap (standard PEFT methods do not consume external user vectors), and the authors ship their preprocessing scripts, which makes the claims externally checkable. The check, however, fails: the shipped scripts contradict the described method on both public datasets, and the two proprietary evaluations rest on a proxy whose external validity is asserted without data. Because the evidence chain is broken at its most verifiable point, the significance of the contribution cannot be assessed from this version of the manuscript.

major comments (6)
  1. [Appendix G.1/G.2; Section 4.1; Eq. (3); Table 1] The reproduction scripts shipped in the appendix directly contradict the method description. Section 4.1 states that Persona-Chat personas and PENS user profiles are embedded with Llama-3.2-1B into a single 2048-dimensional vector per persona or profile, and Appendix F presents the code in G.1 and G.2 as 'Code to reproduce our data splits.' In fact, G.1 and G.2 write 'uservector': [0.]*2048 for every row of both public datasets. If those scripts were used, E2P on Persona-Chat and PENS receives a constant input, so the Table 1 gains (24.67 vs. 45.40 perplexity and ROUGE-L 16.75 vs. 14.66) are effects of a single static learned prefix rather than of user-specific context; moreover, E2P and E2P-Random would then be the same model, making the 24.67 vs. 26.56 perplexity gap interpretable only as training variance. If the scripts were not used, the public-data results are not reproducible from the manuscript. Either way, the central claim that precomputed context embeddings are injected through a learned projection into a single soft token (Eq. 3) is unsupported by the two public datasets.
  2. [Appendix F.1 vs. Appendix G.1] Appendix F.1 says the Persona-Chat data are 'shuffled with seed 42 and partitioned 80/10/10 by persona id, ensuring no persona appears in more than one split,' but the code in G.1 shuffles individual rows with random.shuffle(rows) and splits by row index. The footnote counts (train 49,778, dev 6,222, test 6,223) are exactly 80/10/10 of the 62,223 rows, which is consistent with a row-level split and not with grouping by persona id. Under the shipped code, the same persona can appear in both training and test splits, so the claimed leakage control is not implemented; since the persona description is what the E2P embedding encodes, persona leakage could inflate the reported personalization gap.
  3. [Section 3.4.1; Eq. (4); Appendix G.1] The shipped Persona-Chat file cannot be used to train the objective of Eq. (4). Equation (4) requires a target sequence y given an input x and user vector c, but G.1 writes each row with only a 'uservector' and a single 'text' field that concatenates the persona, h[0], and h[1]; no target response is stored, and the persona description is placed inside the textual prompt rather than being carried by the user vector. If training was instead performed on this 'text' field as a language-modeling target, the model would be trained to reproduce the persona text, which is a different task from the one described in Section 4.1 and would make the comparison with the Prompt-Persona baseline difficult to interpret.
  4. [Section 4.2 vs. Appendix D] The experimental settings in the main text and the appendix are mutually inconsistent. Section 4.2 states that 'LLaMA-3.2-3B [is] the base model for all experiments' with a learning rate of 5e-6, batch size 32, and 5 training epochs, while Appendix D lists Llama-3.1-8B for Music Rec and Llama-3.2-1B for Podcast Rec, with batch sizes 64 (Persona-Chat), 256 (Music Rec), and 16 (Podcast Rec), and 3 epochs for Persona-Chat and PENS. Since the Table 1 gains and significance markers depend on these settings, the paper as written does not provide a consistent experimental record for any of the four tasks, and the claimed parameter-efficiency comparison cannot be reconstructed.
  5. [Section 3.4.2; Eq. (5)] Equation (5) is not the Kahneman-Tversky Optimization objective of the cited reference [8]. The loss in Eq. (5) is a weighted binary cross-entropy with weight alpha on the negative class; the KTO objective of Ethayarajh et al. is defined through an implicit reward r_KTO(x,y) = beta log(p_theta(y|x)/p_ref(y|x)) - KL(pi_theta || pi_ref), with a sigmoid applied to the difference between that reward and a reference threshold, and it contains no explicit alpha-weighted negative-class term. Because Eq. (5) is the stated training objective for both Music Rec and Podcast Rec, the paper should either correct the attribution and name the actual loss used (asymmetric weighted binary cross-entropy) or remove the claim that E2P uses KTO.
  6. [Eq. (5); Appendix F.1; Table 1] The headline Music Rec result (+12.9% predicted engagement, Table 1) is measured by a separate binary engagement classifier (Appendix F.1), and E2P is trained with the objective of Eq. (5) against the same engagement-label distribution. The paper asserts in F.1 that improvements in this offline proxy 'have correlated with online gains in user satisfaction in related production experiments' but provides no data supporting that correlation. As written, the Music Rec numbers demonstrate only that E2P optimizes the proxy; the claim that E2P improves real user engagement requires evidence of external validity (for example, the correlation statistic or a small online A/B result), without which the production-scale personalization claim remains unverified.
minor comments (6)
  1. [Table 1] The header of the last block contains the typo 'Podcacst Rec'; it should read 'Podcast Rec.'
  2. [Sections 3.1 and 3.3] The sentence about single-token interventions being effective, citing [25], appears twice with nearly identical wording (Section 3.1: 'This aligns with prior findings that even single-token interventions can be effective'; Section 3.3: 'This design aligns with prior findings that single-token interventions can be surprisingly effective'). One of the two should be removed.
  3. [Section 4.2] The E2P-Random control is defined as 'a prefix generated from a randomly selected user embedding from the test set,' but it is not stated whether the control is trained with random embeddings at training time or only evaluated with them at test time; the two readings support different interpretations of the Table 1 gaps, so the definition should be made explicit.
  4. [Table 1 caption] The significance-marker legend is confusing: it states that symbols '(‡, ⋄)' are significant 'compared to E2P,' but in the Podcast Rec block the E2P + Prompt row carries both symbols, which suggests the second symbol compares against a different baseline (presumably Prompt-Context); the legend should assign exactly one referent to each symbol.
  5. [Section 1] The paragraph beginning 'Although methods such as Parameter-Efficient Fine-Tuning (PEFT) have emerged...' is a sentence fragment ending in '[27, 25, 17]' followed by a new sentence; it should be reworded grammatically.
  6. [Section 4.2; References] The 'Embedding Retrieval/Reranking' baseline is described only by reference to a GNN model [6], with no retrieval candidate set or implementation details, so the +0.1% and +1.2% baseline numbers cannot be reproduced; in addition, the reference list is inconsistent in capitalization and venue formatting throughout and should be normalized to the journal's style.

Circularity Check

2 steps flagged · score 6.0 of 10

Public-data preprocessing scripts zero out every user vector, collapsing Persona-Chat/PENS to a static prefix, and the music-recommendation gain is measured by an engagement proxy trained on the same labels E2P optimizes; the claimed personalization is only partially supported.

  1. other [Appendix G.1 and G.2 (reproduced pre-processing scripts) vs Section 4.1 (datasets)]
    "G.1: f.write(json.dumps({"uservector": [0.]*2048, "text": prompt})+"\n"); G.2: out.append({"uservector": [0.]*2048, "prompt": p, "completion": h, "label": bit})"

    Section 4.1 says each Persona-Chat persona description and each PENS profile is embedded once with Llama-3.2-1B to obtain a 2048-dimensional user embedding. The two 'Code to reproduce our data splits' scripts instead write the constant zero vector for every row. Under Eq. (3), c is therefore identically zero, so phi(c) is a fixed prefix; the claimed injection of user-specific context reduces by construction to a static soft token. The Persona-Chat and PENS results in Table 1 (24.67 vs 45.40 perplexity; ROUGE-L 16.75 vs 14.66) and the E2P-Random comparison cannot then demonstrate personalization.

  2. fitted input called prediction [Section 3.4.2, Eq. (5); Appendix A; Appendix F.1]
    "Given engagement labels yi∈{0,1}, the objective is: LKT(c) = Σ yi log pθ(yi|[p;xi]) + α(1−yi) log(1−pθ(yi|[p;xi])) (5). ... Music Rec: We evaluate this task using a binary engagement classifier trained on historical user interaction data ... predicts the binary signal whether or not a user engages."

    The E2P projection is trained with Eq. (5) to maximize the probability of engagement labels, i.e., it is fitted to predict exactly the signal that the Appendix F.1 evaluation classifier was trained to predict from the same user embeddings, queries, and track embeddings. The headline Music Rec gain (+12.9% predicted engagement) is therefore not an independent behavioral outcome but a score under a proxy fitted to the same target variable used in training. Eq. (5) and the evaluation classifier are not the same function, but the reported 'prediction' is a fit-to-label metric; the paper itself calls it 'a proxy metric for user satisfaction' (Appendix A), so the result is partially circular by construction rather than an external validation.

full rationale

The core text-generation evaluation is standard held-out supervised training and is not circular: perplexity and ROUGE are computed against ground-truth text, and no load-bearing claim rests on a self-citation. However, the paper's public-dataset validation is undermined by the shipped preprocessing scripts that set every user vector to zero, which collapses E2P to a static prefix and makes the Persona-Chat/PENS results incapable of supporting the personalization claim. In addition, the music-recommendation headline is measured by an engagement classifier trained on the same kind of engagement labels that E2P is optimized to predict, making the +12.9% gain a partially self-scored proxy rather than an independent measurement. These are concrete, quotable reductions, so the score is elevated to 6; the remaining podcast hit-rate and text metrics provide some independent content, preventing a higher score.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The method's learned component is the two-layer MLP (W1, W2, b) fitted per task; these are the intended trainable parameters, not hidden postulates. Two hand-chosen values, the KTO weight alpha and the projection hidden dimension d_c, are never reported. The soft prefix token is borrowed from prefix tuning and has no external falsifiable handle. The three axioms are the paper's empirical premises: user embeddings encode usable preference, a single token can steer a frozen LLM, and the engagement proxy measures real satisfaction. None of the axioms is independently verified in the paper, and the first is contradicted by the shipped zero-vector preprocessing code for the public datasets.

free parameters (3)
  • KTO weighting alpha = Not reported in the manuscript.
    Equation (5) defines the recommendation objective with alpha balancing positive and negative engagement terms; its value is never stated, and it shapes the music and podcast training signal.
  • Projection hidden dimension d_c = Not reported in the manuscript.
    Equation (2) defines W1 in R^{d_c x d} and W2 in R^{d_c x d_h}; d_c is unspecified, so the size of the trained module is unknown.
  • Projection weights W1, W2, b = Trained on each task.
    These are the intended learned parameters of E2P; they are listed for completeness as the method's fitted component rather than as hidden postulates.
assumptions (3)
  • domain assumption Pre-computed user embeddings (120-d behavioral vectors and Llama persona embeddings) carry preference information useful for conditioning generation.
    The entire method presupposes that the given user vector contains a decodable personalization signal; for the public datasets, the shipped preprocessing code writes zero vectors, so this premise is not implemented as claimed.
  • domain assumption A single soft token prefix is sufficient to steer a frozen LLM's generation.
    Section 3.1 justifies this by citing prior findings in Lester et al. [25]; it is an empirical premise about the expressiveness of one token, not proven in this paper.
  • ad hoc to paper The engagement-classifier proxy measures real user satisfaction.
    Appendix F.1 asserts that offline-proxy gains have correlated with online gains in related production experiments, but no correlation data are presented; the music and podcast claims are evaluated through this proxy.
invented entities (1)
  • Soft prefix token p
    purpose: Carries the user embedding into the frozen LLM's hidden space at position 0 of the input sequence.
    Borrowed from prefix tuning [27]; it is a trainable vector with no external falsifiable handle, so it is a design element rather than an independently testable new entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models." pith.science (2026). https://pith.science/paper/FTHTTXYY

@misc{pith2026250517051,
  author       = {Pith},
  title        = {Pith review of: Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FTHTTXYY}},
  note         = {Machine review of arXiv:2505.17051}
}
read the original abstract

Large language models (LLMs) excel at generating contextually relevant content. However, tailoring these outputs to individual users for effective personalization is a significant challenge. While rich user-specific information often exists as pre-existing user representations, such as embeddings learned from preferences or behaviors, current methods to leverage these for LLM personalization typically require costly fine-tuning or token-heavy prompting. We propose Embedding-to-Prefix (E2P), a parameter-efficient method that injects pre-computed context embeddings into an LLM's hidden representation space through a learned projection to a single soft token prefix. This enables effective personalization while keeping the backbone model frozen and avoiding expensive adaptation techniques. We evaluate E2P across two public datasets and in a production setting: dialogue personalization on Persona-Chat, contextual headline generation on PENS, and large-scale personalization for music and podcast consumption. Results show that E2P preserves contextual signals and achieves strong performance with minimal computational overhead, offering a scalable, efficient solution for contextualizing generative AI systems.

Figures

Figures reproduced from arXiv: 2505.17051 by the authors.

Figure 1
Figure 1. Overview of Embedding-to-Prefix (E2P). A projection module maps a user embedding [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Visualization of user embedding properties in the LLM’s hidden representation space: (a) t [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 30 canonical work pages

  1. [1]

    Pens: A dataset and generic framework for personalized news headline generation

    Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. Pens: A dataset and generic framework for personalized news headline generation. In proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 82–92, 2021

    work page 2021

  2. [8]

    Kto: Model alignment as prospect theoretic optimization

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306, 2024

    arXiv 2024

  3. [2]

    Transparent, scrutable and explainable user models for personalized recommendation

    Krisztian Balog, Filip Radlinski, and Shushan Arakelyan. Transparent, scrutable and explainable user models for personalized recommendation. In Proceedings of the 42nd international acm sigir conference on research and development in information retrieval, pages 265–274, 2019

    work page 2019

  4. [3]

    Persona: A reproducible testbed for pluralistic alignment

    Louis Castricato, Nathan Lile, Rafael Rafailov, Jan-Philipp Frankën, and Chelsea Finn. Persona: A reproducible testbed for pluralistic alignment. In proceedings of the 2024 advances in neural information processing systems (neurips 2024), 2024

    work page 2024

  5. [4]

    Maxmin-rlhf: Alignment with diverse human preferences

    Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Furong Huang, Dinesh Manocha, Amrit Singh Bedi, and Mengdi Wang. Maxmin-rlhf: Alignment with diverse human preferences. arXiv preprint arXiv:2402.08925, 2024

    arXiv 2024

  6. [5]

    Direct preference optimization with unobserved preference heterogeneity

    Keertana Chidambaram, Karthik Vinay Seetharaman, and Vasilis Syrgkanis. Direct preference optimization with unobserved preference heterogeneity. arXiv preprint arXiv:2405.15065, 2024

    arXiv 2024

  7. [6]

    Personalized audiobook recommendations at spotify through graph neural networks

    Marco De Nadai, Francesco Fabbri, Paul Gigioli, Alice Wang, Ang Li, Fabrizio Silvestri, Laura Kim, Shawn Lin, Vladan Radosavljevic, Sandeep Ghael, et al. Personalized audiobook recommendations at spotify through graph neural networks. In Companion Proceedings of the ACM Web Conference 2024, pages 403–412, 2024

    work page 2024

  8. [7]

    User embedding model for personalized language prompting

    Sumanth Doddapaneni, Krishna Sayana, Ambarish Jash, Sukhdeep Sodhi, and Dima Kuzmin. User embedding model for personalized language prompting. arXiv preprint arXiv:2401.04858, 2024

    arXiv 2024

Show all 62 references
  1. [9]

    Stylept: Personalized neural text style transfer via prompt tuning.proceedings of the 37th aaai conference on artificial intelligence (aaai 2023), pages 9135–9143, 2023

    Ning Fan, Yichao Mao, Zhiwu Liu, Jinge Zhou, Hongzhi Zhang, and Weiguo Liu. Stylept: Personalized neural text style transfer via prompt tuning.proceedings of the 37th aaai conference on artificial intelligence (aaai 2023), pages 9135–9143, 2023

    2023

  2. [10]

    Generalized user representations for transfer learning

    Ghazal Fazelnia, Sanket Gupta, Claire Keum, Mark Koh, Ian Anderson, and Mounia Lalmas. Generalized user representations for transfer learning. arXiv preprint arXiv:2403.00584, 2024

    2024 arXiv

  3. [11]

    In-context autoencoder for context compression in a large language model

    Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. arXiv preprint arXiv:2307.06945, 2023

    2023 arXiv

  4. [12]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  5. [13]

    Transformer with memory as personalized gpt for dialogue agents

    Kye Han, Dae-Hyun Seo, Minkyoung Seo, Im-Woo Choi, and Hwanjun Park. Transformer with memory as personalized gpt for dialogue agents. In proceedings of the 2019 world wide web conference (www 2019), pages 603–608, 2019

    2019

  6. [14]

    Neural collaborative filtering

    Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173–182, 2017

    2017

  7. [15]

    Persoma: Personalized soft prompt adapter architecture for personalized language prompting

    Liam Hebert, Krishna Sayana, Ambarish Jash, Alexandros Karatzoglou, Sukhdeep Sodhi, Sumanth Doddapaneni, Yanli Cai, and Dima Kuzmin. Persoma: Personalized soft prompt adapter architecture for personalized language prompting. arXiv preprint arXiv:2408.00960, 2024

    2024 arXiv

  8. [16]

    Personalized conversational agents for empathetic support

    Elnaz Hosseini-Asl, Ananya Cowen, and Balaji Reddy. Personalized conversational agents for empathetic support. arxiv preprint arxiv:2112.00507, 2021. 10

    2021 arXiv

  9. [17]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  10. [18]

    Lapdog: Learning retrieval augmentation for personalized dialogue generation

    Qiushi Huang, Shuai Fu, Xubo Liu, Wenwu Wang, Tom Ko, Yu Zhang, and Lilian Tang. Lapdog: Learning retrieval augmentation for personalized dialogue generation. In proceedings of the 2023 conference on empirical methods in natural language processing (emnlp 2023), pages 15383–15...

    2023

  11. [19]

    Aligning language models to user opinions

    EunJeong Hwang, Bodhisattwa Prasad Majumder, and Niket Tandon. Aligning language models to user opinions. arXiv preprint arXiv:2305.14929, 2023

    2023 arXiv

  12. [20]

    Personalized soups: Per- sonalized large language model alignment via post-hoc parameter merging

    Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. Personalized soups: Per- sonalized large language model alignment via post-hoc parameter merging. arXiv preprint arXiv:2310.11564, 2023

    arXiv

  13. [21]

    Personalized language models via privacy- preserving evolutionary model merging

    Kyuyoung Kim, Jinwoo Shin, and Jaehyung Kim. Personalized language models via privacy- preserving evolutionary model merging. arxiv preprint arxiv:2503.18008, 2025

    2025

  14. [22]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  15. [23]

    Compo: Community preferences for language model personalization

    Sachin Kumar, Chan Young Park, Yulia Tsvetkov, Noah A Smith, and Hannaneh Ha- jishirzi. Compo: Community preferences for language model personalization. arXiv preprint arXiv:2410.16027, 2024

    2024 arXiv

  16. [24]

    P5: Plug-and-play persona prompting for personal- ized response selection

    Joosung Lee, Minsik Oh, and Donghun Lee. P5: Plug-and-play persona prompting for personal- ized response selection. In proceedings of the 2023 conference on empirical methods in natural language processing (emnlp 2023), pages 16571–16582, 2023

    2023

  17. [25]

    The power of scale for parameter-efficient prompt tuning

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In proceedings of the 2021 conference on empirical methods in natural language processing (emnlp 2021), pages 3045–3059, 2021

    2021

  18. [26]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  19. [27]

    Prefix-tuning: Optimizing continuous prompts for generation

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021

    2021 arXiv

  20. [28]

    Personalized item embeddings in federated multimodal recommendation

    Zhiwei Li, Guodong Long, Jing Jiang, and Chengqi Zhang. Personalized item embeddings in federated multimodal recommendation. In proceedings of the 2024 international conference on learning representations (iclr 2024), 2024

    2024

  21. [29]

    500xcompressor: Generalized prompt compression for large language models

    Zongqian Li, Yixuan Su, and Nigel Collier. 500xcompressor: Generalized prompt compression for large language models. arXiv preprint arXiv:2408.03094, 2024

    2024 arXiv

  22. [30]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summarization branches out, pages 74–81, 2004

    2004

  23. [31]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  24. [32]

    A survey of personalized large language models

    Jiahong Liu, Zexuan Qiu, Zhongyang Li, Quanyu Dai, Jieming Zhu, Minda Hu, Menglin Yang, and Irwin King. A survey of personalized large language models. arxiv preprint arxiv:2502.11528, 2025

    2025

  25. [33]

    Llms + persona-plug = personalized llms

    Xiaodong Liu, Liang Yu, Xiao Sun, Hao Fu, Kaili Zhang, Jingjing Wang, Mingyuan Fan, Shuyang Tai, Zhiyong Li, and Jian Gao. Llms + persona-plug = personalized llms. In proceed- ings of the 2024 advances in neural information processing systems (neurips 2024), 2024. 11

    2024

  26. [34]

    Xrec: Large language models for explainable recommendation

    Qiyao Ma, Xubin Ren, and Chao Huang. Xrec: Large language models for explainable recommendation. In findings of the 2024 conference on empirical methods in natural language processing, pages 391–402, 2024

    2024

  27. [35]

    Personalizing dialogue agents via meta-learning

    Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. Personalizing dialogue agents via meta-learning. transactions of the association for computational linguistics (tacl), 7:327–342, 2019

    2019

  28. [36]

    Personalized paraphrasing: Controlling the formality and style of text

    Mehri Niknam, Tim Liu, Mu Qu, and Fangqing Tu. Personalized paraphrasing: Controlling the formality and style of text. In proceedings of the 60th annual meeting of the association for computational linguistics (acl 2022), pages 13457–13470, 2022

    2022

  29. [37]

    User-llm: Efficient llm contextualization with user embeddings

    Lin Ning, Luyang Liu, Jiaxing Wu, Neo Wu, Devora Berlowitz, Sushant Prakash, Bradley Green, Shawn O’Banion, and Jun Xie. User-llm: Efficient llm contextualization with user embeddings. arXiv preprint arXiv:2402.13598, 2024

    2024 arXiv

  30. [38]

    Recommender systems with generative retrieval

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al. Recommender systems with generative retrieval. Advances in Neural Information Processing Systems, 36:10299–10315, 2023

    2023

  31. [39]

    Lamp: When large language models meet personalization

    Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization. arxiv preprint arxiv:2304.11406, 2023

    2023 arXiv

  32. [40]

    Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971–30004

    Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971–30004. PMLR, 2023

    2023

  33. [41]

    Llamafusion: Adapting pretrained language models for multimodal generation

    Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, and Lili Yu. Llamafusion: Adapting pretrained language models for multimodal generation. arXiv preprint arXiv:2412.15188, 2024

    2024 arXiv

  34. [42]

    Better generalization with semantic ids: A case study in ranking for recommendations

    Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tandon, et al. Better generalization with semantic ids: A case study in ranking for recommendations. In Proceedings of the 18th ACM Conf...

    2024

  35. [43]

    A multi-stage approach for persona-aware response generation

    Hanwang Song, Weiping Wang, Xixiu Zha, Xiang Fang, and Xianrui Cao. A multi-stage approach for persona-aware response generation. In proceedings of the 58th annual meeting of the association for computational linguistics (acl 2020), pages 677–687, 2020

    2020

  36. [44]

    Persona-db: Efficient large language model personalization for response prediction with collaborative data refinement

    Chenkai Sun, Ke Yang, Revanth Gangi Reddy, Yi R Fung, Hou Pong Chan, Kevin Small, ChengXiang Zhai, and Heng Ji. Persona-db: Efficient large language model personalization for response prediction with collaborative data refinement. arXiv preprint arXiv:2402.11060, 2024

    2024 arXiv

  37. [45]

    Democra- tizing large language models via personalized parameter-efficient fine-tuning

    Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. Democra- tizing large language models via personalized parameter-efficient fine-tuning. arXiv preprint arXiv:2402.04401, 2024

    2024 arXiv

  38. [46]

    Step-back profiling: Distilling user history for personalized scientific writing

    Xiangru Tang, Xingyao Zhang, Xiaojin Yin, Yuwei Niu, Yadong Wang, Kangyong Dong, Xiaodan Liu, and Irwin King. Step-back profiling: Distilling user history for personalized scientific writing. arxiv preprint arxiv:2406.14275, 2024

    2024 arXiv

  39. [47]

    Demystifying embed- ding spaces using large language models

    Guy Tennenholtz, Yinlam Chow, Chih-Wei Hsu, Jihwan Jeong, Lior Shani, Azamat Tulep- bergenov, Deepak Ramachandran, Martin Mladenov, and Craig Boutilier. Demystifying embed- ding spaces using large language models. arXiv preprint arXiv:2310.04475, 2023

    arXiv

  40. [48]

    Modeling users’ personality for response generation in conversational agent

    Xing Wu, Chenghua Qu, Wenhao Yang, and Zhenghao Gu. Modeling users’ personality for response generation in conversational agent. In proceedings of the 2020 international joint conference on neural networks (ijcnn 2020), pages 1–8, 2020

    2020

  41. [49]

    Large language model as a universal clinical multi-task decoder

    Yujiang Wu, Hongjian Song, Jiawen Zhang, Xumeng Wen, Shun Zheng, and Jiang Bian. Large language model as a universal clinical multi-task decoder. arxiv preprint arxiv:2406.12738, 2024. 12

    2024 arXiv

  42. [50]

    Building your own chatbot with customized persona

    Pengfei Yan, Yongchao Ding, Chao Chen, Jing Liu, and Xu Zhao. Building your own chatbot with customized persona. In proceedings of the 2020 international joint conference on artificial intelligence (ijcai 2020), pages 4238–4244, 2020

    2020

  43. [51]

    Personalized llm response generation with parame- terized user memory injection

    Kai Zhang, Yejin Kim, and Xiaozhong Liu. Personalized llm response generation with parame- terized user memory injection. arxiv preprint arxiv:2404.03565, 2024

    2024 arXiv

  44. [52]

    Llama-adapter: Efficient fine-tuning of large language models with zero-initialized attention

    Renrui Zhang, Jiaming Han, Chris Liu, Aojun Zhou, Pan Lu, Yu Qiao, Hongsheng Li, and Peng Gao. Llama-adapter: Efficient fine-tuning of large language models with zero-initialized attention. In The Twelfth International Conference on Learning Representations, 2024

    2024

  45. [53]

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages ...

    2018

  46. [54]

    Personalize your llm: Fake it then align it

    Yijing Zhang, Dyah Adila, Changho Shin, and Frederic Sala. Personalize your llm: Fake it then align it. naacl 2025 findings, 2025

    2025

  47. [55]

    Less is more: Learning to refine dialogue history for personalized dialogue generation

    Hanxun Zhong, Zhicheng Dou, Yutao Zhu, Hongjin Qian, and Ji-Rong Wen. Less is more: Learning to refine dialogue history for personalized dialogue generation. In proceedings of the 2022 conference of the north american chapter of the association for computational linguistics: h...

    2022

  48. [56]

    Useradapter: Few-shot user learning in sentiment analysis

    Wanjun Zhong, Duyu Tang, Jiahai Wang, Jian Yin, and Nan Duan. Useradapter: Few-shot user learning in sentiment analysis. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1484–1488, 2021

    2021

  49. [57]

    Difflm: Controllable synthetic data generation via diffusion language models

    Ying Zhou, Xinyao Wang, Yulei Niu, Yaojie Shen, Lexin Tang, Fan Chen, Ben He, Le Sun, and Longyin Wen. Difflm: Controllable synthetic data generation via diffusion language models. arxiv preprint arxiv:2411.03250, 2024

    2024 arXiv

  50. [58]

    world knowledge

    Yuchen Zhuang, Haotian Sun, Yue Yu, Rushi Qiang, Qifan Wang, Chao Zhang, and Bo Dai. Hydra: Model factorization framework for black-box llm personalization. arXiv preprint arXiv:2406.02888, 2024. 13 A Music Recommendation: Background The large-scale music recommendation task i...

    2024 arXiv

  51. [59]

    We start from the “cleaned” HuggingFace edition (version dated 2024-03-01, 122,499 dialogues)

    Persona-Chat. We start from the “cleaned” HuggingFace edition (version dated 2024-03-01, 122,499 dialogues). We keep every triple (<user0, bot0, user1>) that contains exactly 3 exchanges, yielding 62,223 instances. The data are shuffled with seed 42 and partitioned 80/10/10 by...

    2024

  52. [60]

    We use the official v7 TSV dump distributed on Kaggle and respect its train/valid/personalized_test partition

    PENS. We use the official v7 TSV dump distributed on Kaggle and respect its train/valid/personalized_test partition. From every impression we create up to three positive and three negative (⟨)body, headline⟩) pairs (see next section for the exact loop). Each user profile strin...

  53. [61]

    This internal dataset consists of 300,000 user queries and generated playlists

    Music Playlist Generation. This internal dataset consists of 300,000 user queries and generated playlists. Each row is labeled with an engagement annotation by the listener ((50%/50%)). 1,000 queries were randomly sampled chosen as test set. Every user row is joined with the l...

  54. [62]

    AlekseyKorshuk/persona-chat

    Next Podcast Recommendation. This internal dataset consists of 2,000,000 pairs of previously listened podcasts, metadata, and user context, and a testset of 10,000 samples. Labels are single semantic IDs; evaluation uses hit-rate@30. F.1 Evaluation Metrics Additional informati...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.