REVIEW 4 major objections 6 minor 1 cited by
CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs
T0 review · 4 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read CURP claims that a user's identity can be compressed into a few discrete, interpretable prototypes in a shared codebook, letting a frozen 8B LLM generate personalized text with only about 20M trainable parameters.
desk verdict The codebook idea is worth a look, but the BLEU scores in Table 1 are mathematically impossible, which undermines the empirical claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The core object is the prototype codebook C: 1,000 learnable vectors in each of 4 subspaces, built by balanced k-means on frozen encoder embeddings of user histories, kept frozen during adapter training. Product quantization maps each user event embedding to 4 discrete indices (e.g., [505, 910, 204, 56]), one per subspace; a 2-layer MLP projects these quantized embeddings into the frozen LLM's input space so they can be prepended to the query. The codebook does the load-bearing work: it turns noisy text histories into compact, interpretable, transferable user traits, and the training losses (quantization, diversity, usage) are designed to keep all 1,000 entries active and spread out.
What would settle it
Train the codebook on Reddit QA only, freeze it, and evaluate CURP on a held-out domain that is semantically distant (e.g., medical forum posts or code review comments) against a version whose codebook was trained on that domain's own histories. If the cross-domain version does not beat zero-shot or falls far behind the in-domain version on ROUGE/BLEU/similarity, the generalization claim is refuted. Additionally, the privacy claim could be tested by attempting to reconstruct user histories from codebook indices using the public encoder and codebook.
Extended reading notes
Core claim
The central claim is that user-specific generation can be reduced to selecting a small combination of shared semantic prototypes. CURP encodes each historical behavior with a bidirectional encoder, quantizes the embedding into 4 subspaces via product quantization against a balanced k-means codebook of 1,000 entries, and then concatenates the quantized prototype embeddings (projected by an MLP) to the task query inside a frozen LLM. Training has two stages: constructing the codebook from ~24M Reddit QA histories, and aligning prototypes to generation by maximizing likelihood of the user's response. The method reports best scores on most metrics across four tasks with only ~20M trainable param
Load-bearing premise
The load-bearing premise is that prototypes learned from Reddit question-answer histories are universal enough to represent user traits in other domains (news, tweets, reviews); if those prototypes do not transfer, the headline results on the other three tasks lose their support.
Editorial extensions
If this is right
- A single 20M-parameter adapter can personalize a frozen 8B LLM across multiple tasks, eliminating per-user fine-tuning costs for large user populations.
- The representation is compact enough for cloud-edge pipelines: only 4 discrete indices per history (or per user) need to be transmitted, not raw text or dense embeddings.
- The codebook's subspaces are interpretable: topic anchors, content type, and sentiment/stance can be read off the indices, enabling controllable generation by editing prototype combinations.
- Because the codebook is frozen and shared, the method is model-agnostic; the paper shows competitive results when swapping encoder and decoder backbones.
Reading between the lines
- A natural extension is to treat codebook indices as a compact user feature for standard recommender or RL personalization, where the same discrete code could condition multiple surface tasks; the paper does not explore this.
- The privacy claim rests on indices being non-invertible; a testable follow-up is whether an adversary with the codebook and encoder can reconstruct rough history content from indices, and whether differential privacy would be needed.
- The paper's own limitation—discarding fine-grained lexical habits—suggests a hybrid that augments prototypes with a short textual snippet or a per-user token might beat either extreme; that is inference, not their experiment.
- Cross-domain transfer is measured only by codebook utilization (998/1000 entries used on RQ, 524 on NH, etc.); the real test would be reporting generation metrics on a codebook trained exclusively on Reddit and applied to a genuinely distant domain, e.g., medical advice or code generation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CURP, a framework for personalizing LLM generation through a bidirectional user encoder and a discrete prototype codebook. User histories are encoded with Contriever, quantized via product quantization into a balanced k-means codebook, and projected through an MLP into a frozen Qwen-2.5 LLM. Training proceeds in two stages: prototype codebook construction (PCC) on ALIGNX Reddit QA histories, and prototype behavior aligning (PBA) on task-specific data. Experiments on Reddit QA, News Headline, Tweet Paraphrase, and Review Writing compare CURP against prompting and PEFT baselines using ROUGE, BLEU, and semantic similarity. The paper claims state-of-the-art results on 70% of metrics with only 20M trainable parameters, and reports additional analyses for interpretability, codebook design, cross-architecture generalization, and history length robustness.
Significance. If the reported results are correct, CURP provides a practical middle ground between expensive per-user fine-tuning and noisy text prompting. The decoupling of a shared interpretable codebook from the decoder, the small parameter overhead, and the reported gains across multiple tasks would be useful contributions to personalized generation. The paper also ships a public codebase and specifies a deterministic inference setting, which aids reproducibility. However, the empirical claims rest on several metrics whose validity is currently questionable and on ablations that do not demonstrate a performance contribution from the codebook itself.
major comments (4)
- [§5.1, Table 1] The reported BLEU values for Tweet Paraphrase (LoRA=110, PPlug=109, CURP=108) are mathematically impossible: BLEU is bounded above by 100 when expressed as a percentage, since both the brevity penalty and modified n-gram precision are at most 1. A similar anomaly appears in Table 3, where the Contriever+LLaMA row reports News Headline BLEU as .027 instead of a plausibly scaled value (27). This indicates a bug in the evaluation script or a scaling error, and it undermines the reliability of every BLEU entry in the paper. Since BLEU is one of the five metrics used to support the abstract's 'superior performance' and Section 5.1's '70% of metrics' claim, the authors must rerun all BLEU evaluations with a standard, bounded implementation, report the exact tokenization and smoothing configuration, and correct Tables 1–3. Without this, the numerical foundation of the central claim is unreliabl
- [§5.2, Table 2] The ablation removing the codebook (w/o cb) shows that the codebook contributes little to the reported performance: most differences are below 0.01, and on some metrics the non-quantized version is better (e.g., RQ BLEU 74 vs 71, NH BLEU 32 vs 28, RW R-L .155 vs .157 aside). The paper's framing in the abstract and Section 5.1 attributes the performance gain to the prototype codebook, but the ablation suggests the gain comes almost entirely from the continuous user embedding and MLP projection. The authors should either identify a configuration or task where quantization gives a clear accuracy benefit, or substantially rewrite the claims to present the codebook as an interpretability/privacy mechanism rather than the source of predictive performance. This is load-bearing because the title and core contribution center on the codebook.
- [§5.1, Table 1 (general)] No error bars, multiple seeds, or significance tests are reported. Many of the claimed improvements are very small and could easily arise from noise. For example, RQ R-2 .027 vs .026, RQ Sim .850 vs .845, NH R-L .179 vs .170, RW R-L .157 vs .152, and RW R-2 .058 vs .053. Without at least multiple runs or pairwise significance tests (e.g., bootstrap or paired tests over the evaluation set), the 'superior performance compared to strong baselines' claim is not statistically grounded. The authors should provide confidence intervals or significance tests for the main results, especially for metrics where the margin over the second-best baseline is under 0.01.
- [§5.3.2] The cross-task generalization analysis reports only codebook usage counts (998, 524, 667, 743), not generation quality on downstream tasks. The assertion that these counts 'demonstrate that the learned user prototypes encode transferable semantic patterns' is a leap: high usage of entries does not imply that the transferred codebook yields good outputs. The main experiments already show strong performance with an ALIGNX-trained codebook, but that codebook is used together with task-specific MLP training. To support the transferability claim, the authors should either report generation metrics for the frozen-codebook scenario or remove the overreach from the text.
minor comments (6)
- [§3.3 vs Appendix B] The main text states the decoder is Qwen-2.5-8B, while Appendix B says Qwen-2.5-7B. Please unify the model size. (Qwen-2.5 actually has 7B and 8B variants; clarify which one is used.)
- [§1 (Contributions)] Typo 'prototyoes' should be 'prototypes'.
- [§1 (Introduction)] Typo 'us er baheviors' should be 'user behaviors'.
- [Table 1] The row labeled 'Len&Para' is not clearly explained in the caption. It seems to list average prompt length for prompt-based methods and trainable parameters for training-based methods, but the caption should say so explicitly and define 'L' and 'U'.
- [Appendix A] The text says 'task-specific training data is utilized exclusively for the ID component' but 'ID' is not defined. Presumably this is the MLP projector; please name it consistently.
- [Figure 3] The figure is hard to read in the PDF: axis labels and legend text are very small, and the 'Random' condition in subplot (b) is difficult to distinguish. Please enlarge and clarify.
Circularity Check
No significant circularity: CURP is a standard supervised train/validate pipeline; the few self-citations are background, not load-bearing.
full rationale
The derivation chain is self-contained in the relevant sense. PCC (Eqs. 1-7) clusters frozen Contriever embeddings of historical behaviors from an ALIGNX training subset into a 1000-entry Product-Quantization codebook; PBA (Eqs. 8-10) trains a two-layer MLP to map quantized prototype embeddings into the frozen Qwen-2.5-8B prompt, maximizing likelihood on held-out responses. The reported RQ/NH/TP/RW numbers are outputs on validation/test splits, not quantities fit to redefine the result. The codebook is trained on Reddit QA and then frozen for other tasks, so the NH/TP/RW results are genuine cross-domain transfer evaluations; the fact that the codebook and the RQ evaluation share the ALIGNX source is a standard train/test separation, not a circular fit. The self-citations in the paper (Mou et al. 2024, Sun et al. 2024, Bao et al. 2024) are background citations about user simulation and role-playing agents; none is used as a load-bearing theorem or to forbid alternatives. Section 5.3.2's 'generalization' diagnostic reports codebook entry utilization counts (998, 524, 667, 743); while utilization is weak evidence of transfer because PQ assignment always selects entries by construction, this diagnostic is not the basis of the main performance claims and does not reduce a prediction to an input. The Table 1 BLEU values above 100 (108-110 on Tweet Paraphrase) are a serious evaluation-validity/correctness concern, not a circularity concern under the requested taxonomy.
Assumptions & free parameters
free parameters (5)
- Codebook size K =
1,000
- Number of PQ subspaces L =
4
- Codebook loss weights lambda1, lambda2, lambda3 =
1.0, 0.15, 1.0
- History count per user J =
8
- MLP hidden dimensions =
768-3584-3584
assumptions (5)
- domain assumption Frozen mean-pooled Contriever embeddings of individual behavior items capture user traits.
- domain assumption An ALIGNX (Reddit) codebook transfers to news, tweet, and review domains.
- domain assumption Prototype decomposition based on Self-Categorization Theory is a valid user model.
- domain assumption Discrete codebook indices are non-invertible and privacy-preserving.
- domain assumption ROUGE/BLEU/RoBERTa similarity measure personalization quality.
Cite this review
Pith. "Pith review of CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs." pith.science (2026). https://pith.science/paper/U4DDMUPT
@misc{pith2026260200742,
author = {Pith},
title = {Pith review of: CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/U4DDMUPT}},
note = {Machine review of arXiv:2602.00742}
}
read the original abstract
User modeling characterizes individuals through their preferences and behavioral patterns to enable personalized simulation and generation with Large Language Models (LLMs) in contemporary approaches. However, existing methods, whether prompt-based or training-based methods, face challenges in balancing personalization quality against computational and data efficiency. We propose a novel framework CURP, which employs a bidirectional user encoder and a discrete prototype codebook to extract multi-dimensional user traits. This design enables plug-and-play personalization with a small number of trainable parameters (about 20M parameters, about 0.2\% of the total model size). Through extensive experiments on variant generation tasks, we show that CURP achieves superior performance and generalization compared to strong baselines, while offering better interpretability and scalability. The code are available at https://github.com/RaidonWong/CURP_code
Figures
Forward citations
Cited by 1 Pith paper
-
Large Behavior Model: A Promptable Digital Twin of the Retail Customer
Grounding an LLM in verbalized transaction histories via Person–Environment prompting, continued pre-training, SFT, and GRPO yields stronger retail decision simulation than frontier models, with partial cross-domain transfer.
Reference graph
Works this paper leans on
-
[1951]
Does Donald Trumpgen- uinely think
I love west- erns and this one is one of my fa- vorites. The acting is good, the story is good and the movie is well paced. I highly recommend this movie...... Description: {De- scription} Review: {Review} B Hyperparameter Settings All experiments are conducted on 8 NVIDIA A800 GPUs with a batch size of 8 per GPU. We utilize DeepSpeed ZeRO Stage 2 to enab...
-
[2003]
Building and applying a concept hierarchy representation of a user profile. InProceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 198–204. Lin Ning, Luyang Liu, Jiaxing Wu, Neo Wu, Devora Berlowitz, Sushant Prakash, Bradley Green, Shawn O’Banion, and Jun Xie. 2025. User-llm: Efficien...
arXiv 2025
-
[2010]
Jihwan Jeong, Yinlam Chow, Guy Tennenholtz, Chi- hWei Hsu, Mohammad Ghavamzadeh, and Craig Boutilier
Product quantization for nearest neighbor search.IEEE transactions on pattern analysis and machine intelligence, 33(1):117–128. Jihwan Jeong, Yinlam Chow, Guy Tennenholtz, Chi- hWei Hsu, Mohammad Ghavamzadeh, and Craig Boutilier. 2024. Factual and tailored recommenda- tion endorsements using language models and rein- forcement learning. InFirst Conference...
arXiv 2024
-
[2015]
InProceedings of the 24th international conference on world wide web, pages 278–288
A multi-view deep learning approach for cross domain user modeling in recommendation systems. InProceedings of the 24th international conference on world wide web, pages 278–288. Ghazal Fazelnia, Eric Simon, Ian Anderson, Benjamin Carterette, and Mounia Lalmas. 2022. Variational user modeling with slow and fast features. InPro- ceedings of the Fifteenth A...
2022
-
[2023]
A survey on user behavior modeling in recom- mender systems.arXiv preprint arXiv:2302.11087. Liam Hebert, Krishna Sayana, Ambarish Jash, Alexan- dros Karatzoglou, Sukhdeep Sodhi, Sumanth Dod- dapaneni, Yanli Cai, and Dima Kuzmin. 2024. Per- soma: Personalized soft prompt adapter architecture for personalized language prompting.arXiv preprint arXiv:2408.00...
arXiv 2024
-
[2025]
Yang Deng, Wenxuan Zhang, Wai Lam, See-Kiong Ng, and Tat-Seng Chua
Onerec: Unifying retrieve and rank with gen- erative recommender and iterative preference align- ment.arXiv preprint arXiv:2502.18965. Yang Deng, Wenxuan Zhang, Wai Lam, See-Kiong Ng, and Tat-Seng Chua. 2023. Plug-and-play policy planner for large language model powered dialogue agents.arXiv preprint arXiv:2311.00262. Sumanth Doddapaneni, Krishna Sayana, ...
arXiv 2023
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.