{"id":"f9b5648a-2618-4070-b490-fbcc4815ac13","arxiv_id":"2505.17051","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"E2P projects pre-computed user embeddings into a single soft prefix token for frozen LLMs, reporting gains on four personalization tasks, though its reproduction scripts write zero embeddings.","lead":"The paper proposes Embedding-to-Prefix (E2P), which converts pre-computed user embeddings into a single extra soft token prepended to the input of a frozen large language model, personalizing outputs without retraining. It reports gains on dialogue, news, music, and podcast tasks, but the paper's own appendix code contradicts the described experimental pipeline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix preprocessing scripts write a zero user embedding for every Persona-Chat/PENS row, so the public-data personalization results in Table 1 cannot be attributed to user-specific context.","rationale":"The internal contradiction between Section 4.1 and Appendix G is the load-bearing issue. The paper's central claim is that injecting a user-specific embedding through a learned projection personalizes a frozen LLM. The provided preprocessing code, the only concrete artifact offered for the public datasets, writes a constant zero vector as the user embedding for every example. This means either the experiments used no user-specific signal, in which case the reported Persona-Chat and PENS improvements are not evidence of personalization, or the code does not correspond to the experiments, in which case the results are not reproducible from the manuscript. Both possibilities undermine the public-data support for the central claim. The production music and podcast results cannot compensate because their embeddings are proprietary, the engagement metric is a fitted proxy, and no data linking the proxy to real user satisfaction is provided. The reader's weakest assumption identifies the same contradiction, and my independent reading of the appendix scripts confirms it. The correct verdict is reject, as the central claim lacks verified supporting evidence on the public datasets and the production evidence is unverifiable.","tokens_in":14342,"tokens_out":2994,"duration_ms":29160,"concrete_test":"Execute the exact preprocessing scripts in Appendix G.1 and G.2 and verify in the emitted JSONL files that every 'uservector' field is 2048 zeros. Then train E2P on Persona-Chat under two conditions: (a) using the shipped zero-vector files, and (b) using embeddings computed as Section 4.1 describes (Llama-3.2-1B last hidden state over persona text). If condition (a) reproduces the reported test perplexity 24.67 and ROUGE-L 16.75, the central personalization claim is unsupported on public data; if only condition (b) reproduces them, the appendix code is not the pipeline that produced Table 1 and the public results are not reproducible.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that a learned projection of a pre-computed user embedding into one soft prefix personalizes a frozen LLM (Eq. 3). Section 4.1 says Persona-Chat and PENS profiles are embedded with Llama-3.2-1B to 2048-dimensional vectors. But the reproduction scripts in Appendix G.1 and G.2, presented as 'Code to reproduce our data splits', write 'uservector': [0.]*2048 for every row. If those files were used, E2P on both public tasks degenerates to a single static learned prefix, and the Table 1 gains (24.67 vs 45.40 perplexity; ROUGE-L 16.75 vs 14.66) are prefix-conditioning effects, not personalization. If the scripts are placeholders, the reported numbers are not reproducible from the shipped pipeline. Either way the Persona-Chat and PENS evidence does not support Eq. (3)'s claim of injecting user-specific context. The random-control comparison (E2P-Random 26.56 perplexity vs E2P 24.67) is also uninterpretable if the trained prefix is constant. Production music/podcast results rely on proprietary embeddings and an engagement proxy whose correlation with real engagement is asserted in Appendix F.1 without data, so they cannot independently rescue the claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Embedding-to-Prefix (E2P), a parameter-efficient personalization method that conditions a frozen LLM on a precomputed user embedding c by projecting c through a two-layer MLP with LayerNorm (Eq. 2) into a single soft prefix token prepended to the input embedding sequence (Eq. 3). The projection is trained with a language-modeling objective (Eq. 4) for text tasks and with Kahneman-Tversky Optimization (Eq. 5) for engagement tasks, while all base-model weights stay frozen. E2P is evaluated on Persona-Chat (perplexity), PENS (ROUGE), and two proprietary Spotify tasks: music playlist generation (predicted-engagement proxy) and podcast next-item recommendation (hit-rate@30). Table 1 reports gains over a no-context baseline on all four tasks (24.67 vs. 45.40 perplexity; ROUGE-L 16.75 vs. 14.66; +12.9% predicted engagement; +2.2% hit-rate@30), with an E2P-Random control intended to separate prefix-structure effects from genuine personalization. The appendices describe the splits, hyperparameters, significance tests, and preprocessing scripts, which Appendix F presents as the code that reproduces the data splits.","tokens_in":14418,"tokens_out":18808,"duration_ms":169247,"significance":"The central idea is appealing, and if the reported results held, E2P would be a practical contribution: injecting existing dense user embeddings through a single learned soft prefix keeps the backbone frozen, adds negligible inference overhead, and can generalize to new users through embedding similarity. The random-embedding control is a sound design choice, the paper addresses a real gap (standard PEFT methods do not consume external user vectors), and the authors ship their preprocessing scripts, which makes the claims externally checkable. The check, however, fails: the shipped scripts contradict the described method on both public datasets, and the two proprietary evaluations rest on a proxy whose external validity is asserted without data. Because the evidence chain is broken at its most verifiable point, the significance of the contribution cannot be assessed from this version of the manuscript.","major_comments":[{"comment":"The reproduction scripts shipped in the appendix directly contradict the method description. Section 4.1 states that Persona-Chat personas and PENS user profiles are embedded with Llama-3.2-1B into a single 2048-dimensional vector per persona or profile, and Appendix F presents the code in G.1 and G.2 as 'Code to reproduce our data splits.' In fact, G.1 and G.2 write 'uservector': [0.]*2048 for every row of both public datasets. If those scripts were used, E2P on Persona-Chat and PENS receives a constant input, so the Table 1 gains (24.67 vs. 45.40 perplexity and ROUGE-L 16.75 vs. 14.66) are effects of a single static learned prefix rather than of user-specific context; moreover, E2P and E2P-Random would then be the same model, making the 24.67 vs. 26.56 perplexity gap interpretable only as training variance. If the scripts were not used, the public-data results are not reproducible from the manuscript. Either way, the central claim that precomputed context embeddings are injected through a learned projection into a single soft token (Eq. 3) is unsupported by the two public datasets.","section":"Appendix G.1/G.2; Section 4.1; Eq. (3); Table 1"},{"comment":"Appendix F.1 says the Persona-Chat data are 'shuffled with seed 42 and partitioned 80/10/10 by persona id, ensuring no persona appears in more than one split,' but the code in G.1 shuffles individual rows with random.shuffle(rows) and splits by row index. The footnote counts (train 49,778, dev 6,222, test 6,223) are exactly 80/10/10 of the 62,223 rows, which is consistent with a row-level split and not with grouping by persona id. Under the shipped code, the same persona can appear in both training and test splits, so the claimed leakage control is not implemented; since the persona description is what the E2P embedding encodes, persona leakage could inflate the reported personalization gap.","section":"Appendix F.1 vs. Appendix G.1"},{"comment":"The shipped Persona-Chat file cannot be used to train the objective of Eq. (4). Equation (4) requires a target sequence y given an input x and user vector c, but G.1 writes each row with only a 'uservector' and a single 'text' field that concatenates the persona, h[0], and h[1]; no target response is stored, and the persona description is placed inside the textual prompt rather than being carried by the user vector. If training was instead performed on this 'text' field as a language-modeling target, the model would be trained to reproduce the persona text, which is a different task from the one described in Section 4.1 and would make the comparison with the Prompt-Persona baseline difficult to interpret.","section":"Section 3.4.1; Eq. (4); Appendix G.1"},{"comment":"The experimental settings in the main text and the appendix are mutually inconsistent. Section 4.2 states that 'LLaMA-3.2-3B [is] the base model for all experiments' with a learning rate of 5e-6, batch size 32, and 5 training epochs, while Appendix D lists Llama-3.1-8B for Music Rec and Llama-3.2-1B for Podcast Rec, with batch sizes 64 (Persona-Chat), 256 (Music Rec), and 16 (Podcast Rec), and 3 epochs for Persona-Chat and PENS. Since the Table 1 gains and significance markers depend on these settings, the paper as written does not provide a consistent experimental record for any of the four tasks, and the claimed parameter-efficiency comparison cannot be reconstructed.","section":"Section 4.2 vs. Appendix D"},{"comment":"Equation (5) is not the Kahneman-Tversky Optimization objective of the cited reference [8]. The loss in Eq. (5) is a weighted binary cross-entropy with weight alpha on the negative class; the KTO objective of Ethayarajh et al. is defined through an implicit reward r_KTO(x,y) = beta log(p_theta(y|x)/p_ref(y|x)) - KL(pi_theta || pi_ref), with a sigmoid applied to the difference between that reward and a reference threshold, and it contains no explicit alpha-weighted negative-class term. Because Eq. (5) is the stated training objective for both Music Rec and Podcast Rec, the paper should either correct the attribution and name the actual loss used (asymmetric weighted binary cross-entropy) or remove the claim that E2P uses KTO.","section":"Section 3.4.2; Eq. (5)"},{"comment":"The headline Music Rec result (+12.9% predicted engagement, Table 1) is measured by a separate binary engagement classifier (Appendix F.1), and E2P is trained with the objective of Eq. (5) against the same engagement-label distribution. The paper asserts in F.1 that improvements in this offline proxy 'have correlated with online gains in user satisfaction in related production experiments' but provides no data supporting that correlation. As written, the Music Rec numbers demonstrate only that E2P optimizes the proxy; the claim that E2P improves real user engagement requires evidence of external validity (for example, the correlation statistic or a small online A/B result), without which the production-scale personalization claim remains unverified.","section":"Eq. (5); Appendix F.1; Table 1"}],"minor_comments":[{"comment":"The header of the last block contains the typo 'Podcacst Rec'; it should read 'Podcast Rec.'","section":"Table 1"},{"comment":"The sentence about single-token interventions being effective, citing [25], appears twice with nearly identical wording (Section 3.1: 'This aligns with prior findings that even single-token interventions can be effective'; Section 3.3: 'This design aligns with prior findings that single-token interventions can be surprisingly effective'). One of the two should be removed.","section":"Sections 3.1 and 3.3"},{"comment":"The E2P-Random control is defined as 'a prefix generated from a randomly selected user embedding from the test set,' but it is not stated whether the control is trained with random embeddings at training time or only evaluated with them at test time; the two readings support different interpretations of the Table 1 gaps, so the definition should be made explicit.","section":"Section 4.2"},{"comment":"The significance-marker legend is confusing: it states that symbols '(‡, ⋄)' are significant 'compared to E2P,' but in the Podcast Rec block the E2P + Prompt row carries both symbols, which suggests the second symbol compares against a different baseline (presumably Prompt-Context); the legend should assign exactly one referent to each symbol.","section":"Table 1 caption"},{"comment":"The paragraph beginning 'Although methods such as Parameter-Efficient Fine-Tuning (PEFT) have emerged...' is a sentence fragment ending in '[27, 25, 17]' followed by a new sentence; it should be reworded grammatically.","section":"Section 1"},{"comment":"The 'Embedding Retrieval/Reranking' baseline is described only by reference to a GNN model [6], with no retrieval candidate set or implementation details, so the +0.1% and +1.2% baseline numbers cannot be reproduced; in addition, the reference list is inconsistent in capitalization and venue formatting throughout and should be normalized to the journal's style.","section":"Section 4.2; References"}],"recommendation":"reject","confidential_remarks":"The decisive issue is internal: the paper's own Appendix G scripts, offered as the reproduction code for the public data, nullify the user-specific signal (zero vectors) on both public datasets and implement a different split from the one described in F.1. This is not a disagreement about interpretation or about the quality bar for proxy metrics; the manuscript as submitted fails to support its central empirical claim at the most checkable point. The two proprietary tasks are, by design, not externally checkable, and their evaluation rests on an asserted-but-unsupported proxy correlation. I see no indication of deliberate misrepresentation; the discrepancies are consistent with a documentation and packaging failure, which is nevertheless disqualifying at this stage. I would welcome a corrected submission that re-runs the public experiments with the true embeddings, ships working preprocessing and training code, and reports any available validation of the engagement proxy, but the current version should not proceed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know up front. The core idea—take an existing user embedding, run it through a two-layer MLP, prepend the result as one soft token to a frozen LLM—is clean and practical. The second thing is that the paper's own reproduction scripts write a 2048-dimensional zero vector as the user embedding for every row of Persona-Chat and PENS. Those two facts are in direct tension.\n\nWhat the paper does well: the architecture is minimal, the parameter count is tiny, and the production setting (music and podcast recommendation) is a real deployment scenario. The E2P-Random control is the right instinct. The writing is clear, and the method is easy to reimplement. The paper also honestly cites related work on embedding-to-text bridges (ELM, USER-LLM, soft prompt adapters), which makes the claim of novelty more modest than the title implies.\n\nThe soft spots are not small. Appendix G.1 and G.2 are labeled 'Code to reproduce our data splits' and they write 'uservector': [0.]*2048 for every example. If those files were used in the experiments, then E2P on Persona-Chat and PENS is a static prefix with no user-specific signal, and the Table 1 gains are prefix-conditioning effects, not personalization. If the files are placeholders, the reported numbers are not reproducible from the shipped pipeline. Either way, the public-data evidence does not support the paper's central claim. On top of that, Appendix F.1 says the Persona-Chat split is by persona id, while G.1 random-shuffles the whole dataset. That creates persona leakage. Section 4.2 says all runs use Llama-3.2-3B with batch size 32 for 5 epochs, but Appendix D lists three different base models and batch sizes from 16 to 256. Those mismatches do not individually sink a paper, but they reinforce that the appendix cannot be trusted as a record of what was run.\n\nThe production results are also hard to evaluate: the datasets are proprietary and the music metric is a fitted engagement proxy. The paper asserts that the proxy correlates with online satisfaction but shows no data. That is acceptable for an internal report, but it cannot independently rescue the public-data failure.\n\nThe method itself is not new. It reduces to prefix tuning with a learned projection from an external vector, and the paper's own related work already includes ELM, USER-LLM, and soft prompt adapters doing essentially this. The contribution is the single-token efficiency and the production evaluation, not the architecture.\n\nWho is this for? A practitioner who wants a simple way to condition a frozen LLM on a user embedding could use this as a template. As a scientific paper, though, the central evidence is missing because the shipped code nullifies the personalization mechanism. The zeros in the appendix are not a minor artifact; they break both public-dataset experiments.\n\nMy recommendation: desk reject in current form. If the authors ship corrected code and the numbers reproduce, this could become a serviceable workshop paper, but as it stands the evidence does not support the claims.","headline":"The core idea is clean and practical, but the paper's own reproduction scripts write zero vectors as user embeddings for Persona-Chat and PENS, so the public-data evidence for personalization does not hold together.","tokens_in":15200,"tokens_out":3472,"would_cite":false,"duration_ms":31197,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"E2P claims that injecting a pre-computed user embedding into a frozen large language model through a single learned soft prefix token personalizes generation as well as token-heavy prompts, at a fraction of the cost.","keywords":["embedding-to-prefix","parameter-efficient personalization","soft prefix tuning","frozen language models","user embeddings","dialogue personalization","generative recommendation"],"falsifier":"Run the paper's published preprocessing (Appendix G) on Persona-Chat, train E2P with the resulting all-zero user vectors, and measure test perplexity; if it matches the reported 24.67, the user embedding carried no signal and the personalization gain is an artifact of the prefix structure. Alternatively, train E2P with real persona embeddings and with zero embeddings under identical settings and compare to E2P-Random.","tokens_in":13948,"feed_emoji":"🧠","tokens_out":6159,"duration_ms":56823,"temperature":0.7,"pith_summary":"E2P claims that a frozen large language model can be personalized by converting a pre-computed user embedding into a single soft prefix token through a learned two-layer projection, then prepending that token to the input. The paper reports that this one-token intervention beats the no-context baseline on all four evaluated tasks—dialogue (Persona-Chat), headline generation (PENS), and music and podcast recommendation—while training only the projection and leaving the base model untouched. If correct, the method turns existing user-embedding infrastructure into a cheap, low-latency personalization channel for generative models, with no per-user fine-tuning. The claim is that the user signal itself, not the added prefix structure, drives the gains.","feed_headline":"One soft token personalizes a frozen LLM","feed_subtitle":"The method projects user embeddings into one prefix token, keeping the base model frozen while improving chat, news, and recommendation…","key_machinery":"The load-bearing object is the single soft prefix token $p = \\phi(c)$, a continuous vector in $\\mathbb{R}^{d_h}$ produced by a two-layer MLP with ReLU and LayerNorm, prepended to the input embedding sequence before the frozen transformer processes it. Because the prefix is continuous and generated from the user embedding, it can generalize to unseen users and preserve similarity structure from the user-embedding space. The paper's training objectives are the standard next-token language-modeling loss for generation and a Kahneman–Tversky objective for recommendation engagement, both optimizing only the projection parameters.","core_discovery":"The central claim is that there exists a mapping from user embeddings to the LLM's hidden space that works with a single soft token. Specifically, E2P defines $\\phi(c) = \\mathrm{LayerNorm}(\\mathrm{ReLU}(W_1 c)) W_2 + b$ and inserts $\\phi(c)$ as the first token of the embedding sequence, so the modified input is $[\\phi(c); E(x)]$. With the backbone frozen, only $\\phi$ is trained, via a language-modeling loss for generation tasks and Kahneman–Tversky optimization for engagement prediction. Across the four tasks, E2P reports perplexity 24.67 versus 45.40 for no context on Persona-Chat, ROUGE-L 16.75 versus 14.66 on PENS, +12.9% predicted engagement on music, and +2.2% hit-rate@30 on podcasts, with statistical significance against the no-context baseline.","pith_inferences":["The published preprocessing scripts write a constant zero vector as the user embedding for every row on Persona-Chat and PENS; if those scripts were what actually trained the model, the reported gains over E2P-Random would not be explainable by user-specific information, and the two conditions should collapse. This means the paper's headline public-dataset results may rest on a code-vs-description","A direct way to test the method's core premise is to train E2P on Persona-Chat with real persona embeddings and compare against the zero-vector condition; if perplexity does not differ, the single-token prefix is not carrying user identity.","The engagement-classifier proxy on music recommendation is the only evidence for user satisfaction; an online A/B test with actual save/listen behaviour would settle whether the +12.9% translates to real engagement.","The single-token design suggests a broader principle: that a frozen LLM's conditioning can be a learned continuous function of arbitrary auxiliary embeddings, which could extend to other dense contextual signals (time, location, session state) beyond user profiles."],"forward_implications":["Personalization becomes parameter-efficient: only the projection module is trained, so adapting to new users costs a few million parameters rather than a fine-tuned copy of the base model.","The single-token prefix adds negligible decoding latency, making the approach deployable in real-time recommendation and dialogue systems.","Because user embeddings can be updated continuously, E2P can reflect evolving preferences without retraining the projection unless the embedding distribution shifts.","Dense user embeddings can complement textual prompts: the paper reports that E2P plus a textual prompt beats either alone on podcast recommendation.","The method generalizes across generative and retrieval-style tasks, from open-ended dialogue to semantic-ID-based item ranking."],"supporting_citations":[{"why":"Supplies the soft-prefix mechanism that E2P simplifies to a single token.","marker":"[27]"},{"why":"Provides the Persona-Chat dataset and its next-utterance evaluation setup.","marker":"[53]"},{"why":"Provides the PENS personalized news headline corpus and task formulation.","marker":"[1]"},{"why":"Offers evidence that single-token interventions can be effective, justifying the simplified prefix.","marker":"[25]"},{"why":"Defines the LLaMA-3.2 backbone models used as the frozen base in all experiments.","marker":"[12]"},{"why":"Describes the generalized user representations that E2P injects in the recommendation tasks.","marker":"[10]"},{"why":"Contributes the Kahneman–Tversky optimization objective used for engagement prediction.","marker":"[8]"},{"why":"Establishes the semantic-ID scheme that the podcast task uses as target sequences.","marker":"[42]"}],"fun_headline_variants":["One token personalizes frozen LLMs","E2P: Frozen LLM personalization via one prefix token","Project user embeddings to one token for personalized LLM","Single soft token unlocks LLM personalization without fine-tuning","Parameter-efficient personalization: one prefix token for frozen LLM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported experiments actually conditioned on distinct, meaningful user embeddings — the published preprocessing code for Persona-Chat and PENS writes a constant zero vector for every user, and the proprietary recommendation results assume the behavioral embeddings and the engagement-classifier proxy faithfully represent user preference.","fun_headline_variants_meta":{"raw":{"variants":["One token personalizes frozen LLMs","E2P: Frozen LLM personalization via one prefix token","Project user embeddings to one token for personalized LLM","Single soft token unlocks LLM personalization without fine-tuning","Parameter-efficient personalization: one prefix token for frozen LLM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1365,"prompt_tokens":925,"completion_tokens":440,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":361}},"tokens_in":541,"tokens_out":440,"duration_ms":4202,"temperature":1.0,"reasoning_tokens":361,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:55:56.229905+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's published preprocessing (Appendix G) on Persona-Chat, train E2P with the resulting all-zero user vectors, and measure test perplexity; if it matches the reported 24.67, the user embedding carried no signal and the personalization gain is an artifact of the prefix structure. Alternatively, train E2P with real persona embeddings and with zero embeddings under identical settings and compare to E2P-Random.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Persona-Chat dataset and its next-utterance evaluation setup."},{"cited_title":"Pens: A dataset and generic framework for personalized news headline generation","cited_arxiv_id":null,"evidence_quote":"Provides the PENS personalized news headline corpus and task formulation."},{"cited_title":"The power of scale for parameter-efficient prompt tuning","cited_arxiv_id":null,"evidence_quote":"Offers evidence that single-token interventions can be effective, justifying the simplified prefix."},{"cited_title":"Generalized User Representations for Transfer Learning","cited_arxiv_id":"2403.00584","evidence_quote":"Describes the generalized user representations that E2P injects in the recommendation tasks."}],"review_version":1}