{"id":"94e73fba-372a-4e9d-8942-6e1515e7d813","arxiv_id":"2412.11736","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A dual-tower LLM with a low-rank querier-specific encoder and cluster-restricted contrastive learning generates responses tailored to the person asking, evaluated on a new 173-querier multi-source dialogue dataset.","lead":"This paper builds a chatbot that answers the same question differently depending on who asks, by learning each questioner's personality from past conversations. It also introduces a new dataset and training method for user-aware response generation, a practical step toward assistants that adapt to each person.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The described architecture has no querier-identity input, so Eq. 1's same-context claim cannot hold as stated; the paper must clarify how the model conditions on the querier at inference.","rationale":"The reader's weakest assumption is that similar queries were substituted for identical queries in evaluation, so the quantitative results do not directly support the same-query claim. I agree that this is a real gap, but the more load-bearing issue is at the level of the model: the described dual-tower architecture has no mechanism to inject the querier identity at inference time. The specific encoder is shared and low-rank, and e_i is never used in generation. Therefore the formal condition in Eq. 1 cannot be met for token-identical contexts, and the case studies are the only evidence that purports to show same-query differentiation. That evidence is under-specified: if the inputs differ in context or speaker labels, the claim is not demonstrated; if the inputs are identical, the observed output differences are unexplained by the architecture. This is not an attack on the empirical usefulness of the method for contexts that differ across queriers, nor on the dataset or the ablations. It is a precise internal-consistency problem between the problem statement and the design. The paper can likely be repaired by (a) explicitly adding a querier-identity token or per-querier embedding to the generation path, or (b) redefining x to allow identity-bearing context and scoping the central claim to 'same query within different histories.' Because either repair is feasible and the empirical evidence may still support the practical claim, conditional acceptance remains appropriate. The reader's verdict already asks for the same-query issue to be addressed; my concern adds a specific architectural check that should be part of that condition. Hence I recommend the verdict be left unchanged at CONDITIONAL.","tokens_in":26582,"tokens_out":9857,"duration_ms":97057,"concrete_test":"Inspect the released inference code for any querier-specific tensor (user ID embedding, per-querier global representation e_i, or speaker-name tokens) and then run greedy decoding on two token-identical inputs—the same query string with no speaker labels and no profile—for two different queriers. If the outputs are identical and no querier-specific tensor reaches the decoder, the architecture cannot realize Eq. 1 for identical contexts. If speaker labels are present in the input, delete them and confirm that the output conditioning disappears, which would show the claim depends on hidden identity information not described in the paper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, formalized in Eq. 1, requires that for two different queriers Qi and Qj, the same query context history x_i^k = x_j^l yields different response distributions. The architecture described in Sections 4.2-4.3 does not provide any querier-identity signal to the model: the general encoder and the low-rank specific encoder are both shared across all queriers, and both take only the dialogue text x as input. The per-querier global representation e_i is used only inside the contrastive loss (Eq. 5) and the multi-view loss (Eq. 8); it is not injected into the fused representation G_g(x) + G_s(x) that feeds the language-model head. Consequently, for any two token-identical inputs, the encoders produce identical representations and identical output distributions, so Eq. 1 is unsatisfiable by the described architecture. The same-query case studies in Figure 8 and Appendix C therefore imply either stochastic sampling or unstated input differences (e.g., speaker-name tokens or different surrounding context); neither is specified in the paper. If speaker labels are part of x, then the formal equality of x across queriers is never actually tested, and the quantitative evaluation on ordinary held-out dialogues cannot distinguish querier conditioning from generic context sensitivity. This concern is distinct from, and more fundamental than, the similar-query-versus-identical-query evaluation gap: even a perfectly paired identical-query test would fail under this architecture unless some querier-specific input is added.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies querier-aware LLM personalization, where the model should produce different responses to the same query depending on who asks it. The authors propose a dual-tower architecture consisting of a cross-querier general encoder and a shared low-rank specific encoder, trained with a querier-contrastive loss, multi-view augmentation, and query-similarity clustering. They construct MQDialog, a multi-querier dialogue dataset from English and Chinese scripts and WeChat records, and report BLEU/ROUGE gains over zero-shot, fine-tuned, profile-based, few-shot, and querier-characteristic baselines, plus GPT-4 and human win rates. The paper also provides ablations, t-SNE visualizations, and same-query case studies.","tokens_in":26722,"tokens_out":8864,"duration_ms":83927,"significance":"The task is well-motivated, and the empirical package is extensive: four baseline families, ablations, a new dataset, GPT-4 win rates, a small human evaluation, and public code. If the central claim were established, the proposed parameter-efficient design would be a useful contribution to personalized dialogue. However, the current manuscript does not establish the central claim: the architecture has no querier-identity input at inference, and the quantitative evaluation does not test the identical-query condition of Eq. (1). The case studies are suggestive but not a substitute for a controlled paired-query experiment. These issues are substantive enough that the revision should be major.","major_comments":[{"comment":"The architecture cannot satisfy Eq. (1) as stated. Both the general encoder G_g and the specific encoder G_s take only the dialogue text x as input and are shared across all queriers; the per-querier global representation e_i is used only in the contrastive losses (Eqs. (5) and (8)) and is never injected into the fused representation G_g(x)+G_s(x) that feeds the language-modeling head. Consequently, for any token-identical inputs x_i^k = x_j^l, the encoders produce identical representations and identical output distributions, so P(y_i^k | x_i^k; Q_i; Θ) = P(y_j^l | x_j^l; Q_j; Θ), contradicting Eq. (1). Stochastic sampling from the same distribution would generate different strings but not different distributions. To make the claim meaningful, the model must take an explicit querier-identity signal (e.g., e_i or a learned querier embedding) as input at inference, and the paper must specify how that signal is provided.","section":"Section 4.2-4.3, Eq. (1)"},{"comment":"The quantitative evaluation does not test the same-query condition. Section 4.1 explicitly relaxes x_i^k = x_j^l to similar queries x_i^k ≈ x_j^l, and Section 6.2 measures BLEU and ROUGE on ordinary held-out dialogues, where each querier's context is generally different. The only same-query demonstrations are the case studies in Figure 8 and Appendix C, which are anecdotal and do not report whether the inputs were strictly identical, whether speaker names are part of x, or the decoding parameters used. A paired evaluation is needed: for the same surface-form query and the same context history, with only the querier identity varied, the responses or their distributions should be compared, and the input format should be stated precisely.","section":"Section 4.1, Section 6.2, Figure 8"},{"comment":"The proof of Theorem 1 does not satisfy the conditions of Lemma 1. The lemma requires X to be a set of n features of random samples, with exactly one sample drawn from the conditional distribution p(x_i|y) and the rest from the marginal p(x); here E consists of one running-average global representation per querier, which is not a fresh random sample from any conditional distribution given z_i^k, and the score function is a deterministic function of the model's own parameters. The mutual-information bound is imported from prior work (Ref. [48]) and cannot be invoked without verifying these sampling assumptions. If the bound is intended only as intuition, that should be stated explicitly; if it is a formal claim, the proof needs to be repaired.","section":"Section 4.3, Lemma 1 and Theorem 1"}],"minor_comments":[{"comment":"Equation (4) and the surrounding text: the quantity f_QC is an increasing function of cosine similarity, so the phrase 'proportional to the cosine distance' is misleading; please say 'cosine similarity' or adjust the expression accordingly.","section":"Section 4.3, Eq. (4)"},{"comment":"The text says 'dual-town structure'; this appears to be a typo for 'dual-tower structure'.","section":"Appendix D.1"},{"comment":"The '-' entries for the Real Person row in the RPG and QCG columns are not explained; please state explicitly why those baselines were not run for that responder.","section":"Table 3"},{"comment":"The human evaluation reports win rates but does not report inter-annotator agreement; please include a statistic such as Cohen's kappa or a comparable measure.","section":"Section 6.3"},{"comment":"The decoding method (greedy, top-p, temperature, etc.) used to generate responses is never specified; this matters for interpreting the same-query case studies and win-rate comparisons.","section":"Sections 6.2 and 6.3"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a camera-ready CIKM '25 publication; this report treats it as a submitted manuscript. The architectural gap is a correctness issue rather than a matter of presentation, and I would want to see a revised version that adds explicit querier conditioning and a paired same-query evaluation before endorsing it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is MQDialog, a multi-querier dialogue dataset with 173 queriers and 12 responders sourced from English and Chinese scripts plus real WeChat records, and a dual-tower training scheme that learns a low-rank querier-specific encoder via cluster-restricted contrastive loss. The experiments are extensive and internally consistent: four baseline families, ablations for the loss and clustering, GPT-4 win rates, a small human evaluation, and t-SNE visualizations all point in the same direction. The dataset alone is a useful contribution, and the training recipe is parameter-efficient and easy to reproduce from the public code. The soft spot is load-bearing. The problem statement in Eq. 1 requires that for the same query context history x_i^k = x_j^l, the output distributions differ across queriers. But the architecture gives the model no query identity at all: both the general and specific encoders are shared, take only the dialogue text as input, and the per-querier global representation e_i is used only inside the contrastive loss, never injected into the fused representation that feeds the language-model head. So for token-identical input, the model deterministically produces the same distribution. The case studies in Figure 8 and Appendix C show different responses to the same query, but those examples almost certainly use different full contexts (different prior turns, possibly speaker names), not the identical x required by the formalism. The result is that the paper demonstrates context-sensitive response generation, which is real but not the same as query-aware personalization. The quantitative evaluation compounds this by testing on ordinary held-out dialogues with similar queries rather than matched identical queries, so BLEU/ROUGE and even the GPT-4 win rates do not directly test the central claim. Two smaller issues are worth noting. The mutual-information Lemma 1 is imported from Oord et al. and assumes one positive sample from the conditional distribution with the rest from the marginal; the running-average global representation e_i does not satisfy that setup, so the theorem is informal. And the strongest natural baseline, per-user finetuning or per-user LoRA, is mentioned in Section 3.2 but never evaluated, which weakens the claim that one shared model is better than user-specific models. The WeChat portion is private, which limits reproducibility, though the extraction toolkit helps. On balance: the method is likely useful for generating responses tailored to a user's dialogue history, and the dataset is a real asset. But the title, abstract, and Eq. 1 overstate what the architecture can do. A serious revision should either add a query-identity input (e.g., inject e_i or a learned query embedding into the fused representation) or re-scope the claim to dialogue-history-conditioned response generation. This deserves a careful referee, but it should not be accepted in its current form. I would bring it to a reading group to discuss exactly this gap between problem statement and implementation.","headline":"Good dataset and a sensible contrastive training recipe, but the architecture has no querier-identity input, so the same-query headline in Eq. 1 does not actually hold.","tokens_in":772,"tokens_out":983,"would_cite":false,"duration_ms":41024,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single shared LLM can learn to answer the same query differently for different users.","keywords":["Personalization","Chat LLMs","Querier-Aware","Contrastive learning","Dual-tower architecture","Low-rank adaptation","MQDialog dataset","Response generation"],"falsifier":"Collect a held-out set in which many different queriers ask literally the same question (e.g., 'What is gene sequencing?') to the same responder, then measure whether the model's responses vary systematically with querier identity, for example by computing the cosine distance between generated responses from different queriers versus responses from the same querier asked twice. If the between-querier distance is not significantly larger than the within-querier distance, the central claim is falsified.","tokens_in":26204,"feed_emoji":"💬","tokens_out":5285,"duration_ms":45974,"temperature":0.7,"pith_summary":"The paper tries to establish a new form of LLM personalization: instead of only varying the responder's role, the model should adapt to who is asking, so that identical queries receive different replies tailored to each querier's personality and relationship to the responder. To test this, the authors build a multi-querier dialogue dataset (MQDialog) from TV scripts and real WeChat records, and design a dual-tower architecture that separates a shared general encoder from a low-rank querier-specific encoder. A querier-contrastive loss, restricted to clusters of similar queries, pulls same-querier dialogues together and pushes different queriers apart. Reported results show relative gains of 8.4% to 48.7% in ROUGE-L and a 65.8% average GPT-4-judged winning rate over baselines, with case studies showing the same query answered differently for different queriers.","feed_headline":"Same query, different reply: one LLM adapts to who asks","feed_subtitle":"Dual-tower contrastive model lifts ROUGE-L by up to 48.7% over baselines.","key_machinery":"The load-bearing mechanism is the dual-tower architecture: a general encoder initialized from a pretrained LLM captures the responder's cross-querier personality, while a separate specific encoder, whose feedforward layers are decomposed into two low-rank matrices, captures querier-specific personality; the two towers are fused by element-wise addition before the language-model head. Training is driven by a querier-contrastive loss, which maximizes a lower bound on mutual information between a dialogue representation and the querier's global representation, with multi-view augmentation (two projection views) and a query-similarity clustering step that confines contrastive pairs to dialogues with similar queries.","core_discovery":"The central claim is that a unified, one-for-all model can internalize querier identity: even when two users ask exactly the same question, the model should produce responses whose distribution differs per querier, reflecting the querier's personality and relationship with the responder (formalized as P(y|x; Q_i; Θ) ≠ P(y|x; Q_j; Θ)). The paper argues this is achievable by decomposing dialogue personality into a cross-querier general component (full transformer) and a sparse, low-rank querier-specific component, trained jointly with language modeling and a querier-contrastive loss. Because identical queries across queriers are rare in real data, the method clusters dialogues by query-embedding similarity and performs contrastive learning within clusters. The paper further contributes MQDialog, a 173-querier, 12-responder benchmark built from English and Chinese scripts plus real WeChat records, and reports consistent improvements in BLEU/ROUGE and LLM-judged winning rates against zero-shot, fine-tuning, profile-based, and few-shot baselines.","pith_inferences":["If the central claim holds, then building paired same-query benchmarks (identical queries posed by many users) would likely show larger measured gains than the similar-query evaluation in the paper, because the relaxed clustering setup understates the contrastive signal.","The approach suggests a practical cold-start compromise: for a new querier with no history, the shared towers still produce generic responses, and personalization improves as dialogues accumulate; profile-clustering could scale the method to million-scale user bases, as the paper notes.","The querier-contrastive objective can be seen as a form of user-embedding learning; it might transfer to other personalized generation tasks such as recommendation explanations or customer support, where the responder is fixed and the querier varies.","Because the dataset is built from scripted dialogues and one author's WeChat records, results may depend on how cleanly querier identity is expressed in scripted versus real conversations; a test on naturally occurring multi-querier customer-service logs would be a strong external check."],"forward_implications":["A single shared model can serve many users without per-user fine-tuning, since both encoders are shared and the added parameters are about 1% of the pretrained LLM.","Including the querier side, not just the responder role, improves response quality over role-profile, few-shot, and fine-tuning baselines across English and Chinese.","Query-similarity clustering is necessary: contrastive learning without it hurts performance, and removing the contrastive loss also degrades BLEU/ROUGE, showing both components matter.","The model's representations of dialogues become separated by querier in t-SNE plots, whereas fine-tuning mixes them.","On same-query case studies, the model responds differently to different queriers while fine-tuning produces near-identical replies."],"supporting_citations":[{"why":"Supplies Lemma 1, the mutual-information bound that justifies minimizing the querier-contrastive loss.","marker":"[48]"},{"why":"LoRA motivates the parameter-efficient low-rank specific encoder and is used in the general block.","marker":"[18]"},{"why":"Provides the manual multi-view augmentation technique used to generate two projection views for the contrastive loss.","marker":"[8]"},{"why":"Serves as the zero-shot generation baseline.","marker":"[5]"},{"why":"Provides the few-shot prompting baseline method.","marker":"[10]"},{"why":"One of the responder profile-based generation (RPG) baselines.","marker":"[51]"},{"why":"Pretrained model used for the English datasets.","marker":"[1]"},{"why":"Pretrained model used for the Chinese datasets.","marker":"[4]"},{"why":"ROUGE evaluation metric used for response quality.","marker":"[27]"},{"why":"BLEU evaluation metric used for response quality.","marker":"[34]"}],"fun_headline_variants":["Same prompt, different reply: LLM adapts to who asks","Querier-aware LLM: one model, customized answers per user","Model learns querier identity to personalize each response","Dual-tower LLM lifts ROUGE-L by up to 48.7% over baselines","New MQDialog dataset enables querier-specific response generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central same-query claim is only directly tested in a few case studies; the quantitative evaluation relaxes 'same query' to 'similar queries' clustered by embedding similarity, so if similar-query similarity is not a good stand-in for identical queries, the reported numbers do not directly support the paper's headline claim.","fun_headline_variants_meta":{"raw":{"variants":["Same prompt, different reply: LLM adapts to who asks","Querier-aware LLM: one model, customized answers per user","Model learns querier identity to personalize each response","Dual-tower LLM lifts ROUGE-L by up to 48.7% over baselines","New MQDialog dataset enables querier-specific response generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000349,"raw_usage":{"total_tokens":1931,"prompt_tokens":994,"completion_tokens":937,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":842}},"tokens_in":610,"tokens_out":937,"duration_ms":8346,"temperature":1.0,"reasoning_tokens":842,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:37:58.442742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a held-out set in which many different queriers ask literally the same question (e.g., 'What is gene sequencing?') to the same responder, then measure whether the model's responses vary systematically with querier identity, for example by computing the cosine distance between generated responses from different queriers versus responses from the same querier asked twice. If the between-querier distance is not significantly larger than the within-querier distance, the central claim is falsified.","supporting_citations":[],"review_version":1}