Pith. sign in

REVIEW 3 major objections 5 minor 9 cited by

Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read State-of-the-art LLMs fall below 10% accuracy at following a user's stated preferences after only 10 conversation turns when left unprompted.

desk verdict PrefEval is a genuinely useful benchmark and the qualitative story holds, but the abstract's 'below 10% at 10 turns across most models' contradicts the paper's own explicit-preference tables. read the letter →

arxiv 2502.09597 v1 pith:56JXPVMM submitted 2025-02-13 cs.LG cs.CL

Siyan Zhao , Mingyi Hong , Yang Liu , Devamanyu Hazarika , Kaixiang Lin This is my paper · ORCID
classification cs.LGcs.CL
keywords preferencefollowingLLMpersonalizationlong-contextconversationbenchmarkimplicitinferenceLLM-as-a-judgeretrieval-augmentedgenerationsupervisedfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PrefEval asks whether a chatbot can infer, remember, and apply a user's stated preferences across a long, multi-topic conversation, and answers that current LLMs mostly cannot. The benchmark pairs 1,000 manually curated preferences with queries engineered so that a generic answer would violate the preference, then inserts real conversational distractors of up to 100k tokens between the two. In zero-shot settings, preference-following accuracy across most of the 10 evaluated models falls below 10% once the preference is a mere 10 turns (about 3,000 tokens) old. A simple reminder sentence or retrieval augmentation helps considerably at short distances, but all methods deteriorate as the conversation grows, and error analysis shows models either ignore the preference, hallucinate it, or refuse to answer. Fine-tuning on PrefEval markedly improves adherence, transfers to longer and unseen contexts, and raises attention to preference-relevant tokens.

What carries the argument

The load-bearing object is the PrefEval dataset and its evaluation protocol. Each of 1,000 underlying preferences is rendered in three forms — an explicit one-sentence statement, a two-turn implicit dialogue in which the user reveals the preference by choosing among options, and a 4–8 turn persona-driven dialogue in which the preference is mentioned in passing — yielding 3,000 preference-query pairs across 20 topics. Real user conversations from LMSYS-Chat-1M are inserted between the preference and the query as distractors to create contextual distances up to 100k tokens. The generation task is scored by four independent LLM-judge binary checks (violation, acknowledgment, hallucination, helpfulness) aggregated into four error types, with preference-following accuracy defined as the absence of any error; the classification task asks the model to pick the one of four options that respects the preference, and correlates with generation accuracy at $r=0.73$.

What would settle it

Take a random sample of several hundred generated responses spanning models, methods, preference forms, and turn counts, and have human annotators apply the paper's own four error-type checks; if human-scored preference-following accuracy at 10 turns lands well above 10% for most models, the central claim fails. A cheaper check runs the same protocol with a different judge model and compares verdicts.

Watch

Extended reading notes

Core claim

The paper's central claim is that proactive preference following is a distinct, largely unmet capability of state-of-the-art LLMs, and that PrefEval measures it. On the paper's own numbers, zero-shot generation accuracy in the explicit-preference setting drops from roughly 80% at 0–1 turns to below 30% at 5 turns and below 10% at 10 turns (about 3,000 tokens) across most models, approaching zero at 300 turns (about 100k tokens); the strongest deployed models (Claude 3.5 Sonnet, Gemini 1.5 Pro, GPT-o1-preview) all score at or below 50% at 10 turns without prompting. Prompting and retrieval interventions recover much of the short-context performance — a single reminder sentence matches or beats more complex self-critique and chain-of-thought methods, and RAG is the most consistently strong — yet still decay with context length, with the reminder falling to single digits at 300 turns for several models. Error analysis attributes the zero-shot failures largely to preference-unaware violations, while prompting shifts failures toward hallucinated preferences and unhelpful refusals. Two further findings round out the claim: introducing multiple preferences, even conflicting ones, improves adherence to the original preference; and supervised fine-tuning on PrefEval raises zero-shot adherence above the best prompting baselines while generalizing to unseen topics, longer contexts, and implicit preference forms.

Load-bearing premise

The headline 'below 10% at 10 turns' figure depends on the LLM judge (Claude 3 Sonnet) being as accurate on the full 3,000 evaluation instances as it was on the roughly 200–300 samples where it was compared to human annotators.

Editorial extensions

If this is right

  • Without prompting, current LLMs should not be trusted to carry a user's stated preferences across even a short conversation: zero-shot accuracy is below 10% at 10 turns for most of the ten models tested.
  • A one-sentence reminder to honor earlier discussion is the strongest cheap intervention, matching or beating retrieval-augmented generation at short distances and far outperforming self-critique and chain-of-thought.
  • All evaluated interventions deteriorate with context length, so long-context personalization remains open; RAG and reminder degrade to near single-digit accuracy at 300 turns for several models.
  • Supervised fine-tuning on PrefEval lifts zero-shot adherence above the best prompting baselines, and the gain transfers to unseen topics, longer contexts, and implicit preference settings.
  • The multiple-choice classification variant tracks generation accuracy closely ($r=0.73$), so cheap automatic evaluation of preference following is feasible without an LLM judge.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The success of a single reminder sentence implies the underlying capability is latent: models can retrieve and apply preferences when explicitly told to, so the measured failure is largely a proactiveness deficit rather than a capacity deficit.
  • The appendix's 'lost in the middle' analogue suggests preference adherence will worsen as services interleave more history; memory systems that surface preferences near the query may behave like the reminder intervention.
  • The finding that conflicting preferences improve adherence is striking but fragile-looking; a direct test would vary whether the later preference agrees or conflicts with the first while holding topic density fixed, to separate reinforcement of attention from topic priming.
  • A testable extension: fine-tuning on PrefEval might be improving a general 'attend to user-stated constraints' behavior, in which case the same model should also improve on instruction-following and long-context QA benchmarks — a cheap experiment to run.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces PrefEval, a benchmark with 3,000 preference-query pairs in three preference forms (explicit, implicit choice-based, implicit persona-driven) and two tasks (generation and classification), intended to measure whether LLMs can infer, remember, and proactively follow user preferences in multi-session conversations of up to 100k tokens. The authors evaluate six main open-source and proprietary models across five methods (zero-shot, Reminder, Self-Critic, few-shot CoT, RAG), supplement with results on GPT-4o, o1-preview, Claude 3.5 Sonnet, and Gemini-1.5-Pro on a single topic, analyze error types, study multiple and conflicting preferences, and show that supervised fine-tuning on PrefEval improves Mistral-7B's preference following and length generalization. The manuscript's central qualitative conclusion is that current LLMs struggle with proactive preference following, but several headline quantitative claims in the abstract are not supported by the paper's own tables.

Significance. If the numerical claims are corrected, PrefEval is a valuable resource: it is a substantial, publicly released benchmark targeting an underevaluated capability, it includes both a generation task and a fast classification task with a reported correlation of 0.73, it covers a wide range of models and methods, and it provides human-LLM agreement checks as well as fine-tuning experiments that show meaningful improvements. The main risk is not the benchmark itself but the overstatement of the findings: the abstract's 'below 10% at 10 turns across most models' and 'still deteriorates in long-context conversations' claims are contradicted by specific tables in the paper. These are internal numerical inconsistencies rather than external judgment calls, and they can be fixed by rewording the claims to match the data.

major comments (3)
  1. [Abstract and Table 5] The abstract's claim that 'in zero-shot settings, preference following accuracy falls below 10% at merely 10 turns (~3k tokens) across most evaluated models' is not supported by the paper's own generation-task results for explicit preferences. Table 5 gives 10-turn zero-shot averages of 13.41, 15.78, 9.67, 12.46, 17.35, and 15.18 percent for Claude 3 Sonnet, Claude 3 Haiku, Llama3 8B, Llama3 70B, Mistral 7B, and Mistral 8x7B, so only Llama3 8B is below 10%. The claim holds for implicit choice-based preferences (Table 8, where all six models are between 6.4% and 8.0%) and partially for persona-driven preferences (Table 11, where five of six models are below 10%), but it is not true as stated for explicit preferences or for the benchmark as a whole. Please reword the abstract and Section 3.2 to state which preference form and which model subset the threshold applies to.
  2. [Abstract and Table 4] The statement that 'even with advanced prompting and retrieval methods, preference following still deteriorates in long-context conversations' is too strong as written. Table 4 shows that GPT-o1-preview with the Reminder method achieves 0.98 at both 10 and 300 turns, and Claude-3.5-Sonnet with Reminder drops from 0.45 to 0.02 while Gemini-1.5-Pro drops from 0.91 to 0.05, so the direction and magnitude of the long-context deterioration are model- and method-dependent. The claim should be qualified as 'generally deteriorates' or restricted to the specific models and methods for which the tables show the effect.
  3. [Section 2.5 and Appendix A.17] The headline single-digit accuracy numbers rest on an LLM-as-a-judge evaluation. Section 2.5 reports validation on only 200 randomly sampled evaluations with a 5% error rate, and Appendix A.17 reports per-checker human-LLM agreement rates on 100 samples per preference form. Because the reported values are around 6-17%, a systematic judge error of a few percentage points could change the ordering of models and could move some models across the 'below 10%' threshold. The paper should either provide a stratified judge-error analysis by model, topic, and conversation length, or explicitly state that the precise numerical threshold is only a point estimate subject to evaluator error. The classification task provides useful convergent evidence for the qualitative conclusion, but it does not by itself validate the exact generative accuracy numbers.
minor comments (5)
  1. [Abstract and Section 1] The paper repeatedly says it evaluated '10 state-of-the-art LLMs,' but the full cross-topic evaluation across preference forms and methods is reported for only six models; Table 4 covers four additional models on a single travel-restaurant topic. Please qualify the model count in the abstract and the contribution list.
  2. [Section 2.2 and Appendix A.13] The text calls the dataset 'manually curated,' while Appendix A.13 describes a pipeline in which preferences and queries are generated by GPT-4/Claude and then manually filtered and rated. Please make the generation-and-filtering pipeline explicit in the main text so readers do not infer that all 3,000 pairs are purely human-written.
  3. [Figures 4 and Tables 5-13] The many reported accuracy values are percentages over roughly 50-60 preference instances per topic, but no confidence intervals or variance estimates are provided. A sentence on variance or a small number of bootstrap intervals would make the cross-model and cross-length comparisons easier to interpret.
  4. [Appendix Figures 15-19] There are minor typographical errors in the appendix prompts, including 'respomse' in the Figure 18 prompt and 'shnmp' in Figure 1. The PDF-extracted title also contains rendering artifacts; please provide a clean text version.
  5. [Section 3.7] The fine-tuning result is demonstrated on a single model (Mistral-7B) with a single topic split and no multiple seeds; the conclusion should note this limitation when claiming that fine-tuning on PrefEval 'significantly improves performance.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: PrefEval is an external benchmark with a human-validated LLM judge; the headline numbers are empirical measurements, not derived from fitted parameters.

full rationale

This is an empirical benchmark paper, and its load-bearing claims are measurements made on an externally released dataset rather than conclusions derived from fitted parameters or self-referential definitions. The generation-task accuracy is defined in Section 2.5 as the absence of any of four binary evaluator checks, with the Claude 3 Sonnet judge validated against 200 hand-checked samples (5% error) and against 100 human-annotated samples per preference form in Appendix A.17; the judge is therefore an independent measurement instrument, not a parameter fitted to force the result. The fine-tuning claim is evaluated on 20% held-out topics (Section 3.7), so it is not a training-set restatement. The only self-citation (Zhao et al., 2023, in Related Work) is contextual and not load-bearing. Preference-query pairs were curated to be challenging, including prioritizing pairs with high generic-violation rates (Section A.13), but that is benchmark stress-test design rather than circular derivation: the measured adherence values are outcomes, not logical consequences of the curation rule. The abstract's 'below 10%' phrasing is in tension with the explicit-preference averages in Table 5 unless implicit preference forms are pooled, but that is an internal numerical consistency issue, not a circular reduction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The benchmark uses no fitted physical parameters or invented entities. The main load-bearing assumptions are the validity of the LLM-based judge, the correctness of the gold labels in the benchmark, and the neutrality of the filler context. The RAG top-k is the only hand-chosen experimental hyperparameter that influences reported numbers.

free parameters (1)
  • RAG top-k = 5
    Appendix A.7: k=5 was chosen because it matched k=10 performance in preference following; k=10 improved ground-truth retrieval but acted as distraction. This is a manual, test-set-derived hyperparameter that affects reported RAG results.
assumptions (3)
  • domain assumption LLM-as-a-judge (Claude 3 Sonnet) evaluations reliably measure preference following.
    Sections 2.5 and A.17: validation on 200 samples (5% error rate) and 100 per preference form (92-98% agreement) assumes the judge generalizes to the full 3,000 pairs.
  • domain assumption Manually curated preference-query pairs have correct gold preference alignments.
    Section 2.2 and A.13: pairs generated with LLM assistance and human filtering; the gold labels are treated as ground truth.
  • domain assumption LMSYS-Chat-1M contextual turns are neutral distractors that do not introduce confounding preferences.
    Section 2.4: unrelated conversations are interspersed between preference and query; the benchmark assumes these do not inject conflicting preferences that affect the final query.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs." pith.science (2026). https://pith.science/paper/56JXPVMM

@misc{pith2026250209597,
  author       = {Pith},
  title        = {Pith review of: Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56JXPVMM}},
  note         = {Machine review of arXiv:2502.09597}
}
read the original abstract

Large Language Models (LLMs) are increasingly used as chatbots, yet their ability to personalize responses to user preferences remains limited. We introduce PrefEval, a benchmark for evaluating LLMs' ability to infer, memorize and adhere to user preferences in a long-context conversational setting. PrefEval comprises 3,000 manually curated user preference and query pairs spanning 20 topics. PrefEval contains user personalization or preference information in both explicit and implicit forms, and evaluates LLM performance using a generation and a classification task. With PrefEval, we evaluated the aforementioned preference following capabilities of 10 open-source and proprietary LLMs in multi-session conversations with varying context lengths up to 100k tokens. We benchmark with various prompting, iterative feedback, and retrieval-augmented generation methods. Our benchmarking effort reveals that state-of-the-art LLMs face significant challenges in proactively following users' preferences during conversations. In particular, in zero-shot settings, preference following accuracy falls below 10% at merely 10 turns (~3k tokens) across most evaluated models. Even with advanced prompting and retrieval methods, preference following still deteriorates in long-context conversations. Furthermore, we show that fine-tuning on PrefEval significantly improves performance. We believe PrefEval serves as a valuable resource for measuring, understanding, and enhancing LLMs' preference following abilities, paving the way for personalized conversational agents. Our code and dataset are available at https://prefeval.github.io/.

Figures

Figures reproduced from arXiv: 2502.09597 by the authors.

Figure 1
Figure 1. PREFEVAL setup overview. Key components from left to right: 1) Multi-Session Conver￾sation Setup: PREFEVAL evaluates LLMs’ ability to follow user preferences in multi-session con￾versation, challenging LLMs to handle preference inference, long-range retrieval, and context-aware preference following simultaneously. 2) Preferences and Queries: User preferences can be ex￾pressed through both explicit and implicit forms… view at source ↗
Figure 2
Figure 2. Distribution of domains and topics within P [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Zero-shot performance of LLMs with explicit preferences, aver￾aged across 20 topics. The x-axis rep￾resents the dialogue length between the user’s stated preference and the fi￾nal query, measured by both the num￾ber of tokens in the prompt and the number of conversation turns. All LLMs exhibit a rapid decline in accu￾racy as the number of turns increases. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Performance comparison of 5 methods across 6 LLMs with explicit preferences on the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison of 3 preference forms for 6 LLMs on the generation task, across varying [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Performance of the classification task across models and methods on the explicit prefer [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Distribution of 4 disjoint error types across 2 LLMs and 5 methods in the genera￾tion task with explicit preferences, where the gap between the preference and the query is 10 turns. Prompting methods introduce more hallucination and unhelpful errors, as LLMs either fab…
Figure 9
Figure 9. Figure 9: Effect of adding conflicting versus non-conflicting preferences on adherence. The red bar indicates the performance when only the original preference is present. Results are aver￾aged over five topics using a fixed 100-turn con￾versation. on the remaining unseen 20% to…
Figure 10
Figure 10. Figure 10: Performance on the 20% unseen test top￾ics: After fine-tuning, the Mistral-7B model exhibits superior preference follow￾ing accuracy compared to other baseline models. It also shows enhanced length generalization when trained with longer context inter￾leavings, demons…
Figure 11
Figure 11. Figure 11: Correlation analysis between LLM-based preference following accuracy in generation [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Comparison of RAG retrieval accuracy versus RAG method’s performance in preference [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Trends in Absolute Error Counts Across Conversation Turns for Six LLMs and Five [PITH_FULL_IMAGE:figures/full_fig_p034_13.png]
Figure 15
Figure 15. Figure 15: Evaluation prompt used for checking Preference Violation. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_15.png]
Figure 16
Figure 16. Figure 16: Evaluation prompt used for checking Acknowledgement of Preference. You will analyze a conversation between a user and an assistant, focusing on whether the assistant’s restatement of the user’s stated preference is the same preference. Evaluate the response based on t…
Figure 17
Figure 17. Figure 17: Evaluation prompt used for checking Preference Hallucination. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_17.png]
Figure 18
Figure 18. Figure 18: Evaluation prompt used for checking Helpful Response. 37 [PITH_FULL_IMAGE:figures/full_fig_p037_18.png]
Figure 19
Figure 19. Figure 19: The prompt used for expanding an explicit preference into a multiple-choice implicit [PITH_FULL_IMAGE:figures/full_fig_p038_19.png]
Figure 20
Figure 20. Figure 20: Attention score visualization comparing pre- and post-SFT model behavior on test exam [PITH_FULL_IMAGE:figures/full_fig_p039_20.png]
Figure 21
Figure 21. Figure 21: Distribution of improvements in preference region attention after SFT across 100 test [PITH_FULL_IMAGE:figures/full_fig_p040_21.png]
Figure 22
Figure 22. Figure 22: Comparison of attention score patterns between implicit and explicit preference forms [PITH_FULL_IMAGE:figures/full_fig_p040_22.png]
Figure 23
Figure 23. Figure 23: Preference following accuracy across different preference insertion locations in a fixed [PITH_FULL_IMAGE:figures/full_fig_p043_23.png]
Figure 25
Figure 25. Figure 25: Effect of adding conflicting versus non-conflicting preferences on adherence. The red bar indicates the performance when only the original preference is present. Results are aver￾aged over five topics using a fixed 100-turn con￾versation [PITH_FULL_IMAGE:figures/full…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Interactive Task Alignment as a POMDP

    cs.AI 2026-07 conditional novelty 7.0 of 10

    Under ambiguous user requests, current LLMs recover the intended task only 22–32% of the time, well below human accuracy of 48%, and post-training only partially closes the gap.

  2. TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning

    cs.CL 2026-07 conditional novelty 6.5 of 10

    On 800 jointly constrained trip tasks with a deterministic scorer and achievable gold, the best of 15 LLM agents fully solves only 46.2% of feasible plans, with unstated persona needs as the universal bottleneck.

  3. ClawRec: A Claw-Native Recommender System

    cs.IR 2026-07 conditional novelty 6.5 of 10

    ClawRec turns cross-platform behavior into a temporally managed user state and role-aware complementary slates, beating agentic baselines on a new synthetic life-event benchmark.

  4. TrajWiki: Source-Grounded Memory Trajectories for Long-Horizon Dialogue Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    TrajWiki stores long-dialogue facts as evolving, source-linked claim histories organized into wiki pages, improving long-horizon QA on LoCoMo and MedMT-Bench in reported experiments.

  5. Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Persona2Web is a new open-web benchmark where agents must infer a user's preferences from synthetic browsing history to solve intentionally ambiguous queries; current best agents score 13% success.

  6. After Talking with 1,000 Personas: Learning Preference-Aligned Proactive Assistants From Large-Scale Persona Interactions

    cs.HC 2026-02 conditional novelty 6.0 of 10

    A two-stage framework — category-structured fine-tuning on LLM-simulated personas plus on-device activation steering — improves proactive-assistant timing and perceived quality, though the biggest gains are measured w...

  7. Galaxy: A Cognition-Centered Framework for Proactive, Privacy-Preserving, and Self-Evolving LLM Agents

    cs.AI 2025-08 reject novelty 6.0 of 10

    Galaxy couples a cognitive tree structure with a meta-agent to make LLM assistants proactive, privacy-preserving, and self-evolving.

  8. LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logs

    cs.AI 2026-08 conditional novelty 5.0 of 10

    LUNAR is a synthetic benchmark anchored in real-world logs that shows current LLMs cannot reliably integrate cross-domain behavioral evidence, and that stronger personalization often comes with weaker privacy protection.

  9. RGMem: Renormalization Group-inspired Memory Evolution for Language Agents

    cs.AI 2025-10 conditional novelty 4.0 of 10

    RGMem, a hierarchical memory framework with thresholded updates inspired by renormalization group ideas, reports state-of-the-art scores on the LOCOMO long-term conversational memory benchmark.

Reference graph

Works this paper leans on

46 extracted references · 45 canonical work pages · cited by 9 Pith papers

  1. [1]

    The charming Hˆotel des DeuxˆIles is a cozy three-story boutique hotel near the Seine

    work page

  2. [2]

    The iconic Hyatt Regency Paris´Etoile is a soaring high-rise hotel with panoramic city views

    work page

  3. [3]

    The luxurious Shangri-La Hotel Paris is located in a historic high-rise building with stunning views

    work page

  4. [4]

    User: The H ˆotel des Deux ˆIles sounds perfect for my needs

    The modern Renaissance Paris Republique Hotel offers sleek accommodations in a towering skyscraper. User: The H ˆotel des Deux ˆIles sounds perfect for my needs. The high-rise options don’t really appeal to me. Assistant: Understood, the Hˆotel des DeuxˆIles is an excellent choice. I don’t enjoy games with pixel art graphics. User: What video game would y...

    work page

  5. [6]

    Undertale is a popular indie game with a retro pixel art aesthetic

    work page

  6. [7]

    I’d suggest Horizon Zero Dawn, with its stunning graphics and open world

    work page

  7. [8]

    You might enjoy Stardew Valley, with its charming pixel art style

    work page

  8. [9]

    User: I’ll go with Horizon Zero Dawn

    Celeste is a great platformer with beautiful pixel art graphics. User: I’ll go with Horizon Zero Dawn. The other options don’t quite appeal to me. Assistant: Horizon Zero Dawn is an excellent choice. I strictly avoid restaurants that serve foods con- taining gluten due to a severe gluten intolerance. User: What are some good lunch options near the office?...

    work page

Show all 46 references
  1. [10]

    A Mexican cantina with a wide selection of spicy tacos and enchiladas

  2. [11]

    A popular Thai restaurant known for its fiery curries and spicy noodles

  3. [12]

    A cozy Italian bistro serving mild pasta dishes and salads

  4. [13]

    User: The cozy Italian bistro serving mild pasta dishes and salads sounds perfect

    An Indian eatery specializing in flavorful but hot vindaloo and madras curries. User: The cozy Italian bistro serving mild pasta dishes and salads sounds perfect. The other options seem a bit too spicy for my taste. Assistant: Understood, the Italian bistro is an excellent cho...

  5. [14]

    Homemade peanut butter cookies, a classic crowd-pleaser

  6. [15]

    A fresh fruit salad with a variety of berries and citrus

  7. [16]

    Thai peanut noodle salad, a flavorful and filling option

  8. [17]

    User: I think I’ll go with the fresh fruit salad

    Peanut brittle, a crunchy and sweet treat for everyone. User: I think I’ll go with the fresh fruit salad. The other options sound great, but I’d rather avoid anything with peanuts or peanut butter. Assistant: A fresh fruit salad is an excellent choice. I am not interested in d...

  9. [18]

    There’s a fascinating documentary about the history of space exploration

  10. [19]

    I heard the new nature documentary is really insightful

  11. [20]

    There’s a great new documentary on climate change

  12. [21]

    32 Published as a conference paper at ICLR 2025 Table 16: Implicit Preferences: Persona-driven Dialogue Example

    How about a fun comedy or an action-packed thriller? User: The documentaries don’t really interest me.How about a fun comedy or an action- packed thriller? Assistant: A comedy or thriller sounds great. 32 Published as a conference paper at ICLR 2025 Table 16: Implicit Preferen...

    2025

  13. [22]

    Answer "No" if:

    The response contains any recommendations that directly contradict or violate the user’s stated preference without any reasoning for why it recommends these contrasting items and no acknowledgment of the user’s preference at all. Answer "No" if:

  14. [23]

    The response respects the user’s stated preference

  15. [24]

    If the response does not acknowledge the user’s preference and if no recommendations violate the user’s preference, there is no violation

    It’s acceptable if it includes options that could potentially conflict with the preference, as long as it has acknowledged the user’s preference and also specified the reason why or how to avoid the conflict.3. If the response does not acknowledge the user’s preference and if ...

    2025

  16. [25]

    Check if the response explicitly or implicitly mentions or references a user preference

  17. [26]

    Answer "Yes" if:

    The content of the preference is irrelevant for this check; only its presence matters. Answer "Yes" if:

  18. [27]

    Based on your previous preference for xxxx

    The response explicitly mentions or refers to a user preference in answering the user’s question. Examples include: "Based on your previous preference for xxxx"; "Based on your previous preference, I would suggest you xxxx"; "Since you mentioned you prefer/dislike xxxx"; "Sinc...

  19. [28]

    The response assumes the user preference in answering the user’s question implicitly. For example, when the user asks ’Can you recommend me cars to drive?’, if the response is ’Based on your preference, I will recommend non-EV cars, ...’, then this indicates the assistant assu...

  20. [29]

    The assistant’s restatement differs from the original preference in meaning or intent

  21. [30]

    The assistant’s restatement contradicts the original preference

  22. [31]

    Answer "No" if has not hallucinated:

    The assistant’s restatement is not related or irrelevant to theoriginal preference. Answer "No" if has not hallucinated:

  23. [32]

    The assistant’s restatement matches the original preference

  24. [33]

    The assistant’s restatement is a minor paraphrase that fully preserves the meaning and intent of the original preference

  25. [34]

    The restatement is just empty, no hallucination. Here is the information:Original user preference: [preference] Assistant’s restatement: [assistant restatement] Examine the original preference and the assistant’s restatement meticulously and answer. Please answer in this exact...

    2025

  26. [35]

    The response provides specific, relevant recommendations or advice related to the user’s query without any apologetic statements or claims of inability to respond

  27. [36]

    Answer "No" if any of the following:

    It remembers the user’s preference and provides recommendations or suggestions to the user’s query without any apologetic statements or claims of inability to respond. Answer "No" if any of the following:

  28. [37]

    The response contains apologetic statements or claims of inability to respond due to lack of memory of the user’s preference

  29. [38]

    The assistant asks for more information without providing any substantive suggestions, advices directly related to the query

  30. [39]

    The response doesn’t address the user’s specific query

  31. [40]

    Unfortunately, I don’t actually have enough context about your specific preferences for xxx

    Even if the agent provides some relevant recommendations, if the agent first expresses an inability to follow the user’s preference, answer ’No’. Examples of unhelpful responses (Answer: No): "Unfortunately, I don’t actually have enough context about your specific preferences ...

    2025

  32. [41]

    For each topic, we crafted detailed descriptions and subtopics to ensure comprehensive coverage of various preference domains, utilizing Claude 3 for assistance

    Topic Generation: We began by generating and selecting 20 distinct topics (shown in Fig- ure 2) that are diverse and commonly encountered during advice-seeking or recommendation- focused conversations with chatbots. For each topic, we crafted detailed descriptions and subtopic...

  33. [42]

    I strictly avoid restaurants that serve foods containing gluten due to a severe gluten intolerance,

    Large-scale Sampling of Preferences and Queries: Using Claude 3 Sonnet, we generated approximately 10,000 preference-question pairs. Each pair comprises an explicit preference statement and a related query (e.g., preference: “I strictly avoid restaurants that serve foods conta...

  34. [43]

    Pairs with higher violation rates were prioritized to create a more challenging dataset

    Extensive Manual Filtering Process: We implemented a multi-stage filtering approach in- volving human labelers and LLM-based evaluators (using GPT-4o, Claude 3 Sonnet) to evalu- ate each preference-question pair based on the following criteria: • Validity Assessment: Labelers ...

  35. [44]

    We then manually selected approximately 50 preferences per topic, resulting in a final dataset of 1,000 high-quality explicit preference-question pairs

    Final Selection: The filtering process yielded approximately 3,000 high-quality pairs. We then manually selected approximately 50 preferences per topic, resulting in a final dataset of 1,000 high-quality explicit preference-question pairs. Step 2: Generation of Implicit Choice...

    2025

  36. [45]

    We first generated and filtered 100 distinct and diverse personas using Claude 3.5 Sonnet, ensuring that the personas were topic-independent to prevent preference conflicts

  37. [46]

    lost in the middle

    For each of the 1,000 explicit preferences, we randomly assigned one persona and we make sure the persona does not conflict with or reveal the preference. Using Claude 3.5 Sonnet, we then generated 5–8 turn conversations that incorporated both the explicit preference and the a...

    2024

  38. [2018]

    needle-in-a-haystack

    Association for Computational Linguistics. doi: 10.18653/v1/P18-1205. URL https: //aclanthology.org/P18-1205. Xinrong Zhang, Yingfa Chen, Shengding Hu, Zihang Xu, Junhao Chen, Moo Khai Hao, Xu Han, Zhen Leng Thai, Shuo Wang, Zhiyuan Liu, et al. ∞ bench: Extending long context ...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.