{"id":"a9b587d5-c8ce-47e6-9022-e6084333aebe","arxiv_id":"2505.04260","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Preference-based activation steering lets users control chatbot personalization with a linear factor, and a small user study found that steered chatbots align better with self-reported preferences than prompting alone.","lead":"This paper tests a new way to personalize chatbots: instead of describing preferences in words, users move a slider or answer quick comparisons that adjust how strongly the model expresses traits like luxury or budget. A small user study found that these steered chatbots matched users' stated preferences better than a standard prompt-only chatbot, though the three interface designs performed unevenly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"U2's message-level correlations ignore participant clustering; the headline 'steering beats prompting' may not survive a participant-level reanalysis.","rationale":"The reader's weakest assumption is that the expressed-preference metric is unvalidated against human judgments. That is a legitimate concern, and I agree it affects every quantitative claim. However, the single most load-bearing problem for the paper's central claim is more direct: Section 5.3 U2 computes correlations over pooled individual messages while the ground-truth variable is participant-level and constant within each participant-interface cell. Because the paper has n=14 participants and no clustering correction, the reported p<0.001 for All Steering versus the non-significant PROMPT baseline does not establish that steering beats prompting. The paper itself acknowledges that participants' in-task preferences diverged from the pre-task 'ground truth' ratings, which further weakens the criterion. This is not a matter of external consensus; it is an internal statistical validity issue that can be settled by reanalysis. The computational experiments (E1-E4) provide suggestive evidence that steering can modulate preference-related text, and the qualitative findings about interface values are useful, so the paper is not without merit. But the user-study headline should be accepted only conditionally, with the required reanalysis and data release. The final verdict therefore stays CONDITIONAL, but the binding condition should be the participant-level, cluster-aware reanalysis rather than only metric validation.","tokens_in":28717,"tokens_out":5181,"duration_ms":61058,"concrete_test":"Recompute the U2 alignment analysis at the participant-interface level: for each participant and each interface, average the preference effect across that participant's messages; then fit a linear mixed model with a random intercept for participant, fixed effects for interface and preference dimension/task, and compare the All Steering contrast against the PROMPT baseline using a likelihood-ratio test or cluster bootstrap. Report the participant-level correlation and its confidence interval. If the All Steering versus PROMPT contrast is not significant at p<0.05, or if the participant-level correlation is no longer distinguishable from PROMPT, the headline claim is unsupported and the conclusion must be weakened. Releasing de-identified message-level data and analysis code would make this check possible.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that steered chatbots align better with underlying user preferences than prompting alone rests on the U2 correlation analysis in Section 5.3, Figure 8b. In that analysis, each LLM message is treated as an independent observation, but the independent variable ('ground truth preference') is a single self-report per participant per task, constant across all messages from that participant. With only 14 participants and several messages per participant per interface, the effective sample size is far smaller than the number of plotted points, so the reported p<0.001 for All Steering (r=0.38) is not trustworthy. Collapsing to participant-level or participant-interface-level units would likely remove or substantially weaken the significance. Moreover, the paper never tests whether the All Steering correlation is significantly higher than the PROMPT correlation (r=0.03); separate p-values do not establish a difference. The metric-validity concern raised by the reader is real, but this statistical issue is more directly load-bearing: even if the embedding-similarity metric perfectly measured expressed preference, the headline claim would still not follow from the data as analyzed. The manuscript also releases no code or de-identified data, so the analysis cannot currently be independently checked.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes activation steering as a lightweight method for personalizing LLM responses along interpretable preference dimensions, exposing the steering strength as a linear factor that users can control directly. It constructs steering vectors for five preference dimensions using GPT-4o-generated contrastive samples, and evaluates the resulting control through four computational experiments (E1-E4) across five open-source LLMs, measuring expressed preference as relative cosine similarity to Yelp review corpus embeddings. It then reports a within-subjects user study (n=14) comparing three steering-based interfaces (SELECT, CALIBRATE, LEARN) against a prompting-only baseline, with quantitative analyses of steering-effect correspondence, alignment with self-reported ground-truth preferences, subjective perceptions, and qualitative interview themes. The central claim is that steered chatbots align with users' underlying preferences significantly better than prompting alone, while survey and interview data reveal heterogeneous preferences among users regarding control, transparency, and persistence.","tokens_in":28903,"tokens_out":4149,"duration_ms":44801,"significance":"If the central measurement and statistical claims held, this would be a useful contribution: it provides a resource-efficient, cold-start personalization mechanism, a broad computational exploration across five models and five preference dimensions, and a genuinely user-centered comparison of three interaction designs for exposing a linear steering factor. The computational dose-response results in E1 are internally consistent across models and dimensions, and the qualitative analysis in U4 is informative and well-grounded in participant quotes. The manuscript is also honest about several limitations of the prototype learning and calibration algorithms. However, the headline empirical claim rests on a message-level correlation analysis that ignores participant nesting, and the core metric for 'expressed preference' is never validated against human judgments. These are load-bearing issues that require reanalysis and additional validation rather than mere editing. The paper does not release code, data, or analysis scripts, which limits independent verification.","major_comments":[{"comment":"The U2 analysis treats each LLM message as an independent observation, but the predictor (ground-truth preference) is a single self-report per participant per task, repeated for every message from that participant. With n=14 participants, the effective sample size is far smaller than the number of plotted points, so the reported p<0.001 for All Steering does not by itself support the headline claim that steering outperforms prompting. The authors should reanalyze at the participant level (or with a mixed-effects model including random intercepts for participants) and, crucially, report a direct test of whether the All Steering correlation (r=0.38) is significantly higher than the PROMPT correlation (r=0.03); separate p-values do not establish a difference between two correlations.","section":"Section 5.3, Figure 8b"},{"comment":"The central measurement premise, expressed preference as Effect = cos(e_o, \\bar e_+) - cos(e_o, \\bar e_-) against mean Yelp review embeddings, is never validated against human judgments. Every quantitative claim in E1-E4 and U1-U2 depends on this operationalization, but the paper provides no evidence that relative cosine similarity to mean review embeddings tracks how strongly a preference is expressed in text, as opposed to topic or lexical overlap. A validation study with human ratings of preference expression on the same steered outputs, or use of an existing labeled preference-strength corpus, is needed before the dose-response and user-alignment claims can be interpreted.","section":"Section 4.2"},{"comment":"The 'Favourite' comparison is circular: the favourite interface is defined as the one with the highest Likelihood-to-Use rating, and that same outcome is then compared between Favourite and PROMPT. This guarantees by construction that Favourite scores at least as high as PROMPT on Likelihood to Use, so the reported 'highly significant differences across all perception categories' do not constitute evidence that steerable chatbots are preferred. The authors should define favourite from an independent measure (for example, an explicit preference ranking question) or otherwise adjust the analysis for the selection rule.","section":"Section 5.3, U3 analysis"}],"minor_comments":[{"comment":"The computational experiment is introduced as E4 in Section 4.3 but is labelled E5 in Figure 5 and Appendix F; the numbering should be harmonized throughout.","section":"Section 4.3, Figure 5, Appendix F"},{"comment":"The update rule is underspecified because p(·) is only described as a 'simple linear transformation' and the functional steering range used by p is determined post hoc per model in Appendix A; the exact mapping and the criterion used to set the range should be stated explicitly.","section":"Equation (1)"},{"comment":"The scatterplots plot many overlapping points, making the reported correlations difficult to assess visually; adding transparency or jitter and reporting participant-level summary statistics would improve transparency.","section":"Figure 8"},{"comment":"There are minor typographical errors ('jewlery' in Table 1, 'disatisfied', 'feedback feedback', 'althougth'), and the manuscript does not state a data/code availability plan, which would help independent verification of the computational experiments.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version. The paper has a good idea — expose activation-steering strength as a user-facing control for cold-start personalization — and the computational experiments are more thorough than most in this space. But the user study's central quantitative claim, that steered chatbots align better with underlying preferences than prompting alone, doesn't hold up as analyzed. The U2 correlation treats each LLM message as independent while the ground truth is a single self-report per participant per task. With 14 participants, the effective sample is much smaller than the plotted points suggest, and the paper never tests whether All Steering (r=0.38) is actually significantly different from PROMPT (r=0.03). That's a load-bearing flaw, not a minor one.\n\nWhat's genuinely new: the three-interface comparison (SELECT, CALIBRATE, LEARN) and the qualitative findings about control, transparency, and persistence are useful. The LEARN update rule, despite being simple, is a reasonable proof-of-concept. The computational work — dose-response curves across 5 models and 5 dimensions, prompting interaction, multi-steering, simulated learning — is coherent and, as far as I can tell, honestly reported. The limitations section is candid about the proof-of-concept status of the learning and calibration algorithms.\n\nSoft spots, in proportion. The unvalidated evaluation metric (relative cosine similarity to mean Yelp review embeddings) is a real concern but secondary; the clustering issue is the one that directly undermines the headline. The post-hoc 'favourite interface' analysis is weak, as the authors admit. No code or de-identified data is released, which makes independent checking impossible. The experiment numbering is inconsistent (E4 vs E5) — cosmetic but sloppy.\n\nWho benefits: HCI researchers and anyone building cold-start personalization interfaces. The design insights and qualitative themes are worth engaging with, even if the quantitative support needs reanalysis. I'd send it to review with a request for major revision — specifically, a participant-level or mixed-effects reanalysis of U2, a direct test of the steering-vs-prompt difference, and ideally some validation of the metric.\n\nRecommendation: conditional accept. The computational core is suggestive, the interface work is valuable, but the user-study claim needs to be re-analyzed before this should be cited as evidence that steering beats prompting.","headline":"The interface work is genuinely interesting, but the headline 'steering beats prompting' relies on a correlation analysis that ignores participant clustering and never compares the two correlations directly.","tokens_in":29470,"tokens_out":2573,"would_cite":false,"duration_ms":24949,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Activation steering lets a user tune an LLM's output along a preference axis, and steered chatbots matched hidden preferences where prompting did not.","keywords":["LLM personalization","activation steering","steerable chatbots","preference expression","cold-start personalization","chatbot interfaces","user study","sentiment-based learning"],"falsifier":"Have human raters read pairs of chatbot replies that differ in steering strength and rank which reply expresses, say, budget versus luxury more strongly, then compare those rankings with the relative-cosine-similarity metric used in the paper. If the metric's ordering disagrees with raters on a substantial fraction of pairs, the paper's quantitative claims about alignment with true preferences lose their support; a second falsifier would be re-running the user study with a different embedding model or different trait corpora and checking whether the correlations survive.","tokens_in":28487,"feed_emoji":"🎛️","tokens_out":7034,"duration_ms":68922,"temperature":0.7,"pith_summary":"Large language models default to the average user, so personalizing them usually means prompting, which cold-start users struggle to do well. This paper proposes making one internal dial visible: activation steering, where a single linear strength factor added to the model's hidden activations amplifies or dampens a preference such as budget-versus-luxury. Across five open-source LLMs, steered outputs track the applied strength nearly linearly within a per-model functional range, and steering stacks with prompting and with multiple preference dimensions. In a within-subjects user study, steered chatbots' expressed content correlated with participants' self-reported true preferences (r=0.38, p<0.001 across all steered conditions) while a prompt-only baseline showed no correlation (r=0.03, p=0.84). The paper's case is that steering can replace part of the burden of prompt specification and gives users a concrete, controllable handle on personalization.","feed_headline":"A single steering dial beats prompting for chatbot personalization","feed_subtitle":"Cold-start users can match their hidden preferences by moving a slider, while prompt-only chat shows no correlation.","key_machinery":"The load-bearing mechanism is a preference steering vector: a logistic-regression probe is trained to separate positive from negative exemplars of a trait inside the model's own activations, and the probe's coefficients become the vector $v_i$ that is added to the hidden state as $h_{\\text{steered}} = h + d_i v_i$, where $d_i$ is the user-facing strength and negative values steer toward the opposite trait. To measure whether a reply expresses a preference, the paper computes the relative cosine similarity $\\cos(e_o, \\bar e_+) - \\cos(e_o, \\bar e_-)$ between the reply embedding and the mean embeddings of filtered Yelp review corpora for the two traits; this number is the dependent variable in most of the computational and user-study analyses. For the LEARN interface, a second mechanism updates the scalar from user feedback via $d^*_{t+1} \\leftarrow d^*_t + p(\\text{dissatisfaction}(x_t) \\cdot \\text{direction}(x_t))$, using a pretrained sentiment classifier for the dissatisfaction term and reference-phrase similarity for the direction term.","core_discovery":"On its own terms, the paper's central claim is that soft user preferences can be operationalized as a scalar on a precomputed direction in the LLM's activation space, and that injecting that direction into the residual stream yields outputs that express the preference in proportion to the scalar. The computational experiments show a near-linear relationship between steering strength and a perplexity-normalized preference effect within a per-model functional range; they also show that prompting shifts but does not erase the steered baseline, and that two preference dimensions can be steered additively when they are not too collinear. The user study then reports that, aggregated across the SELECT, CALIBRATE, and LEARN interfaces, steered chatbots' expressed preferences correlate with participants' self-reported ground truth at r=0.38 (p<0.001), versus r=0.03 (p=0.84) for the PROMPT baseline, and the authors therefore claim steering achieves significant improvement in generating relevant preference content over prompting alone. The study simultaneously documents that participants disagree sharply about which interface they want, revealing heterogeneous values around control, transparency, effort, and persistence.","pith_inferences":["If the preference-effect metric is valid, the near-linear strength-response curves suggest a calibration protocol: a few pairwise comparisons could place a user on each dimension's monotone segment and map their answer to a steering strength, which is essentially what CALIBRATE approximates with only two or three rounds.","The per-model functional range implies that the same numeric dial means different things on different models; an interface that remaps slider positions to each model's linear zone could outperform the fixed -100 to 100 mapping used here.","The user study's heterogeneity hints that a combined interface—direct slider for immediate control, a calibration pass for initialization, and transparent learned updates for persistence—might satisfy more users than any single design, though the paper only suggests this as future work.","A direct test of the framework would be whether steering strengths inferred from one conversation predict a user's choices in a new task domain; if they do, the scalar profile is a genuine preference model rather than a per-task fitting artifact."],"forward_implications":["Cold-start personalization no longer requires users to articulate preferences: one scalar per dimension can be set directly or learned within a few turns of conversation.","Steering and prompting are complements rather than competitors: the vector sets a latent baseline, and natural-language follow-ups add contextual variation on top of it.","Multiple preference dimensions can be combined into a single additive profile, at least when the dimensions are reasonably orthogonal, opening a path from single-dial demos to multi-axis assistants.","Interface choice is not neutral: because users split over control, transparency, and effort, a deployable steerable chatbot would likely need more than one interaction modality, or an easy way to switch.","Steering's inference-time-only cost makes the approach practical for on-device assistants and avoids storing an explicit user profile, which the paper argues is a privacy advantage over history-based personalization."],"supporting_citations":[{"why":"Supplies the layer-wise logistic-probe method for computing steering vectors and the parameter-selection procedure the paper adopts.","marker":"[68]"},{"why":"Establishes contrastive activation addition, the base paradigm of injecting a positive-minus-negative vector into the residual stream at inference time.","marker":"[49]"},{"why":"Provides the sentence embedding model used to score preference effects and to determine update direction in LEARN.","marker":"[54]"},{"why":"Supplies the sentiment classifier whose negative and neutral probabilities form the dissatisfaction signal in the LEARN update rule.","marker":"[7]"},{"why":"Provides the OASST2 real-user queries that the computational experiments run over.","marker":"[34]"}],"fun_headline_variants":["Steer, don't prompt: dial beats typing for chatbot taste","One slider to match your preferences: steering beats prompting","LLM personalization: a turning knob outperforms typed requests","Forget prompts, turn a dial: chatbot personalization redefined","Chatbot steering: adjust a scalar, not your phrasing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the metric used to score how strongly a preference appears in a reply—closeness of the reply's embedding to the average embedding of Yelp reviews for that trait—actually matches how strongly a person would perceive the preference, since every correlation result in the paper is computed against that metric.","fun_headline_variants_meta":{"raw":{"variants":["Steer, don't prompt: dial beats typing for chatbot taste","One slider to match your preferences: steering beats prompting","LLM personalization: a turning knob outperforms typed requests","Forget prompts, turn a dial: chatbot personalization redefined","Chatbot steering: adjust a scalar, not your phrasing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1560,"prompt_tokens":943,"completion_tokens":617,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":532}},"tokens_in":559,"tokens_out":617,"duration_ms":6691,"temperature":1.0,"reasoning_tokens":532,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:33:54.939914+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human raters read pairs of chatbot replies that differ in steering strength and rank which reply expresses, say, budget versus luxury more strongly, then compare those rankings with the relative-cosine-similarity metric used in the paper. If the metric's ordering disagrees with raters on a substantial fraction of pairs, the paper's quantitative claims about alignment with true preferences lose their support; a second falsifier would be re-running the user study with a different embedding model or different trait corpora and checking whether the correlations survive.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the OASST2 real-user queries that the computational experiments run over."}],"review_version":1}