{"id":"1dc070a9-79c3-48e3-b2eb-216bfd1e92b0","arxiv_id":"2504.17390","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Introduces PicPersona-TOD, the first task-oriented dialogue dataset with user images as persona, together with a multimodal NLG baseline called Pictor.","lead":"This paper builds a dataset that pairs task-oriented conversations with user photos, letting dialogue systems adjust tone, greetings, and recommendations to the person in the image. The authors also train a small vision-language model, Pictor, to use this dataset for personalized responses.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Emotion-label distribution in §2.2 sums to 103.91%, so the image-dialogue pairing step is not trustworthy; an audit and coherence test are required.","rationale":"I agree with the reader's weakest assumption: the image-dialogue pairing is the load-bearing step for the dataset's central claim, and the reported emotion distribution is internally inconsistent. This is a concrete, verifiable flaw rather than a matter of taste or consensus. I do not think it warrants rejection, because the rest of the pipeline could still produce a useful personalized dataset through style transfer guided by the image; the concern is that the claimed grounding of the persona in the image is not established. The conditional verdict already captures this uncertainty, so I recommend no change. Other concerns, such as the small human evaluation set, absent error bars, and the lack of baselines in the unseen-domain generalization test, are real but secondary; they would not settle the central claim as sharply as an audit of the emotion-alignment step. The paper does have independent support in its detailed prompts, explicit filtering procedures, and the DST/policy results showing that task information is largely preserved, which is why the contribution remains salvageable.","tokens_in":21370,"tokens_out":5460,"duration_ms":57104,"concrete_test":"Download the released dataset and regeneration prompts; recompute the per-image and per-dialogue emotion label counts and the joint distribution after the §2.2 matching rule. If the three class percentages still fail to sum to 100 after correcting any aggregation bug, the alignment is unreliable. Then, with corrected labels, draw 50 same-label and 50 mismatched image-dialogue pairs from the same pool, mask the labels, and have three annotators judge whether each dialogue's content is consistent with the image persona description. If same-label pairs are not judged significantly more coherent than mismatched pairs, the emotion-matching step does not ground the image-persona claim and the dataset premise is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 builds the core novelty on emotion-based alignment: an LLM labels each image positive/neutral/negative, a sentiment model labels each dialogue, and an image-dialogue pair is accepted only when labels match. The paper then reports the dataset emotion distribution as 50.92% positive, 52.44% neutral, and 0.55% negative (repeated in Appendix B). Since these are mutually exclusive predicted labels for the same dataset, the percentages must sum to 100%; they sum to 103.91%. The labels or the aggregation are therefore unreliable, and the 'same predicted emotion label' criterion cannot be verified from the manuscript. Because the dataset's central claim is that user images are genuinely part of the persona, a broken or unverifiable alignment step is load-bearing: it opens the possibility that image-dialogue pairs are effectively arbitrary, with the image only serving as a post-hoc style-transfer target after the dialogue has been rewritten to match the image. The filtering pipeline (§2.5) checks style strength and direction, DST/policy consistency, and naturalness, but it never checks whether the image and dialogue plausibly belong to the same person; the 100-dialogue human evaluation in §4.1 also does not isolate the alignment step. An audit of the emotion labels and a direct pair-coherence test is therefore the minimal evidence needed before the dataset premise can be accepted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PicPersona-TOD, a task-oriented dialogue dataset that pairs FFHQ user images with dialogues from MultiWOZ 2.2 and SGD, rewrites user and system utterances for style personalization using GPT-4o, filters the result with style-strength, style-direction, semantic, and naturalness checks, and trains an NLG baseline, Pictor, on the dataset. The authors report human evaluations showing high dataset quality and personalization preference, lexical and emotional analyses, and a generalization experiment on unseen domains. The central claim is that this is the first visual-persona TOD dataset and that it enables personalized responses without compromising task information.","tokens_in":21683,"tokens_out":4166,"duration_ms":43531,"significance":"If the dataset is sound, it fills a clear gap: existing TOD datasets use only textual personas, while prior visual-persona dialogue work targets chit-chat. PicPersona-TOD is large (18,148 dialogues, 18 services), grounded with external knowledge, and released with a public repository, which are practical strengths. The paper also deserves credit for including human evaluation with strong inter-rater agreement, for comparing against textual-personalization baselines, and for candidly discussing its limitations, including the absence of a direct human-curated benchmark. The main risks are the emotion-based image-dialogue alignment step, the lack of baselines/error bars in the generalization and model-comparison claims, and the reliance on GPT-4-family models for both generation and evaluation. These issues are local and fixable, but they are load-bearing for the dataset premise and for the headline empirical claims.","major_comments":[{"comment":"The reported emotion distribution in the dataset is 50.92% positive, 52.44% neutral, and 0.55% negative; these sum to 103.91%, not 100%. Since the paper states that image-dialogue pairs are accepted only when the predicted emotion labels match, these mutually exclusive labels underlie the entire alignment step, and the inconsistent sum means the labels or their aggregation cannot be fully trusted. This opens the possibility that a non-trivial fraction of image-dialogue pairs are effectively arbitrary. I request a full audit of the emotion labels (including the dialogue sentiment classifier output), a corrected distribution, and a direct pair-coherence test, for example human judges rating whether each image and dialogue plausibly belong to the same person, on a random sample. Without this evidence, the central dataset premise that user images are genuinely part of the persona is not verified.","section":"2.2 (and Appendix B)"},{"comment":"The unseen-domain generalization claim is presented without any baseline. The reported scores of 3.51-3.96 on a 4-point scale are only interpretable relative to a comparison system, such as Pictor trained on seen domains only, a non-personalized NLG, or the original unpersonalized system responses from MultiWOZ/SGD. I recommend adding such a baseline and reporting annotator-level agreement or confidence intervals, since the current table does not establish that the high scores are due to personalization rather than to the generic quality of the generated utterances.","section":"6.2, Table 4"},{"comment":"The model comparison in Section 6.1 is adjudicated by GPT-4, which belongs to the same model family as the GPT-4o generator used to create the training data. Human evaluation in Section 4.1 mitigates this concern for dataset quality, but for the specific comparison between Pictor and GPT-4o-mini there is no human validation. The reported superiority could partly reflect a judge-side preference for GPT-4o-style generations. I recommend adding a human evaluation on the sampled 100 dialogues or, failing that, reporting the human-GPT agreement on this specific comparison.","section":"6.1, Figure 9"},{"comment":"Table 5 states that each experiment was repeated three times and averaged, but no variance, standard deviation, or significance test is reported, so the claimed differences, such as the impression component improving style scores, are not statistically supported. Table 6 similarly reports DST and policy differences without error bars or significance tests, which weakens the conclusion that PicPersona-TOD maintains information accuracy on par with MultiWOZ. Please report mean and variance (or bootstrap confidence intervals) and, where feasible, a paired significance test for the key comparisons.","section":"Tables 5 and 6"},{"comment":"The style-strength and style-direction filtering thresholds, 2.5×IQR and 4.5×IQR, are introduced without justification or sensitivity analysis. Because filtering removes 1.49% and 1.98% of the dataset in these stages, the final dataset properties depend on these choices. I ask for a short sensitivity analysis showing how the final dataset size, personalization strength, and downstream evaluation results change under a range of threshold values, or at least a rationale for why these specific multipliers are appropriate.","section":"2.5"}],"minor_comments":[{"comment":"Typographical errors are frequent in the appendix prompts: 'Dose' should be 'Does', 'Promps' in the Figure 14 caption should be 'Prompts', and 'sLLM' in Section 6.1 should be 'small LLM' or similar. These should be corrected for a camera-ready version.","section":"Various prompt templates (Appendix I)"},{"comment":"The model name 'LLaV A' appears with an erroneous space throughout the paper and should be 'LLaVA'.","section":"Throughout"},{"comment":"The phrase 'lacks details and concurrency about the users' appears to use 'concurrency' where 'consistency' or 'currency' was intended; please revise.","section":"Introduction"},{"comment":"The paper specifies that 100 dialogues are sampled for Section 6.1 and 100 per domain for Section 6.2, but it does not state whether the Section 6.1 sample overlaps with the Section 6.2 unseen-domain samples; please clarify to avoid ambiguity about the evaluation protocol.","section":"Appendix G"},{"comment":"The Limitation paragraph 'Lack of Direct Benchmark Comparison' is directly relevant to Section 6.2's generalization claim; consider moving or cross-referencing it there so that the absence of a baseline is acknowledged at the point of the claim.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the emotion distribution in Section 2.2 is valid and should be the main focus of revision. I do not see a fundamental circularity problem because the human evaluation and human-GPT correlation provide external grounding, but the model comparison in Section 6.1 would be considerably stronger with human adjudication. The paper is within the journal's scope, and the requested fixes, while substantial, do not require abandoning the approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the artifact: PicPersona-TOD is the first task-oriented dialogue dataset that ties user images to persona and personalizes system responses accordingly. The pipeline is spelled out in unusual detail, from emotion-based pairing through style transfer, RAG-grounded recommendations, and multi-stage filtering. The authors also train a baseline NLG model, Pictor, and test it on unseen domains with human ratings. That is real work, and the paper is honest about its limitations, including the lack of a direct benchmark comparison and the reliance on GPT-4o for generation and GPT-4 for evaluation.\n\nThe main soft spot is exactly where the stress-test note points. The emotion distribution in Section 2.2 and Appendix B sums to 103.91% (50.92 + 52.44 + 0.55). That is not rounding; it means the labels or their aggregation are broken. Since emotion matching is the criterion for pairing images with dialogues, this undermines the verifiability of the alignment step. I would not call it fatal, though. The final human evaluation in Section 4.1 directly rates whether user utterances match the image, with a score of 3.89/4 and Krippendorff's alpha of 0.85. That suggests the finished pairs are coherent even if the original pairing criterion was flawed. But the reported distribution needs a correction, and the alignment step deserves an explicit audit and a pair-coherence test.\n\nOther issues are minor in comparison. Tables 5 and 6 have no error bars, even though the ablation repeats experiments three times. The unseen-domain generalization claim in Table 4 lacks a non-personalized baseline, so we do not know whether the high scores come from personalization or just from the underlying TOD data. The GPT-4o/GPT-4 circularity is real but modest, given the human-model correlation of 0.84.\n\nWho is this for? Researchers working on personalized TOD, multimodal dialogue, or dataset distillation will find it useful. It is not a theoretical contribution, but it is a solid dataset-plus-baseline paper. With the emotion statistics fixed and the alignment step audited, it would be a reasonable addition to the literature. I would send it to peer review, but I would ask the authors to release the audit results and the corrected numbers as part of the revision.","headline":"First visual-persona TOD dataset with a mostly sound pipeline, but the emotion-alignment statistics are internally inconsistent and need an audit before the dataset premise is fully trusted.","tokens_in":22174,"tokens_out":1794,"would_cite":true,"duration_ms":20832,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"User images can serve as a task-oriented dialogue persona: PicPersona-TOD pairs photos with dialogues and rewrites both sides of the conversation to match the user's apparent age, formality, and emotion, and its trained model Pictor…","keywords":["task-oriented dialogue","personalization","visual persona","multimodal dialogue","LLM dataset distillation","dialogue natural language generation","retrieval-augmented generation","human evaluation"],"falsifier":"Re-annotate a random sample of the claimed image-dialogue pairs with independent human raters or a different emotion classifier: if the image's emotion and the dialogue's emotion match no more often than chance, or if the labels cannot be reproduced, then the alignment stage contributes no persona coherence and the dataset premise is unsupported. A simpler check is arithmetic: the stated distribution (50.92% positive, 52.44% neutral, 0.55% negative) sums to 103.91%, so the counts behind it must be re-derived before the pairing criterion can be trusted.","tokens_in":21217,"feed_emoji":"🖼️","tokens_out":9579,"duration_ms":85362,"temperature":0.7,"pith_summary":"PicPersona-TOD is a new dataset for task-oriented dialogue in which a user photo is part of the persona: each of 18,148 dialogues is paired with an image, and both user and system utterances are rewritten so that age, formality, greetings, and emotional tone match what the photo conveys. The paper's claim is that this visual-persona signal produces measurably more engaging system responses than generic or text-only personalization, without losing the underlying task information. Human raters scored the dataset close to 4 on a 4-point scale for style appropriateness, semantic consistency, and satisfaction, and preferred it over two text-persona baselines. A companion model, Pictor, trained on the dataset, keeps that personalization when tested in bus, movie, and home domains it never saw during training.","feed_headline":"User photos can make task-oriented chatbots reply in a personal style","feed_subtitle":"PicPersona-TOD pairs photos with dialogues; its Pictor model keeps task accuracy and personalizes unseen domains.","key_machinery":"The load-bearing mechanism is first-impression-guided prompting. For each image, an LLM writes a one-sentence first impression covering age, gender, emotion, outfit, and overall vibe; that impression is then inserted into prompts that restyle the user's utterances and generate personalized system utterances. The pipeline also aligns images to dialogues through shared emotion labels, filters dialogues by embedding-based personalization strength and direction vectors, and grounds recommendations in retrieved reviews and encyclopedia entries. Pictor reproduces this at inference time by taking the image, the generated impression, turn progress, dialogue policy, and database results as inputs to a vision-language model.","core_discovery":"The central discovery the paper argues for is that user images can carry the persona in task-oriented dialogue: a system that first forms a first impression from the image and then uses that impression, the dialogue policy, and retrieved facts to rewrite its utterances can tailor formality, vocabulary, greetings, and emotional awareness to the individual user. The paper builds PicPersona-TOD to demonstrate this, starting from MultiWOZ-2.2 and SGD dialogues, pairing them with FFHQ images by matching predicted emotion labels, restyling user utterances, personalizing system responses with first-impression prompts and retrieval-augmented recommendations, and filtering by personalization strength, direction, semantics, and naturalness. The demonstration includes human evaluations showing high preference over text-based personalization, DST and policy results comparable to MultiWOZ, and zero-shot personalization by Pictor in held-out domains. The intended conclusion is that a vision-based persona is a workable, information-preserving route to personalized task-oriented dialogue.","pith_inferences":["Because the reported emotion distribution sums to 103.91%, the emotion-alignment stage may not be doing what it appears to do; a clean test is to compare PicPersona-TOD built with emotion-matched pairs against a version built with randomly paired images and see whether downstream personalization scores change.","The style-transfer results might be driven mainly by the first-impression prompt rather than by the image itself; training Pictor with text-only impressions (no image input) would separate the contribution of raw pixels from the contribution of the inferred persona description.","The requirement of a camera for deployment could be relaxed by using images only as a training-time supervisor and letting the deployed model infer persona from ordinary descriptions, which would extend the personalization to text-only interactions.","The same automatic pipeline should transfer to other service domains or privacy-safe synthetic faces; the filtering thresholds would need recalibration, but the core first-impression and retrieval components are domain-agnostic."],"forward_implications":["If the central claim is right, a task-oriented dialogue system that can see its user can adjust formality, word difficulty, greetings, and emotional tone to the user's apparent age and mood.","Personalization does not have to come at the cost of task accuracy: DST joint-goal accuracy and policy entity-F1 on PicPersona-TOD stay close to MultiWOZ, the human-curated source dataset.","The visual signal generalizes: Pictor's zero-shot scores in bus, movie, and home domains are near 4 on a 4-point scale, suggesting the model learned a transferable mapping from image to response style rather than domain-specific templates.","An automated LLM pipeline with filtering can produce a large, multi-domain personalized TOD dataset (18,148 dialogues across 18 services) at far lower cost than human annotation.","Retrieval-grounded recommendation generation is a workable guard against hallucinated personalized suggestions, since the model is told to connect recommendations to real review and entry information."],"supporting_citations":[{"why":"Supplies the base MultiWOZ-2.2 task-oriented dialogues and the slot labels that the pipeline restyles into personalized form.","marker":"Zang et al. (2020)"},{"why":"Supplies the SGD dialogues that extend the dataset to more services and provide the held-out bus, movie, and home domains for the generalization test.","marker":"Rastogi et al. (2020)"},{"why":"Supplies the Flickr-Faces-HQ user images that serve as the visual persona for each dialogue.","marker":"Karras et al. (2019)"},{"why":"GPT-4o is the LLM that performs image-dialogue alignment, style transfer, response personalization, and filtering.","marker":"Achiam et al. (2023)"},{"why":"Sentence-BERT embeddings define the personalization strength and direction metrics used to filter the dataset.","marker":"Reimers (2019)"},{"why":"Retrieval-augmented generation grounds recommendation personalization in retrieved reviews and encyclopedia text to reduce hallucination.","marker":"Lewis et al. (2020)"},{"why":"Supplies the TinyLLaVA backbone for the smaller Pictor variant used in the ablation experiments.","marker":"Zhou et al. (2024)"},{"why":"Provides the PPTOD architecture on which the DST and policy baselines are trained.","marker":"Su et al. (2021)"}],"fun_headline_variants":["Images as persona: dataset for personalized task dialogues","Pictor: user photos customize chatbot style, zero-shot","Let chatbots see you: image-based persona for TOD","See the user, speak the style: image personas for task bots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The dataset's premise rests on the claim that two independently computed emotion labels—one for an image and one for a dialogue—are reliable enough that matching them produces coherent persona-dialogue pairs; the paper's reported emotion distribution sums to 103.91%, so that reliability is not demonstrated.","fun_headline_variants_meta":{"raw":{"variants":["Images as persona: dataset for personalized task dialogues","Pictor: user photos customize chatbot style, zero-shot","Let chatbots see you: image-based persona for TOD","See the user, speak the style: image personas for task bots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000255,"raw_usage":{"total_tokens":1547,"prompt_tokens":894,"completion_tokens":653,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":586}},"tokens_in":510,"tokens_out":653,"duration_ms":6811,"temperature":1.0,"reasoning_tokens":586,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:41:32.204394+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random sample of the claimed image-dialogue pairs with independent human raters or a different emotion classifier: if the image's emotion and the dialogue's emotion match no more often than chance, or if the labels cannot be reproduced, then the alignment stage contributes no persona coherence and the dataset premise is unsupported. A simpler check is arithmetic: the stated distribution (50.92% positive, 52.44% neutral, 0.55% negative) sums to 103.91%, so the counts behind it must be re-derived before the pairing criterion can be trusted.","supporting_citations":[],"review_version":1}