{"id":"f1157281-dcd5-40bf-98b7-4a98046cf61c","arxiv_id":"2504.18397","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"UV-CoT trains MLLMs to perform image-level chain-of-thought reasoning by optimizing preferences over model-generated bounding boxes, removing the need for human bounding-box annotations.","lead":"UV-CoT trains a vision-language model to reason about an image by first locating and cropping the relevant region, using preference data auto-generated by another model instead of human bounding-box labels. It reports gains on several VQA benchmarks, especially spatial and high-resolution reasoning, and better zero-shot transfer than a supervised visual chain-of-thought baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The preference signal from the evaluator MLLM (Eq. 1) is never validated against human judgments of region relevance, so the unsupervised training data could encode evaluator biases rather than true region utility.","rationale":"The reader's verdict is CONDITIONAL, focused on evaluator reliability and missing error bars. I agree that the evaluator preference signal in Eq. (1) is the least secure assumption in the pipeline. The paper provides only indirect evidence: the self-evaluated variant still improves over LLaVA-1.5-7B, and qualitative examples in Fig. 4 look reasonable. However, none of these checks compare the evaluator's judgments against human labeling of region relevance, and the bounding box evaluation in Fig. 3 uses another MLLM (GPT-4o) as scorer, creating a potential circularity. If the evaluator is biased toward text-rich or high-contrast crops, the preference data would corrupt training, and the gains on TextVQA/VSR might reflect exploiting those biases rather than learning general visual CoT. That said, the empirical results are reproducible (code released), the sDPO loss is a properly-cited DPO variant, and the zero-shot generalization is consistent with the method's intent. The abstract's overclaim about 'superiority' on six datasets is also a real issue, since UV-CoT (0% label) averages 0.473 vs 0.482 for Visual-CoT in Tab. 1; this should be corrected. The single most load-bearing concern, though, is the unvalidated preference signal, which is why I recommend keeping the CONDITIONAL verdict pending human validation of region preferences.","tokens_in":16719,"tokens_out":8007,"duration_ms":72735,"concrete_test":"Take a random sample of 200 image-question pairs from the training data, extract the preferred and dispreferred crops for each pair, and ask human annotators (e.g., via Mechanical Turk) to select which crop better supports answering the question. Compute the agreement (e.g., accuracy or Kendall's tau) between the evaluator MLLM's preference and the human majority. If agreement is not significantly above 0.7 (or whatever threshold), the preference data is unreliable and the reported gains should be re-attributed to the CoT inference mechanism, not to preference learning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (1) defines s_i = s_i^cur + γ s_i^nxt, where the evaluator MLLM scores responses generated from each candidate crop. The entire preference dataset D is built from these scores; if the evaluator assigns high scores to crops that contain answer text, high-contrast patches, or other superficial cues rather than the semantically relevant region, the DPO-style loss (Eq. 2) will train the model to produce such crops. The paper's only validation of the evaluator is indirect: Tab. 5 shows self-evaluation still beats the base model, and Fig. 4 shows a few qualitative examples. The bounding-box evaluation in Fig. 3 uses GPT-4o as a scorer, which does not break the circularity because it is also an MLLM. Critically, no human-annotated measure of region relevance is used, so we cannot distinguish whether UV-CoT learns to find key regions or learns to exploit evaluator biases. Since the central claim is that unsupervised preference data substitutes for human bounding-box annotation, the reliability of these preferences is the linchpin of the paper.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"UV-CoT proposes to train MLLMs for visual chain-of-thought reasoning without human bounding-box annotations. The target model samples candidate bounding boxes, uses each cropped region to answer the question, and an evaluator MLLM scores the resulting responses (Eq. 1); these scores define preference pairs. The target model is trained with Score-DPO (Eq. 2), which augments the DPO objective with a score-dependent margin, over four iterative rounds. Experiments compare against Visual-CoT-7B, LLaVA-1.5-{7B,13B}, MiniCPM-o-8B, and OmniLMM-12B on six VQA benchmarks, with additional zero-shot results on four unseen benchmarks. The reported results are that UV-CoT(10%) exceeds Visual-CoT-7B by 2.1% on average, while the fully unsupervised UV-CoT (0% label) averages 0.473 versus 0.482 for Visual-CoT-7B on the same six benchmarks.","tokens_in":16948,"tokens_out":7558,"duration_ms":72439,"significance":"If fully validated, the core idea of replacing human bounding-box annotations with model-generated preference data would be practically significant: it removes the main annotation bottleneck of Visual-CoT and could improve generalization to unseen tasks. The paper also has concrete strengths: the Gumbel-based derivation in Appendix B is mathematically correct and reduces to a shifted sigmoid objective, an ablation isolates the contribution of sDPO and iterative learning, and the authors release code. Nevertheless, the evidence for the headline claim is currently mixed: the fully unsupervised model does not beat the supervised baseline on average, and key design choices that the method depends on are underspecified. The central claim therefore needs either stronger evidence or more careful qualification.","major_comments":[{"comment":"The paper's central quantitative claim that UV-CoT outperforms the supervised Visual-CoT-7B is not supported by the fully unsupervised model: UV-CoT (0% label) averages 0.473 versus 0.482 for Visual-CoT-7B, a decrease of 0.9 points. The claimed 2.1-point average gain comes only from UV-CoT (10% label), which uses 10% of the labeled Visual-CoT data and is therefore not fully unsupervised. The abstract and introduction should be revised to state explicitly which variant achieves state-of-the-art performance and to avoid implying that pure unsupervised learning surpasses supervised visual CoT on average.","section":"§4.2, Table 1"},{"comment":"The validity of the entire preference dataset rests on the untested assumption that the evaluator MLLM's score s_i is a faithful proxy for the usefulness of the cropped region. No experiment in the paper compares the evaluator's region-quality judgments against human annotations or ground-truth region relevance. The bounding-box evaluation in Fig. 3 uses GPT-4o as the scorer, which is another MLLM and does not break the circularity; Table 5 and Fig. 4 provide only indirect evidence about final answers. Since the central claim is that automatically generated preferences substitute for human bounding-box annotations, this missing validation is load-bearing and should be addressed with either a human study of region relevance or a comparison of the preference ordering against ground-truth box utility on a subset.","section":"§3.1, Eq. (1)"},{"comment":"The function g(·) is central to sDPO but is never specified. The offset g(sw) − g(sl) determines the margin in the loss; without knowing g (for example, identity, log, or a scaled score), the reader cannot reproduce the method or interpret why sDPO outperforms naive DPO in Table 4. Please specify g and report any associated hyperparameters, along with the values of n, k, T, and γ used in Algorithm 1 and Eq. (1).","section":"§3.2, Eq. (2)"},{"comment":"All comparisons are reported as point estimates without variance, multiple seeds, or significance testing. Many differences are small (for example, 0.632 versus 0.652 on Flickr30k and 0.536 versus 0.546 on GQA), and the headline average gain of 2.1% could be within run-to-run noise. Please report standard deviations or bootstrap confidence intervals for the main comparisons, at least for the leading baseline Visual-CoT-7B and the two UV-CoT variants.","section":"Tables 1–3"},{"comment":"The zero-shot section is internally confusing about training exposure. Section 4.3 says the default UV-CoT is tested on SROIE, DUDE, Visual7W, and V∗ Bench 'without any training exposure to these datasets,' but Appendix C.1 lists Visual7W among the source datasets used for preference data, and Table 6 includes all ten datasets in one table. The paper should specify exactly which model variant was trained on which splits, and clarify that UV-CoT∗ is trained on unlabeled data from parts of the zero-shot datasets and is therefore not itself a zero-shot model.","section":"§4.3 and Appendix C.1"}],"minor_comments":[{"comment":"The text says 'nine source datasets' while Table 6 lists ten datasets including V∗ Bench; please reconcile the count.","section":"Appendix C.1"},{"comment":"The figure contains the typos 'Prefered' and 'Dis-prefered'; these should read 'Preferred' and 'Dis-preferred.'","section":"Figure 4"},{"comment":"The bullet 'UV-CoT outperforms the supervised Visual-CoT' should explicitly refer to UV-CoT(10%) rather than the 0%-label model, since Table 1 shows the latter is worse on average.","section":"§4.2"},{"comment":"Equation (10) is written with π∗ while the loss in Eq. (6) uses πθ; a sentence explaining the standard DPO parameterization step would improve readability.","section":"Appendix B, Eq. (10)"},{"comment":"Please report the number of random samples used to estimate the expectation in the definition of s_i^nxt, and the value of γ used in the main experiments.","section":"§3.1, Eq. (1)"},{"comment":"The phrase 'state-of-the-art' is stronger than Table 1 supports for the fully unsupervised 0%-label model; consider qualifying the claim with the 10%-label variant or reporting the 0%-label comparison more prominently.","section":"Abstract and §1"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and a correct technical derivation, but the main claim needs to be re-centered on the fully unsupervised model, which currently underperforms Visual-CoT-7B on average. The most consequential fix is the validation of the evaluator's region-quality judgments against human or ground-truth relevance; without that, the unsupervised preference signal is an assumption rather than a demonstrated result. The missing definition of g is easily fixable. If the authors add the proposed validation and adjust the claims, I would be willing to consider a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, honest engineering paper that deserves a proper referee. The new thing is the automatic two-stage pipeline — the target MLLM samples bounding boxes, an evaluator MLLM scores the resulting answers, and a DPO-style loss with a score offset (sDPO) turns those scores into training signal. The offset is correctly disclosed as an instance of DPO with an offset (ODPO). The Gumbel derivation in App. B is correct.\n\nWhat the paper does well: the experiments are broad, ten datasets, clean ablations (naive DPO, no iterative learning, no gamma all hurt), and they release code. The zero-shot generalization numbers are genuinely encouraging — UV-CoT beats Visual-CoT-7B by 2.5 points on average on unseen datasets, and the V* Bench gain is large.\n\nThe soft spots are mostly about framing. In Tab. 1 the fully unsupervised UV-CoT (0% label) averages 0.473 vs Visual-CoT-7B's 0.482. So the claim of 'superiority over state-of-the-art visual CoT' is only true for the variant that uses 10% of the labeled Visual-CoT data. The abstract overstates. That should be fixed.\n\nOn the evaluator concern: yes, the paper never validates the evaluator's region judgments against human ground truth. The stress-test note is right that GPT-4o box scoring in Fig. 3 is still an MLLM. But I don't think this is load-bearing: Tab. 5 shows self-evaluation (LLaVA-1.5-7B as its own evaluator) still lifts the model 4.8 points over the base, and applying UV-CoT-style boxes at inference improves even the untrained OmniLMM. If the preferences were mostly garbage, you wouldn't expect that consistency. Still, a small human-annotated region-relevance check would settle it cheaply.\n\nMinor: no error bars anywhere, and the paper says 'state-of-the-art' when the strict table reading doesn't support that.\n\nBottom line: it's a real contribution, not a revolution. The data pipeline and the empirical pattern are worth a close read. I'd send it to review and ask for reframing, error bars, and a human-validated region-relevance spot check.","headline":"A useful label-free pipeline for visual CoT, but the headline 'unsupervised superiority' only holds after adding 10% of the supervised data; the evaluator-reliability concern is real but not fatal.","tokens_in":17502,"tokens_out":2542,"would_cite":true,"duration_ms":24388,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UV-CoT replaces human-annotated bounding boxes with AI-ranked preferences, letting a multimodal model learn which image region to zoom into, and reports gains over supervised visual chain-of-thought on six benchmarks and four unseen…","keywords":["visual chain-of-thought","preference optimization","multimodal large language models","bounding-box localization","visual question answering","zero-shot generalization","Score-DPO","region selection"],"falsifier":"Take a random sample of the preference pairs generated by Algorithm 1 and have humans judge which crop is more useful for answering each question. If the evaluator's preferred crop agrees with the human choice at no better than chance on pairs where the model answered correctly, the preference signal is not learning useful visual grounding; alternatively, train UV-CoT on the same pairs but with the evaluator scores replaced by random scores and check whether the reported gains vanish.","tokens_in":16500,"feed_emoji":"🔍","tokens_out":9211,"duration_ms":85949,"temperature":0.7,"pith_summary":"UV-CoT is an attempt to train image-level chain-of-thought reasoning without any human bounding-box annotations. The target multimodal large language model (MLLM) proposes several candidate crops for a question, answers using each crop, and an evaluator model scores the answers; these scores become preference pairs that train the target model through a modified direct-preference-optimization loss called Score-DPO, which uses the score gap as an extra decision margin. The paper reports that this unsupervised recipe outperforms the supervised Visual-CoT-7B on TextVQA and VSR with 0% labels, and with only 10% labeled data beats it by 2.1 percentage points on average across six benchmarks. On four unseen datasets, zero-shot UV-CoT is reported to outperform all baselines, with the largest gains on high-resolution reasoning, suggesting the ranking formulation generalizes better than supervised imitation.","feed_headline":"No-label visual chain-of-thought beats supervised boxes on unseen data","feed_subtitle":"UV-CoT turns crop selection into AI-ranked preference pairs, beating supervised Visual-CoT with zero human labels.","key_machinery":"The load-bearing object is the automatic preference-data pipeline plus the Score-DPO loss. The pipeline turns a single image-question pair into pairwise training data: stochastic decoding of the target model gives $n$ seed boxes, each box is used to answer the question, and an evaluator model scores each candidate with a current score plus a discounted next-step score, $s_i = s_i^{\\mathrm{cur}} + \\gamma s_i^{\\mathrm{nxt}}$. This scoring design is what lets the model judge a box by whether it helps reasoning continue, not merely by whether it contains the answer text. Score-DPO then carries the signal into training by inserting the mapped score gap $g(s_w)-g(s_l)$ as an offset inside the sigmoid of the standard DPO objective; the paper derives this offset from Gumbel-distributed rewards, so the loss explicitly optimizes both the ordering and the magnitude of preference. Iterative learning regenerates preference data with the updated model over four rounds, keeping the data distribution aligned with the evolving policy.","core_discovery":"The paper's central claim is that the key-region selection step in visual chain-of-thought reasoning can be learned from relative comparisons instead of absolute labels. Starting from a question, the target multimodal model samples several bounding boxes, crops each one, and answers; an evaluator scores each response and a look-ahead estimate, $s_i = s_i^{\\mathrm{cur}} + \\gamma s_i^{\\mathrm{nxt}}$, so the score reflects not just whether the answer came out right but whether the crop helps the next step. The preferred and dispreferred chains are then used to train the target model with Score-DPO, $\\sigma(\\beta \\log \\frac{\\pi_\\theta(y_w|x)}{\\pi_{\\mathrm{ref}}(y_w|x)} - \\beta \\log \\frac{\\pi_\\theta(y_l|x)}{\\pi_{\\mathrm{ref}}(y_l|x)} - (g(s_w)-g(s_l)))$, which is DPO with a Gumbel-motivated offset equal to the mapped score difference. The paper claims this produces bounding boxes that are more useful for downstream answers than those from supervised Visual-CoT, makes the model beat a larger evaluator model on average, and generalizes to unseen datasets without retraining.","pith_inferences":["If the evaluator's preferences are the active ingredient, then improving the evaluator's look-ahead quality should directly improve region selection; a cheap test is to vary $\\gamma$ or the number of sampled next responses and watch whether downstream accuracy tracks the resulting score quality.","The ranking formulation is essentially weakly supervised visual grounding: the model is never told what a useful region looks like, only which of two crops led to a better answer. That suggests UV-CoT could be repurposed as a general-purpose weak localizer for tasks beyond VQA, such as referring expression comprehension or visual grounding.","Because UV-CoT beats its own evaluator on average, the gap between teacher and student may be a general property of preference optimization rather than a quirk of this setup; if so, iterative self-evaluation might push performance past the initial evaluator's ceiling without ever introducing human labels."],"forward_implications":["Bounding-box annotation is no longer a prerequisite for visual CoT: preference pairs can be generated automatically, so the same recipe can be applied to new datasets or domains without human labeling.","Zero-shot transfer improves because the model learns a relative ranking of regions rather than memorizing annotation patterns; the paper reports it outperforms all baselines on four unseen datasets.","A small amount of supervised data can still be used: UV-CoT trained with 10% of Visual-CoT's labels outperforms the fully supervised Visual-CoT-7B by 2.1 percentage points on average across six benchmarks.","The learned region-selection behavior transfers at inference time: adding UV-CoT-generated bounding boxes to models that were not fine-tuned for CoT improves their scores, so the box-producing step itself is a reusable capability."],"supporting_citations":[{"why":"provides the supervised Visual-CoT baseline and labeled dataset that UV-CoT is compared against and whose 10% split is used for the UV-CoT (10%) variant.","marker":"[35]"},{"why":"supplies the DPO objective that Score-DPO modifies by inserting the preference-score gap in the logit margin.","marker":"[33]"},{"why":"motivates using AI-generated feedback and the iterative learning loop that regenerates preference data over rounds.","marker":"[47]"},{"why":"gives the offset-DPO derivation with Gumbel-distributed rewards that the paper adapts to obtain the score-offset in Score-DPO.","marker":"[1]"},{"why":"defines the target multimodal model LLaVA-1.5-7B whose stochastic decoding generates the seed bounding boxes and answers.","marker":"[21]"},{"why":"is the GPT-4o-based automatic scorer used to evaluate model answers on the ten benchmarks.","marker":"[28]"}],"fun_headline_variants":["Preference-based visual CoT beats supervised boxes, no labels","Visual CoT from AI rankings, no bounding-box labels","Unsupervised visual CoT: preference beats labeling for boxes","Score-DPO visual CoT: ranking regions beats supervised SFT","Zero-shot visual CoT from AI preferences, no human labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the evaluator model's score for a candidate crop, computed as the quality of the answer that crop enables plus a look-ahead term, is a faithful measure of whether that crop is the right region to focus on; if the evaluator instead prefers crops that merely contain the answer text or other superficial cues, the preference data used for training is corrupted.","fun_headline_variants_meta":{"raw":{"variants":["Preference-based visual CoT beats supervised boxes, no labels","Visual CoT from AI rankings, no bounding-box labels","Unsupervised visual CoT: preference beats labeling for boxes","Score-DPO visual CoT: ranking regions beats supervised SFT","Zero-shot visual CoT from AI preferences, no human labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000872,"raw_usage":{"total_tokens":3859,"prompt_tokens":1116,"completion_tokens":2743,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":732,"completion_tokens_details":{"reasoning_tokens":2657}},"tokens_in":732,"tokens_out":2743,"duration_ms":18848,"temperature":1.0,"reasoning_tokens":2657,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:16:40.145475+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of the preference pairs generated by Algorithm 1 and have humans judge which crop is more useful for answering each question. If the evaluator's preferred crop agrees with the human choice at no better than chance on pairs where the model answered correctly, the preference signal is not learning useful visual grounding; alternatively, train UV-CoT on the same pairs but with the evaluator scores replaced by random scores and check whether the reported gains vanish.","supporting_citations":[{"cited_title":"Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning","cited_arxiv_id":null,"evidence_quote":"provides the supervised Visual-CoT baseline and labeled dataset that UV-CoT is compared against and whose 10% split is used for the UV-CoT (10%) variant."},{"cited_title":"Direct preference optimization: Your language model is secretly a reward model","cited_arxiv_id":null,"evidence_quote":"supplies the DPO objective that Score-DPO modifies by inserting the preference-score gap in the logit margin."},{"cited_title":"Chatgpt, 2023","cited_arxiv_id":null,"evidence_quote":"is the GPT-4o-based automatic scorer used to evaluate model answers on the ten benchmarks."}],"review_version":1}