{"id":"c3e4958d-6fbb-49e6-ace0-a5553f83a356","arxiv_id":"2505.20777","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"TACO couples thinking with final answers, filters unstable training samples, reweights easy or hard samples, and adds multi-scale test inference, improving LVLM visual reasoning accuracy over VLM-R1.","lead":"The paper introduces TACO, a reinforcement learning recipe that trains vision-language models to reason before answering and to keep the two consistent. It reports accuracy gains on referring-expression and visual-question-answering benchmarks over the VLM-R1 baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TTME selection rule is underspecified and its reported gains are internally inconsistent; the largest OOD improvements depend on it.","rationale":"The reader's weakest_assumption is that TTME's selection rule is underspecified and depends on ground-truth information. My stress-test confirms this and adds a new internal inconsistency: Table 1 reports 70.81 for TTME on LISA whereas Table 3 reports 75.1 for the same configuration, so even the magnitude of the largest OOD gain is not stable within the manuscript. This is load-bearing because the central claim in Section 4.2 highlights the 75.1 LISA and 78.7 RefGTA numbers, and the ablations in Table 7 attribute most of the LISA improvement to TTRS/TTME. If TTME is removed or its selection rule fails, the strongest claim shrinks to the non-TTME results (66.5 and 74.9), which are still above VLM-R1 but materially smaller. The training-side components (TAC, RRS, ADS) are described in enough detail to be plausible, and the ID gains over VLM-R1 (+1.7% average) do not depend on TTME, so the appropriate verdict remains CONDITIONAL rather than REJECT or ACCEPT. The test I propose would settle the ambiguity by empirically comparing the literal least-overlap rule, oracle selection, and a simple majority vote on the same benchmarks, and by fixing the Table 1/Table 3 discrepancy. This does not change the reader's verdict but reinforces it with a concrete check.","tokens_in":13134,"tokens_out":3993,"duration_ms":42698,"concrete_test":"Run TTME on LISA and RefGTA with β=3 under four selection rules: (a) oracle selection using ground-truth IoU, (b) the paper's literal 'least IoU overlap' rule computed among the β predicted boxes, (c) majority voting among the β boxes, and (d) single-scale TTRS at 672px. Report all four accuracies, with code or detailed pseudocode, and re-evaluate the exact 'w/ TTME' configuration to reconcile Table 1 (70.81) with Table 3 (75.1). If literal least-overlap without ground truth matches oracle within 1 point and reproduces 75.1, the concern is resolved; otherwise the headline OOD claim should be stated without TTME.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest empirical claim, Section 4.2, rests substantially on TTME: Table 3 reports that TTME raises LISA from 66.5 to 75.1 and RefGTA from 74.9 to 78.7, and Table 7 shows TTRS/TTME contributes the bulk of the LISA gain. However, Section 3.5 describes TTME as follows: 'calculates β accuracy rewards through the model, and selects the answer with the least number of intersections as the final answer... For REC, we calculate the answer with the least IoU overlap.' At inference there is no ground truth against which to compute an accuracy reward, and 'least IoU overlap' has no well-defined reference: IoU requires two boxes, and among the β multi-scale predictions there is no stated anchor. If the intended rule is pairwise overlap among the β boxes, the claim that the least-overlap box is the correct one is asserted without justification and is not derivable from the described algorithm. There is also an internal inconsistency: Table 1 reports 'w/ TTME' at 70.81 on LISA, while Table 3 reports 75.1 for the same 'Ours(w TTME)' configuration, an unexplained 4.3-point gap. Consequently, the headline OOD numbers are not reproducible from the manuscript as written. The non-TTME TACO results (66.5 LISA, 74.9 RefGTA) are still positive over VLM-R1, so the central claim should be weakened to those numbers, not the TTME-boosted ones, until TTME is specified and the table conflict is resolved.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TACO, an RL fine-tuning method for LVLMs built on GRPO. It introduces a Think-Answer Consistency reward that couples the model's reasoning trace with its final answer and ground truth; a Rollback Resample Strategy that masks high-KL 'dirty' samples and down-weights their sampling rate; an Adaptive Difficulty Sampling schedule that resamples moderate-difficulty examples more often; and a Test-Time Resolution Scaling scheme plus a multi-scale ensemble (TTME) to bridge training/test resolution gaps. Using Qwen2.5-VL-3B, the authors report gains over VLM-R1 and the base model on REC benchmarks (RefCOCO/+/g, LISA, RefGTA) and several VQA benchmarks, with the largest out-of-domain gains attributed to TTME.","tokens_in":13470,"tokens_out":6573,"duration_ms":65492,"significance":"If the results hold, the paper offers a useful recipe for stabilizing long-chain RL in LVLMs and improving data efficiency; the component-level ablations and training-step curves are valuable. The paper is honest about its limitations and does not oversell the ID gains. However, the central empirical claims are currently weakened by an underspecified and internally inconsistent TTME description, a lack of seed-level variance, and unvalidated hand-set hyperparameters. The non-TTME results still show positive gains over VLM-R1, so the core direction is promising but needs substantial revision before the headline numbers can be trusted.","major_comments":[{"comment":"Section 3.5 describes TTME as 'calculates β accuracy rewards through the model' and 'selects the answer with the least number of intersections,' with 'least IoU overlap' for REC. At inference there is no ground truth, so an 'accuracy reward' cannot be computed, and IoU requires an anchor box; the selection rule is therefore not implementable as written. This is load-bearing because the headline OOD numbers (75.1 on LISA, 78.7 on RefGTA in Table 3) and the largest ablation gain in Table 7 (66.5 to 75.1) depend on TTME. In addition, Table 1 reports 'w/ TTME' as 70.81 on LISA, while Table 3 reports 75.1 for 'Ours(w TTME)' and Table 7's final row gives 75.1; these numbers are mutually inconsistent. Please specify the exact reference-free selection rule, justify why the least-overlap prediction is the correct one, clarify whether the ablated 'TTRS' includes TTME or only single-scale TTRS, and resolve the table conflict.","section":"3.5, Tables 1, 3, 7"},{"comment":"The paper reports no random seeds, no variances, and no error bars; Table 2's caption says 'The best performance is reported here for each method.' Since GRPO training is stochastic and the claimed margins are small (e.g., +1.7% average over VLM-R1 in Table 2, +5.34 on LISA at 800 steps in Table 4), the results could be within run-to-run noise. Please provide means and standard deviations over at least three independent runs for the main comparisons, or release the exact training scripts and checkpoints so the results can be reproduced.","section":"4.2, Tables 2-7"},{"comment":"All of the new mechanisms rely on hand-set thresholds and coefficients (κ=0.5, γ=0.8, θH=0.5, θL=0.2, αeasy=0.1, αhard=0.8, αmoderate=1.5) with no sensitivity analysis. The KL-based dirty-sample criterion in Eq. (5) is particularly fragile: the KL magnitude depends on training dynamics, and because RRS masks these samples, the reported KL stabilization in Figure 3 is partly by construction. Please include sensitivity sweeps for these hyperparameters and an evaluation of RRS that reports accuracy and KL on samples that are not masked, so the effect of the gradient mask is separated from the effect of the sampling-rate update.","section":"3.3-3.4, Eqs. (5)-(7)"}],"minor_comments":[{"comment":"The offline curation step says samples are 'randomly combined in a 1:2 ratio' but does not state the final training-set size or whether the 1:2 ratio is difficult-to-simple; please clarify.","section":"3.4"},{"comment":"The supervisor prompt for VQA mentions 'given corpus' while the equation inputs are Q, T, and GT; please align the prompt text with the notation and explain what the supervisor actually receives.","section":"3.2, Eq. (4)"},{"comment":"The SFT comparison lacks training details (epochs, learning rate, data mixture); without these, the SFT-vs-RL comparison is difficult to interpret.","section":"Table 4"},{"comment":"The caption ends with the incomplete sentence 'reasoning time remains nearly.'; please complete it.","section":"Figure 2"},{"comment":"Reference [36] is cited for the 'R1-Vision collection' but the entry names R1-Onevision; please clarify whether these are the same resource or different datasets.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The TTME inconsistency between Table 1 (70.81) and Tables 3/7 (75.1) is a red flag that should be resolved before publication; please ask the authors to provide the exact inference code and clarify how TTME selects answers. The paper would also benefit from releasing checkpoints and training code, since the central claims are empirical and no reproducibility artifacts are mentioned."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TACO's training-side recipe is real and worth reading. The triple-IoU think-answer consistency reward for REC is a clean idea, the rollback resampler and adaptive difficulty sampling are plausible and individually ablated, and the training is remarkably data-efficient (1,000 steps, 6 samples per step, 1.875% of the RefCOCO descriptions). The reported gains over VLM-R1 on in-domain REC and on VQA benchmarks are consistent across tasks. That is the core contribution.\n\nThe soft spot is the test-time multi-scale ensemble (TTME). As written, Section 3.5 says the method 'calculates β accuracy rewards through the model' and selects the answer with the least number of intersections. At inference there is no ground truth to compute accuracy rewards against, and 'least IoU overlap' needs a reference among the β predicted boxes that the paper never specifies. The stress-test note is right: without a stated rule, the headline OOD numbers are not reproducible. There is also an internal inconsistency: Table 1 reports w/ TTME at 70.81 on LISA, while Table 3 reports 75.1 for the same configuration, an unexplained 4.3-point gap. The largest OOD gain (LISA 66.5 to 75.1) comes entirely from TTME, so this is not a minor detail.\n\nEverything else is in proportion. No seeds or error bars appear anywhere; all hyperparameters (κ, γ, θH/θL, αs, β, Starget) are hand-set with no sensitivity analysis. The RRS has a mild self-confirmation flavor – it defines dirty samples by KL divergence and then masks them, so KL stability is partly a construction – but that is a property of the design, not a logical failure.\n\nThe training-side results without TTME (66.5 LISA, 74.9 RefGTA) are still positive over VLM-R1 and the ablation in Table 7 shows each component helps. So the central empirical claim survives if you strip out TTME. My advice: send it to peer review, but require the authors to (1) specify a computable test-time selection rule, (2) resolve the Table 1 vs Table 3 conflict, and (3) report multiple seeds. The paper is worth a serious referee.","headline":"Credible RL training recipe for LVLMs with consistent gains, but the headline OOD numbers rest on an under-specified test-time ensemble that needs clarification before the numbers can be trusted.","tokens_in":14071,"tokens_out":3000,"would_cite":true,"duration_ms":27555,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tying RL rewards to reasoning lifts OOD vision-language scores by 20","keywords":["reinforcement learning","vision-language models","visual reasoning","referring expression comprehension","visual question answering","GRPO","test-time resolution scaling","think-answer consistency"],"falsifier":"On a labeled out-of-domain set, run TACO's three-scale ensemble and compare the least-overlap selection against single-scale inference at 672px, oracle selection of the scale with highest IoU, and majority vote. If the least-overlap rule is not substantially better than majority vote and is much worse than oracle, the TTME component is not selecting correctness; the same comparison on LISA and RefGTA would settle whether the 75.1% and 78.7% figures are attributable to the selection rule or to the multi-scale computation itself.","tokens_in":12867,"feed_emoji":"🎯","tokens_out":6769,"duration_ms":63685,"temperature":0.7,"pith_summary":"This paper claims that vision-language models can be trained with reinforcement learning to reason first and answer second, provided the reward checks both steps against the ground truth. The proposed algorithm, TACO, couples the model's thinking output and its final box or text answer into a single consistency-aware reward, and adds three stabilizers: it rolls back samples whose policy drift is too large, re-weights sampling toward moderate-difficulty examples, and rescales test images across multiple resolutions. On referring-expression comprehension the method reports gains of about four points over the base model in-domain and roughly twenty points on an out-of-domain reasoning-grounding benchmark, with smaller but consistent gains on a suite of VQA tasks. The paper's point is that long-chain reasoning in multimodal models is learnable and stable if the reward architecture enforces a tight think-answer link.","feed_headline":"Tying RL rewards to reasoning lifts OOD vision-language scores by 20","feed_subtitle":"TACO makes thought and final box answer share one reward, stabilizing RL and sharpening grounding.","key_machinery":"The load-bearing mechanism is the Think-Answer Consistency reward, a reward that measures agreement among the model's reasoning box, its final answer box, and the ground-truth box via three-way IoU, and, in VQA, via a supervisor-assessed semantic consistency score. Around this reward sit three stabilizers: Rollback Resample Strategy, which computes per-sample KL divergence between current and reference policies and masks gradients of samples above a threshold while down-weighting their sampling probability; Adaptive Difficulty Sampling, which raises the sampling weight of medium-accuracy samples and cuts easy and hard ones; and Test-Time Resolution Scaling, which resizes the short side of test images to a fixed length and, in ensemble mode, selects the least-overlapping prediction among three scales.","core_discovery":"The central discovery is that the instability and poor generalization of R1-style reinforcement learning in vision-language models can be traced to a mismatch between the chain of thought and the final answer, and that making the reward a joint function of thought, answer, and ground truth removes the incentive to produce either a long irrelevant chain or a correct-looking answer without reasoning. For grounding tasks this joint reward is realized as a three-way intersection-over-union among the box mentioned in the thinking text, the box in the final answer, and the ground-truth box; for open-ended VQA it is an external supervisor score of the thought-answer pair against the reference. The paper further argues that long-chain exploration collapses not from length itself but from temporary 'dirty' samples whose policy-reference KL divergence spikes, and that masking their gradients while lowering their resampling probability keeps training stable. A test-time multi-scale ensemble that picks the prediction with least overlap across scales is presented as bridging the resolution gap between compressed training images and varied test images.","pith_inferences":["Editorial inference: if the three-way IoU is the active ingredient, the recipe should transfer to any task whose output can be decoded into two comparable structured predictions, such as segmentation masks or keypoint sets, where the same thought-versus-answer-versus-ground-truth overlap can be computed.","Editorial inference: the least-overlap selection rule in Test-Time Multi-Scale Ensemble is a strong prior that can be tested directly against an oracle that sees ground truth; if the oracle disagrees frequently, a learned scale aggregator or simple majority vote would be a safer default.","Editorial inference: the KL-threshold dirty-sample detector suggests that long-chain collapse in RL may be a policy-drift phenomenon rather than a reward-hacking one, which would make the same rollback principle applicable to other RL-trained sequence models."],"forward_implications":["An R1-style GRPO run with TACO is stable for 1,000 steps on a 3-billion-parameter vision-language model without the response-length collapse seen in the baseline.","The gains transfer out of domain: the reported LISA and RefGTA accuracy improvements are larger than the in-domain RefCOCO/+/g gains, suggesting the consistency reward teaches a generalizable answer-grounding habit rather than dataset memorization.","Data efficiency improves: training uses only about 1.875% of the available region descriptions, and the adaptive schedule concentrates repeated learning on moderate-difficulty samples.","The same single model improves a broad set of VQA benchmarks, including math, charts, diagrams, and OCR, while keeping document VQA essentially flat, indicating the method does not sacrifice general visual QA for grounding."],"supporting_citations":[{"why":"Supplies GRPO, the policy-optimization loop that TACO modifies.","marker":"[14]"},{"why":"Defines the R1-style visual RL baseline and the four failure modes TACO targets.","marker":"[18]"},{"why":"The base vision-language model that is fine-tuned; its numbers anchor the reported gains.","marker":"[5]"},{"why":"Provide the RefCOCO/+/g training and in-domain evaluation data used in the REC experiments.","marker":"[32, 33]"},{"why":"RefGTA benchmark used to measure out-of-domain grounding generalization.","marker":"[34]"},{"why":"LISA benchmark where TACO reports its largest out-of-domain gains.","marker":"[35]"},{"why":"Source of the VQA training mixture used in the 9,600-sample VQA schedule.","marker":"[36]"}],"fun_headline_variants":["Think-answer consistency reward stabilizes visual RL","Rollback resampling tames long-chain RL in LVLMs","TACO lifts OOD vision-language scores by 20%","Three-way IoU links reasoning to final answer in vision","TACO: tying thought and answer rewards boosts vision RL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline out-of-distribution gains rest on the assumption that, at test time, the model's prediction with the least overlap across image scales is the correct one; without ground truth to verify that rule, the large LISA and RefGTA improvements shrink to the smaller single-scale numbers.","fun_headline_variants_meta":{"raw":{"variants":["Think-answer consistency reward stabilizes visual RL","Rollback resampling tames long-chain RL in LVLMs","TACO lifts OOD vision-language scores by 20%","Three-way IoU links reasoning to final answer in vision","TACO: tying thought and answer rewards boosts vision RL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000377,"raw_usage":{"total_tokens":2015,"prompt_tokens":960,"completion_tokens":1055,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":971}},"tokens_in":576,"tokens_out":1055,"duration_ms":10503,"temperature":1.0,"reasoning_tokens":971,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:47:10.179061+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a labeled out-of-domain set, run TACO's three-scale ensemble and compare the least-overlap selection against single-scale inference at 672px, oracle selection of the scale with highest IoU, and majority vote. If the least-overlap rule is not substantially better than majority vote and is much worse than oracle, the TTME component is not selecting correctness; the same comparison on LISA and RefGTA would settle whether the 75.1% and 78.7% figures are attributable to the selection rule or to the multi-scale computation itself.","supporting_citations":[{"cited_title":"Generating easy-to-understand referring expressions for target identifications","cited_arxiv_id":null,"evidence_quote":"RefGTA benchmark used to measure out-of-domain grounding generalization."},{"cited_title":"R1-onevision: A unified benchmark for vision-language reasoning and generation, June","cited_arxiv_id":null,"evidence_quote":"Source of the VQA training mixture used in the 9,600-sample VQA schedule."}],"review_version":1}