{"id":"d9aaa678-6508-49f1-9f7b-3bcaceb99fd8","arxiv_id":"2602.14147","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A unified SFT-plus-RL post-training recipe with answer-forcing, tree search, and complementary-mask likelihood estimation improves a multimodal diffusion LLM on visual math, QA, grounding, and editing benchmarks.","lead":"This paper introduces LaViDa-R1, a training recipe that combines supervised fine-tuning with reinforcement learning to make an image-and-text diffusion language model reason across visual math, visual QA, object grounding, and image editing. It reports gains over its own base model on several benchmarks, with the largest on reason-intensive grounding.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix B.1 lists GSM8K and MATH as RL training data while Table 1 reports GSM8K/MATH-500 accuracy; absent a train/test decontamination statement, the largest text-only gains may reflect memorization.","rationale":"The reader's weakest assumption—benchmark contamination—is correct, but the most direct instance is even sharper: Appendix B.1 explicitly puts GSM8K and MATH in the RL training data while Table 1 reports performance on GSM8K and MATH-500. Since MATH-500 is a subset of MATH, and no train/test split or decontamination step is stated, the largest reported text-only gains may simply reflect training on test answers. The grounding and editing overlaps (ReasonSeg/Lisa-CoT vs Lisa-Grounding; EditScore-RL/GPT-Edit/UniWorld vs ImgEdit) compound the concern. Other potential issues, such as the heuristic complementary-masking likelihood estimator or missing error bars, are secondary: regardless of the method's internal merits, the benchmark evidence must first be decontaminated. The concrete split audit is feasible and decisive. If no overlap is found, the concern is resolved; if overlap is found, the central generalization claims are unsupported. Since the reader already issued a CONDITIONAL verdict conditioned on decontamination, this read does not change the verdict.","tokens_in":30988,"tokens_out":13348,"duration_ms":137610,"concrete_test":"Ask for the exact data manifests of Stage-2 item J and check whether the GSM8K and MATH files contain the same question instances as the GSM8K test split and MATH-500 (e.g., exact prompt match after normalization); also run perceptual-hash near-duplicate checks between ReasonSeg/Lisa-CoT and Lisa-Grounding images and between EditScore-RL/GPT-Edit-1.5M/UniWorld and ImgEdit images. If any eval instance appears in training, recompute all tables excluding those tasks or with holdout splits.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Direct train/eval overlap: Stage-2 RL data (Appendix B.1, item J) explicitly includes GSM8K and MATH (Lightman et al., 2023), and Table 1 evaluates on GSM8K and MATH-500. MATH-500 is a subset of the MATH test set; if the \"MATH\" training data is the full dataset, the reported MATH-500 jump (23.4 to 38.6; +7.6 over SFT) and GSM8K jump (47.4 to 81.5; +10.9 over SFT) are not held-out numbers. The paper never states it used only train splits or that benchmark instances were removed. The same issue appears in grounding/editing: ReasonSeg and Lisa-CoT (H/L) vs Lisa-Grounding, and EditScore-RL/GPT-Edit-1.5M/UniWorld (D/K) vs ImgEdit. The GSM8K/MATH case is the cleanest because the dataset names match the eval names exactly. This is a benchmark-validity concern, not a claim of intentional misconduct; a split audit settles it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"LaViDa-R1 proposes a unified post-training recipe for multimodal discrete diffusion language models. It combines SFT, online GRPO, and best-of-N self-distillation into a single weighted policy-gradient objective (Eq. 5), and introduces answer-forcing, tree search, and a complementary-masking likelihood estimator with w(t)=1. The paper reports consistent improvements over the LaViDa-O base and an SFT baseline on visual math reasoning, VQA, language-only math, reason-intensive grounding, and image editing. The training pipeline and ablations are described in detail.","tokens_in":31350,"tokens_out":5971,"duration_ms":54256,"significance":"If the empirical claims hold, this is a useful step toward general-purpose reasoning for diffusion-based unified multimodal models. The unification of SFT, GRPO, and self-distillation into one objective is clean and the Appendix A.2 derivation is a strength; the pseudocode for answer-forcing and tree search is concrete; and the component ablations are informative. The paper also honestly documents limitations in Appendix C. However, the empirical validation is currently undermined by likely training/evaluation overlap for several benchmarks and by the absence of uncertainty estimates on small gains. The central contribution is defensible but needs substantial validation work before the claims can be accepted.","major_comments":[{"comment":"The listed training data overlap evaluation benchmarks. Stage-2 RL item J includes GSM8K and MATH; Table 1 reports GSM8K and MATH-500, and MATH-500 is a subset of the MATH test set. Unless the authors explicitly used only train splits and removed any evaluation instances, the largest reported gains (+10.9 GSM8K, +7.6 MATH-500 over SFT) are not held-out numbers. Similarly, items H/L include ReasonSeg and Lisa-CoT while the paper evaluates on Lisa-Grounding, and items D/K include GPT-Edit-1.5M, UniWorld-V1, and EditScore-RL while Table 2 reports ImgEdit. No decontamination statement appears anywhere in the manuscript. A detailed split audit, and re-evaluation if needed, is required before the empirical claims can be assessed.","section":"Appendix B.1 with Tables 1–3"},{"comment":"Several gains over the SFT baseline are small enough to be benchmark noise: ChartQA +0.9, MMMU-Pro +0.9, and ImgEdit +0.1. No error bars, multiple seeds, or significance tests are reported anywhere in the paper. The claim of 'strong performance' on these tasks is therefore not yet substantiated; the authors should report variance or at least a small number of repeated runs.","section":"Tables 1 and 2"},{"comment":"The unification proof assumes pure on-policy GRPO with β=0 and inactive clipping. In the actual algorithm, answer-forced samples and tree-search restarts are generated from non-π_old initial states, and self-distillation selects the best-of-N sample after reward evaluation; all are then assigned advantages in the same objective. No importance weighting or off-policy correction is provided. Eq. 5 is therefore not a valid policy-gradient estimator for the mixed training distribution. This may still work as a heuristic, but the theoretical framing overclaims the equivalence.","section":"Section 3.1, Eq. 5; Appendix A.2"},{"comment":"The complementary-masking likelihood estimator sets w(t)=1 instead of the standard 1/t weighting of the ELBO for masked diffusion. The paper does not show that the resulting expression remains a lower bound or that its bias is controlled. Since this estimator supplies log πθ(y|x) inside Eq. 5, any bias propagates into the RL update. The ablation in Table 6 shows an empirical benefit, but the 'estimator' terminology needs a consistency/bias analysis or a clear statement that this is a heuristic weighting.","section":"Section 3.3; Appendix A.5"}],"minor_comments":[{"comment":"Typo: 'Likelyhood' should be 'Likelihood'.","section":"Table 6 title"},{"comment":"Typo: 'adapting the GPRO algorithm' should be 'GRPO'.","section":"Appendix D"},{"comment":"'UniGPRO' should be 'UniGRPO'.","section":"Fig. 6 label"},{"comment":"UniWorld-V1 is cited as (Hu et al., 2022), but the reference list and Table 2 identify it as (Lin et al., 2025). Please correct the citation.","section":"Appendix B.1 item D"},{"comment":"'underwriting 56 steps each' should likely be 'requiring 56 steps each' or 'running 56 steps each'.","section":"Section 5, Tree Search paragraph"},{"comment":"The paper consistently spells 'DeepScalar' while the reference is to (Luo et al., 2025a), which is 'DeepScaler'. Please correct for consistency.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central idea is worth pursuing, but the contamination issue is the most serious problem. If the split audit confirms that evaluation instances were trained on, the paper would need to be rejected; I am not recommending rejection now only because a decontaminated rerun is within scope. The off-policy and likelihood-estimator concerns also need to be addressed in a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the takeaway: LaViDa-R1 has a real, practical contribution — a unified post-training recipe for multimodal diffusion LMs that combines SFT, GRPO, and best-of-N distillation into one weighted policy-gradient objective, with answer-forcing and tree search to generate high-quality rollouts. The complementary-masking likelihood estimator with w=1 is a simple fix that appears to help. The ablations are honest and the paper is clearly written. But the empirical evidence has a serious hole: Appendix B.1 lists GSM8K and MATH as Stage-2 RL data, and Table 1 evaluates on GSM8K and MATH-500, with no statement that those instances were removed or split by train/test. MATH-500 is a subset of the MATH test set. If the training data was the full dataset, the reported jumps (81.5 GSM8K, 38.6 MATH-500) are not held-out numbers. The same issue appears in grounding and editing: ReasonSeg and Lisa-CoT are in training while Lisa-Grounding is evaluated, and GPT-Edit-1.5M/UniWorld/EditScore-RL appear in training while ImgEdit is evaluated. The especially large gains, +22.1 mIoU on Lisa-Grounding and +34.1 on GSM8K, need a decontamination audit. \n\nThat said, the methodology is largely coherent. The unified weighted-policy-gradient view is not new — the paper credits Shao et al. and Lv et al. — but extending it to multimodal understanding and generation, addressing the vanishing-signal problem with answer-forcing and tree search, and carefully ablating likelihood estimators are legitimately useful. The authors also acknowledge residual gaps to AR MLLMs and the reward-model limitations for T2I. \n\nOther soft spots: no error bars or significance tests, several gains are small (ChartQA +0.9, MMMU-Pro +0.9), and no code or data release. The answer-forcing ablation showing collapse at 100% injection is a useful caveat. \n\nThis paper is for researchers working on RL for dLLMs or unified multimodal models. The recipe and ablation details retain value even if the benchmark results are scaled down. I'd bring it to a reading group, but only with the decontamination question on the table. \n\nRecommendation: send it to peer review. The framework deserves scrutiny from a serious referee, who should demand a train/test split check, re-evaluation on uncontaminated subsets, and ideally release of code and data. If the overlap is confirmed, the text-only and grounding gains should be re-reported or removed.","headline":"A genuinely useful training recipe for multimodal dLLMs, but the lack of train/eval decontamination statements makes the headline numbers, especially GSM8K/MATH, untrustworthy until audited.","tokens_in":31807,"tokens_out":2577,"would_cite":false,"duration_ms":23455,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LaViDa-R1 shows that a single unified post-training objective—combining supervised fine-tuning, group-relative reinforcement learning, and best-of-N self-distillation—can improve a multimodal diffusion language model's reasoning on visual m","keywords":["diffusion language models","multimodal reasoning","reinforcement learning","policy gradient","object grounding","image editing","visual math reasoning","post-training"],"falsifier":"Run a held-out re-split of Lisa-Grounding and ImgEdit, removing any images or prompts that overlap with ReasonSeg, Lisa-CoT, RefCOCO, GPT-Edit-1.5M, or UniWorld-V1. If the +22.1 mIoU and +0.09 ImgEdit gains shrink to near SFT levels, the reported improvements measure memorization rather than reasoning.","tokens_in":30907,"feed_emoji":"🧠","tokens_out":4978,"duration_ms":44873,"temperature":0.7,"pith_summary":"This paper tries to show that a diffusion-based multimodal language model can be made to reason better across many tasks with one shared post-training recipe, instead of task-specific fine-tuning. The recipe combines supervised fine-tuning, online reinforcement learning, and self-distillation in one weighted policy-gradient objective, with two mechanisms to keep the training signal strong when rollouts are poor: answer-forcing, which masks in a ground-truth answer and lets the diffusion model invent the reasoning trace, and tree search, which branches from the best partial trajectory. It also replaces the usual Monte Carlo likelihood estimate with a complementary-masking estimator that covers all tokens with balanced weights. On the paper's benchmarks, the recipe lifts the SFT baseline from 57.6 to 60.0 on MathVista, from 36.9 to 60.0 mIoU on Lisa-Grounding, and from 3.81 to 3.90 on ImgEdit. The larger point is that reinforcement learning for diffusion language models need not diverge or shrink to a single task.","feed_headline":"One unified RL pass lifts multimodal diffusion reasoning","feed_subtitle":"The same weighted policy-gradient objective now covers math, grounding, and image editing in one diffusion model.","key_machinery":"The load-bearing object is the unified policy-gradient objective J(θ) = (1/N) Σ_i A_i log π_θ(y_i|x_i), where the same form covers supervised fine-tuning (constant advantage 1), online GRPO (normalized group rewards), and best-of-N self-distillation (advantage 1 on the best rollout, 0 elsewhere), combined by weighted averaging. Around this objective sit three supporting mechanisms: answer-forcing, which inpaints a reasoning trace conditioned on a known answer; tree search, which branches future rollouts from the best earlier diffusion state; and a complementary-masking likelihood estimator, which couples two masked views so every token contributes once with equal weight.","core_discovery":"The central claim is that a unified post-training framework—not task-specific RL—is enough to elicit reasoning in a multimodal diffusion language model. Formally, the paper expresses SFT, online GRPO, and best-of-N distillation as instances of one weighted policy-gradient objective J = (1/N) Σ A_i log π_θ(y_i|x), then mixes them in a single data engine. Because diffusion language models cannot compute exact sequence likelihoods, the paper estimates log π_θ via two complementary-masked samples with uniform per-token weighting. To avoid vanishing advantage when a group has no good rollouts, answer-forcing injects the ground-truth answer into a masked sequence and lets the model inpaint the rea","pith_inferences":["Editorial inference: the complementary-masking estimator with uniform weighting should transfer to other discrete diffusion models and to non-reasoning objectives such as long-sequence SFT, since token imbalance is a general problem.","Editorial inference: if the grounding gains survive on truly disjoint splits, answer-forcing effectively distills a hidden search process into the model weights, so test-time tree search may become unnecessary for tasks with verifiable answers.","Editorial inference: a testable extension is to apply answer-forcing to other verifiable-reward tasks without reasoning annotations, such as spatial planning or embodied instruction following, where the final answer pins down the solution.","Editorial inference: the paper's own caveat about reward models for text-to-image generation suggests the framework's ceiling is currently set by reward quality rather than by the policy-gradient machinery; better reasoning-aware reward models could unlock the same recipe for generation-only tasks."],"forward_implications":["A single multimodal diffusion language model can be trained to reason across understanding and generation tasks in one RL stage, removing the need for dataset-specific fine-tuning.","Replacing KL regularization with SFT regularization offers a stability recipe for diffusion-LLM RL, avoiding collapse while allowing exploration, and reduces memory and compute because no reference model is needed.","The complementary-masking likelihood estimator is a drop-in estimator for any diffusion-LLM RL objective, improving gradient coverage and balance over single-sample Monte Carlo estimates.","Answer-forcing provides a way to generate high-quality reasoning traces from just final answers, useful for tasks like grounding where reasoning traces are scarce.","The reported +22.1 mIoU on Lisa-Grounding suggests that reasoning-centric RL can substantially improve tasks requiring multi-step visual inference, not just mathematical reasoning."],"fun_headline_variants":["Unified RL objective trains one diffusion model for math, grounding, and editing","Answer-forcing and tree search let one diffusion model reason across tasks","From visual math to image editing: one RL framework lifts diffusion reasoning","Single weighted policy gradient unifies SFT and RL for multimodal diffusion"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that the grounding and editing gains measure generalizable reasoning rests on the assumption that the evaluation benchmarks (Lisa-Grounding, ImgEdit) are not contaminated by training data; the appendix lists ReasonSeg, Lisa-CoT, RefCOCO, GPT-Edit-1.5M, and UniWorld-V1 in training, and it evaluates on Lisa-Grounding and ImgEdit.","fun_headline_variants_meta":{"raw":{"variants":["Unified RL objective trains one diffusion model for math, grounding, and editing","Answer-forcing and tree search let one diffusion model reason across tasks","From visual math to image editing: one RL framework lifts diffusion reasoning","Single weighted policy gradient unifies SFT and RL for multimodal diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000615,"raw_usage":{"total_tokens":2664,"prompt_tokens":686,"completion_tokens":1978,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":1901}},"tokens_in":430,"tokens_out":1978,"duration_ms":13498,"temperature":1.0,"reasoning_tokens":1901,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T23:16:15.784098+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a held-out re-split of Lisa-Grounding and ImgEdit, removing any images or prompts that overlap with ReasonSeg, Lisa-CoT, RefCOCO, GPT-Edit-1.5M, or UniWorld-V1. If the +22.1 mIoU and +0.09 ImgEdit gains shrink to near SFT levels, the reported improvements measure memorization rather than reasoning.","supporting_citations":[],"review_version":1}