REVIEW 15 cited by
MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Open-source multimodal large language models (MLLMs) have shown significant potential in a broad range of multimodal tasks. However, their reasoning capabilities remain constrained by existing instruction-tuning datasets, which were predominately repurposed from academic datasets such as VQA, AI2D, and ChartQA. These datasets target simplistic tasks, and only provide phrase-level answers without any intermediate rationales. To address these challenges, we introduce a scalable and cost-effective method to construct a large-scale multimodal instruction-tuning dataset with rich intermediate rationales designed to elicit CoT reasoning. Using only open models, we create a dataset containing 12M instruction-response pairs to cover diverse, reasoning-intensive tasks with detailed and faithful rationales. Experiments demonstrate that training MLLMs on this dataset significantly improves reasoning capabilities, achieving state-of-the-art performance on benchmarks such as MathVerse (+8.1%), MMMU-Pro (+7%), and MuirBench (+13.3%). Additionally, the model demonstrates notable improvements of up to 4% on non-reasoning-based benchmarks. Ablation studies further highlight the importance of key components, such as rewriting and self-filtering, in the dataset construction process.
Forward citations
Cited by 15 Pith papers
-
ScaleLong: A Multi-Timescale Benchmark for Long Video Understanding
ScaleLong embeds four timescale question types into the same long videos, and evaluation of 23 MLLMs reveals a U-shaped accuracy curve across timescales.
-
Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
Point-RFT uses point-grounded chain-of-thought with GRPO reinforcement to improve chart reasoning, reporting 90.04% on ChartQA, though internal tables and the OOD setup weaken the claim.
-
Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
Sparse-LaViDa speeds up masked diffusion LLMs by truncating masked tokens, using register tokens and a step-causal attention mask, reaching ~2-2.8x speedup over LaViDa-O with comparable generation quality.
-
Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
Lavida-O introduces an elastic mixture-of-transformers architecture that brings high-resolution text-to-image generation, object grounding, and image editing into a single masked diffusion model, using planning and se...
-
The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs
Long-CoT SFT and RL trade off strengths in reasoning VLMs, and five combination strategies fail to produce additive gains.
-
BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.
-
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
DenseWorld-1M provides one million images with detailed object captions, pixel masks, and spatial relations by chaining SAM, APE, RAM++, and VLMs through a three-stage labeling pipeline.
-
MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings
A two-stage training recipe that converts causal VLMs into bidirectional multimodal embedding models, achieving SOTA on MMEB.
-
VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
A vision-expert-filtered, chain-of-thought-guided, iteratively fine-tuned reward model boosts a compact 7B model's ability to judge vision-language responses, especially detecting hallucinations.
-
Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification
An iterative, verifier-guided visual token scaling framework improves multi-step visual reasoning in both closed and open multimodal models on BLINK and related benchmarks.
-
VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism
VReST combines Monte Carlo tree search with a self-reward signal inside a vision-language model to get higher accuracy than CoT, ToT, or voting baselines on MathVista, MathVision, and CharXiv, while spending several t...
-
Multimodal Tabular Reasoning with Privileged Structured Information
An 8B multimodal LLM trained on 9k reasoning traces distilled from structured tables reaches state-of-the-art open-source accuracy on table-image question answering and fact verification.
-
Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start
Using a two-stage SFT-plus-GRPO pipeline, the authors train 3B and 7B multimodal models that beat prior open-source models on four math reasoning benchmarks, while showing that reflective 'aha moment' text is not a re...
-
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.
-
A Survey on Diffusion Language Models
A comprehensive survey of diffusion language models covering taxonomy, training and inference techniques, and comparisons with autoregressive models.
Discussion (0). Sign in to comment.