REVIEW 23 cited by
LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Reasoning is a fundamental capability for solving complex multi-step problems, particularly in visual contexts where sequential step-wise understanding is essential. Existing approaches lack a comprehensive framework for evaluating visual reasoning and do not emphasize step-wise problem-solving. To this end, we propose a comprehensive framework for advancing step-by-step visual reasoning in large language models (LMMs) through three key contributions. First, we introduce a visual reasoning benchmark specifically designed to evaluate multi-step reasoning tasks. The benchmark presents a diverse set of challenges with eight different categories ranging from complex visual perception to scientific reasoning with over 4k reasoning steps in total, enabling robust evaluation of LLMs' abilities to perform accurate and interpretable visual reasoning across multiple steps. Second, we propose a novel metric that assesses visual reasoning quality at the granularity of individual steps, emphasizing both correctness and logical coherence. The proposed metric offers deeper insights into reasoning performance compared to traditional end-task accuracy metrics. Third, we present a new multimodal visual reasoning model, named LlamaV-o1, trained using a multi-step curriculum learning approach, where tasks are progressively organized to facilitate incremental skill acquisition and problem-solving. The proposed LlamaV-o1 is designed for multi-step reasoning and learns step-by-step through a structured training paradigm. Extensive experiments show that our LlamaV-o1 outperforms existing open-source models and performs favorably against close-source proprietary models. Compared to the recent Llava-CoT, our LlamaV-o1 achieves an average score of 67.3 with an absolute gain of 3.8\% across six benchmarks while being 5 times faster during inference scaling. Our benchmark, model, and code are publicly available.
Forward citations
Cited by 23 Pith papers
-
Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark
MMRB is the first benchmark combining multi-image inputs with chain-of-thought reasoning annotations, and its evaluation shows open-source MLLMs trail commercial models while multi-image reward models are unstable.
-
More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Reasoning models trade visual grounding for language-based inference, and this paper measures that trade-off with a new metric and benchmark.
-
Towards Spoken Mathematical Reasoning: Benchmarking Speech-based Models over Multi-faceted Math Problems
A new spoken math benchmark, Spoken-MQA, shows that current speech-based AI models reason poorly from spoken math input, especially for arithmetic and knowledge-heavy problems.
-
SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning
SyncLoop jointly evolves multimodal training data and model capability through alternating SFT and RL, selecting error-prone samples to improve geometry reasoning.
-
Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need
A language model fine-tuned on knowledge-graph-path reasoning tasks (QwQ-Med-3) beats strong baselines on a same-style benchmark but shows mixed gains on external medical QA tests.
-
VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
A vision-language model learns via reinforcement learning when to upscale a low-resolution image, cutting visual tokens roughly in half while preserving accuracy on most benchmarks.
-
MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning
MagiC evaluates answer correctness, reasoning validity, grounding fidelity, and self-correction on about 900 hand-annotated visual questions across 15 vision-language models.
-
Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models
Vision-language models can handle some 2D shape puzzles but nearly all fail at multi-step 3D spatial deformation reasoning.
-
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
A 3B agent trained with supervised tool-use traces and reinforcement learning answers week-long egocentric video questions by dynamically selecting hierarchical retrieval, video-LLM, and VLM tools.
-
VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos
VRBench is a benchmark of 960 long narrative videos with 8,243 human-written multi-step questions, plus a two-level evaluation of answer accuracy and reasoning quality for 31 large models.
-
Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoning
Training a multimodal LLM on GPT-4o-generated chain-of-thought referring traces, then optimizing with GRPO, improves referring accuracy and abstention on HumanRef.
-
VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking
GRPO post-training with structured thinking and dual think/caption rewards improves Qwen2-VL-7B video captioning over the base model and SFT on DREAM-1K, VDC, and CAREBENCH.
-
GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking
A cue-tagging and rethinking training recipe lifts a 7B multimodal model to 81.5% on M3CoT, though the gain may reflect training on the same benchmark.
-
Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks
Agent-X, a new 828-task benchmark with step-level scoring, finds that the best multimodal agents still fail over half of real-world visual tool-use tasks.
-
MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs
A new 1,188-question multimodal benchmark covering deductive, inductive, and abductive reasoning shows that leading MLLMs score around 60% and are especially weak at abductive reasoning.
-
Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought
A two-stage trained MLLM with a perception-to-cognition chain-of-thought and a self-verification RL reward outperforms prior models on video anomaly detection and reasoning.
-
ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark
ARB provides 1,356 Arabic multimodal questions with 5,119 human-reviewed reasoning steps and shows leading models score much higher on reasoning fluency than on correct answers.
-
WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning
WeThink, a 120K-image QA dataset with AI-generated reasoning paths, combined with hybrid-reward reinforcement learning, improves a 7B vision-language model across 14 benchmarks.
-
From Street Views to Urban Science: Discovering Road Safety Factors with Multimodal Large Language Models
UrbanX uses LLM-generated visual questions, MLLM answers, and linear regression to predict crash rates, claiming better performance than ResNet/ViT while keeping features interpretable.
-
Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start
Using a two-stage SFT-plus-GRPO pipeline, the authors train 3B and 7B multimodal models that beat prior open-source models on four math reasoning benchmarks, while showing that reflective 'aha moment' text is not a re...
-
See, Think, Learn: A Self-Taught Multimodal Reasoner
A self-training loop that structures model rationales into image-caption, reasoning, and conclusion, and adds negative rationales, improves VLM accuracy on M3CoT over answer-only and STaR baselines.
-
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.
-
Enhancing Scientific Visual Question Answering through Multimodal Reasoning and Ensemble Modeling
On the SciVQA 2025 benchmark, an InternVL3 model with optimized prompts and chain-of-thought instructions reaches ROUGE-1 and ROUGE-L F1 of 0.740, and a figure-type-aware ensemble ranks 5th.
Discussion (0). Sign in to comment.