REVIEW 14 cited by
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
When we look around and perform complex tasks, how we see and selectively process what we see is crucial. However, the lack of this visual search mechanism in current multimodal LLMs (MLLMs) hinders their ability to focus on important visual details, especially when handling high-resolution and visually crowded images. To address this, we introduce V*, an LLM-guided visual search mechanism that employs the world knowledge in LLMs for efficient visual querying. When combined with an MLLM, this mechanism enhances collaborative reasoning, contextual understanding, and precise targeting of specific visual elements. This integration results in a new MLLM meta-architecture, named Show, sEArch, and TelL (SEAL). We further create V*Bench, a benchmark specifically designed to evaluate MLLMs in their ability to process high-resolution images and focus on visual details. Our study highlights the necessity of incorporating visual search capabilities into multimodal systems. The code is available https://github.com/penghao-wu/vstar.
Forward citations
Cited by 14 Pith papers
-
SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning
A small tool-free MLLM plus answer-separability gating bypasses agentic tool loops for many queries, yielding 1.1–3.35× speedup with preserved or higher accuracy.
-
HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling
HiDe uses token-wise attention decoupling and layout-preserving decoupling to build compact crops that push Qwen2.5-VL and InternVL3 to state-of-the-art scores on high-resolution VQA benchmarks.
-
VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation
Counterfactual present/removed teacher views attribute visually supported corrections and reconstruct student-anchored distillation targets that beat source-mixed multimodal OPD.
-
ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding
ST-Veto improves reasoning in diffusion MLLMs by vetoing temporally unstable tokens and tokens with weak image grounding, swapping in safer near-boundary candidates.
-
What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs
VLM accuracy can be predicted from a scalar capability score derived from LLM text benchmarks plus multimodal data volume via a fitted transfer-absorption scaling law.
-
Starve to Perceive: Taming Lazy Perception in VLMs with Constrained Visual Bandwidth
Constraining visual token budgets during SFT and RL forces VLMs to learn functional active perception, yielding ~5% relative gains and strong transfer to unconstrained evaluation.
-
Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models
Multimodal reasoning models hallucinate at high-entropy cognitive bifurcation points due to loss of visual semantic anchoring, and the V-STAR training paradigm with HVAR rewards and FRM reflection mitigates this by re...
-
Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
Blink dynamically expands high-saliency visual tokens and drops them when attention shifts, improving LLaVA-1.5 and LLaVA-NeXT across seven multimodal benchmarks.
-
Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models
A dynamic replay and reweighting scheduler (RECAP) preserves general capabilities during RLVR while keeping reasoning performance at least as good as reasoning-only finetuning.
-
LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model
RL training on preference-labeled critic data transforms a 7B vision-language model into both a stronger critic and a stronger generative policy, improving average benchmark accuracy by 5.7% and enabling self-critique...
-
Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations
STARE is a 4K-task benchmark showing multimodal LLMs perform near random chance on multi-step spatial simulation tasks such as cube net folding and tangrams, despite strong 2D transformation results.
-
LanteRn: Latent Visual Structured Reasoning
A 3B vision-language model trained to emit latent visual thought tokens interleaved with text, then refined by reinforcement learning, outperforms a matched text-only baseline on several visual reasoning benchmarks.
-
MiMo-VL Technical Report
MiMo-VL-7B-RL, a 7B open-source vision-language model, reports state-of-the-art results on 35 of 40 benchmarks and a 59.4 OlympiadBench score, with the report crediting long-CoT pretraining data and mixed on-policy RL.
-
Unifying Language Agent Algorithms with Graph-based Orchestration Engine for Reproducible Agent Research
AGORA is a graph-based agent framework that standardizes ten reasoning algorithms; its evaluations show simple Chain-of-Thought prompting is often the most cost-effective, though without statistical rigor.
Discussion (0). Sign in to comment.