Pith. sign in

REVIEW 9 cited by

TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.09797 v1 pith:Y5WVQSNY submitted 2024-04-15 cs.CV

Bozhi Luan , Hao Feng , Hong Chen , Yonghui Wang , Wengang Zhou , Houqiang Li This is my paper · ORCID
classification cs.CV
keywords imagetextcotgloballmmstext-richunderstandingstageability
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The advent of Large Multimodal Models (LMMs) has sparked a surge in research aimed at harnessing their remarkable reasoning abilities. However, for understanding text-rich images, challenges persist in fully leveraging the potential of LMMs, and existing methods struggle with effectively processing high-resolution images. In this work, we propose TextCoT, a novel Chain-of-Thought framework for text-rich image understanding. TextCoT utilizes the captioning ability of LMMs to grasp the global context of the image and the grounding capability to examine local textual regions. This allows for the extraction of both global and local visual information, facilitating more accurate question-answering. Technically, TextCoT consists of three stages, including image overview, coarse localization, and fine-grained observation. The image overview stage provides a comprehensive understanding of the global scene information, and the coarse localization stage approximates the image area containing the answer based on the question asked. Then, integrating the obtained global image descriptions, the final stage further examines specific regions to provide accurate answers. Our method is free of extra training, offering immediate plug-and-play functionality. Extensive experiments are conducted on a series of text-rich image question-answering benchmark datasets based on several advanced LMMs, and the results demonstrate the effectiveness and strong generalization ability of our method. Code is available at https://github.com/bzluan/TextCoT.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework

    cs.CV 2026-02 conditional novelty 6.0 of 10

    HART uses a closed-loop 'crop-and-answer' training scheme plus a dynamic-weight GRPO variant to improve LMM grounding and high-resolution reasoning without bounding-box annotations.

  2. Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Selecting the visual input that minimizes an MLLM's output entropy (or maximizes its yes/no confidence) improves fine-grained visual search, long-video QA, and temporal grounding without any training.

  3. VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A vision-language model learns via reinforcement learning when to upscale a low-resolution image, cutting visual tokens roughly in half while preserving accuracy on most benchmarks.

  4. Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinement

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A training-free two-step pipeline, localize a bounding box then refine the answer from a high-resolution crop, improves MLLM accuracy on high-resolution image benchmarks.

  5. GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A cue-tagging and rethinking training recipe lifts a 7B multimodal model to 81.5% on M3CoT, though the gain may reflect training on the same benchmark.

  6. Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A two-stage layout-focused visual reasoning method, Doc-CoB, improves document question answering by having the model select key layout boxes and then answer from those boxes.

  7. M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding

    eess.IV 2026-01 conditional novelty 5.0 of 10

    A medical-image benchmark that scores the step-by-step reasoning chains of multimodal LLMs shows current models explain poorly and chain-of-thought prompting frequently reduces diagnostic accuracy.

  8. Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A 3B VLM trained with GRPO to call a zoom tool improves V*Bench accuracy by 5.7% over its base model but degrades TextVQA and HR-Bench performance.

  9. Training-Free Reasoning and Reflection in MLLMs

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Training-free, layer-wise weight merging of an MLLM with a reasoning LLM, using attention-derived priors, raises MMMU accuracy from 63.9 to 69.2 at the 38B scale.

Pith tools