REVIEW 6 cited by
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Document content extraction is a critical task in computer vision, underpinning the data needs of large language models (LLMs) and retrieval-augmented generation (RAG) systems. Despite recent progress, current document parsing methods have not been fairly and comprehensively evaluated due to the narrow coverage of document types and the simplified, unrealistic evaluation procedures in existing benchmarks. To address these gaps, we introduce OmniDocBench, a novel benchmark featuring high-quality annotations across nine document sources, including academic papers, textbooks, and more challenging cases such as handwritten notes and densely typeset newspapers. OmniDocBench supports flexible, multi-level evaluations--ranging from an end-to-end assessment to the task-specific and attribute--based analysis using 19 layout categories and 15 attribute labels. We conduct a thorough evaluation of both pipeline-based methods and end-to-end vision-language models, revealing their strengths and weaknesses across different document types. OmniDocBench sets a new standard for the fair, diverse, and fine-grained evaluation in document parsing. Dataset and code are available at https://github.com/opendatalab/OmniDocBench.
Forward citations
Cited by 6 Pith papers
-
LayoutLite: Token-Level Implicit Layout Analysis for Efficient Document OCR
Token-level importance scoring trained by RL plus a layout-detection teacher prunes half the visual tokens in VLM document OCR while holding OmniDocBench scores within about 1.3-2.2 points and cutting prefill cost by ...
-
Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild
A full-scale physical reconstruction of OmniDocBench with five distortion scenarios shows all document-parsing models degrade in the real world, with the authors' PaddleOCR-VL-1.5 topping the leaderboard.
-
Ming-Omni: A Unified Multimodal Model for Perception and Generation
A single model with modality-specific routing processes image, text, audio, and video inputs and generates text, speech, and images, with public benchmarks reported across all of these abilities.
-
OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoning
OCR-Reasoning, a 1,069-question benchmark with reasoning-chain annotations for text-rich images, finds that no evaluated multimodal model surpasses 50% accuracy.
-
Heterogeneous Element-Aware Cross-Version Differencing of Scientific Documents via Layout-Aware Alignment and Structure-Aware Reasoning
A layout-aware, alignment-first framework decomposes two PDF versions into typed elements, aligns them, and reports detection, localization, and structure-aware changes, outperforming element-specific baselines on a p...
-
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
A structured survey of multimodal RAG systems, covering datasets, benchmarks, methods, and open challenges, with a public resource repo.
Discussion (0). Continue with ORCID to comment.