The reviewed record of science sign in
Pith

arxiv: 2503.11576 · v1 · pith:JJCLIWZU · submitted 2025-03-14 · cs.CV

SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

Ahmed Nassar , Andres Marafioti , Matteo Omenetti , Maksym Lysak , Nikolaos Livathinos , Christoph Auer , Lucas Morin , Rafael Teixeira de Lima
show 5 more authors
Yusik Kim A. Said Gurbuz Michele Dolfi Miquel Farr\'e Peter W. J. Staar
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:JJCLIWZUrecord.jsonopen to challenge →

classification cs.CV
keywords documentmodelsmoldoclingconversionend-to-endmodelsvision-languageavailable
0
0 comments X
read the original abstract

We introduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms -- significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding

    cs.CV 2026-05 conditional novelty 7.0

    Injecting pre-computed layout priors from RT-DETR into VLM prompts raises markdown F1 from 0.37 to 0.92 on a 10k-page OOD benchmark and cuts infinite-loop failures across domains.

  2. Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models

    cs.CV 2026-03 unverdicted novelty 7.0

    Q-Mask uses query-conditioned causal masks to separate text location from recognition in OCR VLMs, backed by a new benchmark and 26M-pair training dataset.

  3. POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction

    cs.CV 2026-06 unverdicted novelty 6.0

    POTATR extends TATR into a 29M-parameter image-to-graph model for contextual page-level table extraction, reporting 0.964 GriTS_Con on PubTables-v2 Single Pages while running 130x faster and 300x cheaper than tested a...

  4. TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?

    cs.AI 2026-05 conditional novelty 6.0

    TeleCom-Bench reveals LLMs reach 90% on telecom intent and entity tasks but drop to 30% on solution generation and root cause analysis in live network scenarios.

  5. DocAtlas: Multilingual Document Understanding Across 80+ Languages

    cs.CL 2026-05 unverdicted novelty 6.0

    DocAtlas creates multilingual document datasets across 82 languages and shows DPO with rendered ground truth improves model accuracy by 1.7-1.9% without degrading base-language performance, unlike supervised fine-tuning.

  6. DocAtlas: Multilingual Document Understanding Across 80+ Languages

    cs.CL 2026-05 unverdicted novelty 6.0

    DocAtlas introduces model-free rendering pipelines to create DocTag-annotated datasets across 82 languages and shows DPO adaptation improves multilingual performance without base-language degradation.

  7. DeepSeek-OCR: Contexts Optical Compression

    cs.CV 2025-10 unverdicted novelty 6.0

    DeepSeek-OCR compresses text contexts up to 20x via 2D optical mapping while achieving 97% OCR accuracy below 10x and 60% at 20x, outperforming prior OCR tools with fewer vision tokens.

  8. MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing

    cs.CV 2025-09 unverdicted novelty 6.0

    MinerU2.5 uses a two-stage decoupled vision-language architecture to achieve state-of-the-art document parsing accuracy with lower computational overhead than existing general and domain-specific models.

  9. PaddleOCR 3.0 Technical Report

    cs.CV 2025-07 unverdicted novelty 4.0

    PaddleOCR 3.0 releases compact open-source models for OCR, document structure parsing, and information extraction that rival billion-parameter VLMs.