Pith. sign in

REVIEW 3 major objections 5 minor 138 references

Open document parser tops two benchmarks, 87.6% and 74.3%

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

An end-to-end document parser trained on 5M synthetic and mined pages with multi-task RL reaches 87.6% on olmOCR-Bench and 74.3% on ParseBench.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection Serious systems paper with a plausible SOTA claim, but the flywheel's tight coupling to the fixed benchmark suite is the load-bearing question to answer before trusting the headline numbers. the 3 major comments →

arxiv 2607.07836 v3 pith:FZIPTAFU submitted 2026-07-08 cs.AI

Infinity-Parser2 Technical Report

Zuming Huang , Jun Huang , Kexuan Ren , Baode Wang , Weizhen Li , Jianming Feng , Yu Wang , Yichen Yao
show 7 more authors
Shijun Lin Yige Tang Cheng Peng Weidi Xu Wei Chu Yinghui Xu Yuan Qi
This is my paper
classification cs.AI
keywords document parsingmultimodal large language modeldata synthesisreinforcement learningDOM renderinglayout analysisstructured extractiondocument VQA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Infinity-Parser2 tries to show that the two bottlenecks in document parsing — scarce faithful training data and fragmented per-task optimization — can be broken by coupling a controllable data-synthesis engine with joint multi-task reinforcement learning. The paper reports that its Pro model reaches 87.6% on olmOCR-Bench and 74.3% on ParseBench, ahead of other open parsers, while the Flash variant keeps most of that accuracy at 3.68x the throughput of the team's previous parser. It also argues the same recipe transfers to tables, charts, chemical formulas, and document VQA because all eight co-trained objectives share one optimization signal. If that is right, end-to-end parsing can be driven by synthetic documents whose annotations are exact by construction, rather than by expensive human labeling.

Core claim

The central claim is that one vision-language model can map a document page to a structured, reading-ordered representation — JSON with per-element categories, bounding boxes and text, or Markdown/HTML/LaTeX — as accurately as pipeline systems, provided it is first trained on a large corpus of exact synthetic labels and then refined with a reward that credits where an element is and what it says separately. On the paper's evidence, this is realized by the Pro model's top scores on olmOCR-Bench and ParseBench and its strong results on layout analysis, table parsing, math formulas, charts, chemical structures, and document QA. The authors' own framing: the DOM-based synthesis engine makes trai

What carries the argument

The load-bearing object is a DOM-based document synthesis engine. It uses a browser layout engine to typeset documents from a logical tree of typed elements, then reads bounding boxes, reading order, element categories, and structured text directly off the laid-out tree, so pixels and labels are consistent by construction. Around it sits a four-stage data flywheel: evaluate the current model on a fixed benchmark suite, mine its weak cases into tags, collect and synthesize new data targeting those tags, fine-tune, and repeat until gains saturate. The second mechanism is a reward router for joint reinforcement learning: each of eight tasks is scored by its own verifiable metric, the spatial an

Load-bearing premise

The flywheel's guarantee that evaluation-benchmark samples never enter the training pool: if benchmark-like pages leak into Infinity-Doc2-5M, or if the training distribution is tuned to the benchmarks' specific layouts, the reported state-of-the-art numbers would not reflect general document parsing.

What would settle it

Take the released Pro model, evaluate it on a freshly collected document set that was never part of olmOCR-Bench, ParseBench, or the training corpus (for example, recent legal filings or handwritten archives), and search Infinity-Doc2-5M for near-duplicates of benchmark pages. If accuracy on the fresh set falls far short of the reported margins, or if duplicate pages are found, the flywheel's benchmark-safety claim is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the reported scores hold, the released open-weight Pro model is the strongest open end-to-end document parser among the compared systems on both olmOCR-Bench and ParseBench.
  • The two variants give a concrete accuracy-versus-latency trade: Flash scores about 1.6 points lower on olmOCR-Bench but runs about 3.7x faster, so high-volume document pipelines can choose the cheaper model without leaving most of the quality behind.
  • The 5M-sample corpus, the synthesis code, and both model weights are released, which means the annotation bottleneck the paper targets is directly lowered for anyone training a document parser.
  • Because the same instruction-selected model handles layout, tables, math, charts, chemical formulas, and document VQA, users no longer need a separate system for each structured output type.
  • The spatial component of the reward demonstrably feeds back into text fidelity: adding per-category mIoU to the textual reward improved both layout and parsing scores in the ablations, tying localization accuracy to reading-order quality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The DOM-as-single-source-of-truth trick is not limited to documents: any structured output that can be rendered from a typed tree (diagrams, music scores, CAD, timeline graphics) could get cheap exact annotations by registering a renderer, an extension the paper gestures at but does not test.
  • The disentangled spatial-plus-textual reward is a general recipe for any detection-and-recognition task; a testable extension would be applying it to forms processing or receipt extraction and checking whether localization accuracy improves downstream field-level extraction.
  • The flywheel's fixed benchmark suite means the reported gains are measured on the same task distribution the loop optimizes against; running the same loop on a freshly built benchmark from an unseen domain would test whether the per-round gains transfer or are specific to the suite.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents Infinity-Parser2, a family of end-to-end document parsing VLMs built on Qwen3.5 backbones. It combines a DOM-based synthesis engine and a data iteration flywheel to construct a 5M-sample bilingual corpus (Infinity-Doc2-5M), then trains the models with SFT followed by multi-task RL using verifiable task-native metrics as rewards. The Pro variant claims state-of-the-art results on olmOCR-Bench (87.6) and ParseBench (74.3), while the Flash variant offers a 3.68x throughput gain over the prior Infinity-Parser-7B. The paper includes extensive evaluations across layout analysis, table/formula/chart/chemical parsing, document VQA, general multimodal understanding, and a downstream financial-information-extraction task, along with ablations on resolution, reward design, data flywheel rounds, and training strategies.

Significance. If the reported results hold, the paper makes a substantial contribution: an open end-to-end document parser with state-of-the-art scores on two recent benchmarks, a publicly released 5M-sample corpus, and a transparent metric-as-reward RL recipe. The design is methodologically clean in several places: the reward is explicitly the evaluation metric for each task, the DOM-based synthesis engine produces exact labels by construction, and the ablations are well structured and informative. The limitations section is candid about bilingual scope, pseudo-label noise, and formatting losses. However, the headline SOTA claim depends on two unverified links: the flywheel's non-contamination guarantee (Sec. 3.1), and the comparability of the mixed re-scored/cited baseline numbers in the main tables. These need to be strengthened before the claim can be considered settled.

major comments (3)
  1. [Sec. 3.1 (Stage 1) and Table 10] The claim that benchmark samples are 'never reused as training data' is asserted but not supported by any deduplication or overlap audit. The flywheel evaluates on the fixed suite (olmOCR-Bench, ParseBench, OmniDocBench), mines weakness tags from low-scoring samples, and then drives web scraping and synthesis with those same tags. Table 10 attributes a 54.9-point gain on olmOCR-Bench (28.4 to 83.3) to pseudo-labeled web data. If any pages resembling the benchmark distribution enter the training pool, or if the tag-driven acquisition implicitly selects for benchmark-specific layout/domain features, the headline 87.6/74.3 numbers would be inflated. Please report a concrete contamination analysis, e.g., near-duplicate detection (hash or embedding similarity) between Infinity-Doc2-5M and each benchmark's test pages, and/or a holdout evaluation on a benchmark never used in the flywheel loop.
  2. [Tables 2, 4, 5, 7 and Sec. 5.1] The headline comparisons mix re-scored baselines (marked '*') with numbers cited from original reports, and the main SOTA tables lack error bars/confidence intervals except for olmOCR-Bench. For example, the ParseBench margin (74.3 vs. 70.1) is a 4.2-point gap on a roughly 2,000-page benchmark, and OmniDocBench-v1.6 is reported as a point estimate. The text says re-evaluation follows 'officially recommended' prompts, but the reader cannot tell which cells are re-scored and which are quoted, and prompt/output-format differences can shift scores materially. Please state for every cell which source applies, report standard errors or bootstrap CIs for the key comparisons, and make the evaluation code/prompts available.
  3. [Sec. 5.7, Table 14] The FinIE downstream evaluation uses 'Infinity-Parser2-Flash*' — a variant further finetuned on internal financial tables — to claim the highest F1 (95.75). This is not one of the two released models (Flash/Pro), so the result does not directly support the paper's claims about the released system. Clarify whether the finetuned checkpoint will be released, or reposition this result as an illustration of the architecture's headroom rather than a property of the released models.
minor comments (5)
  1. [Sec. 4 (before Figure 4)] There is a stray placeholder text '在此处键入公式。' in the manuscript; please remove it.
  2. [Table 3 and Sec. A.1] The five broad categories used in the mIoU protocol should be explicitly listed in the table caption, and the nonstandard label notation for D4LA should be explained.
  3. [Sec. A.1 (CoSyn-Chemical)] CoSyn-Chemical validation has only 128 image-text pairs; this should be stated in the main text near Table 6, since the reported differences of a few InChI points are within sampling noise.
  4. [References] Reference [14] is cited for the CoSyn-Chemical benchmark, but the reference describes code-guided synthetic data generation in general; please cite the actual benchmark release or clarify the source.
  5. [Figure 1 (bottom panel)] The 'N/A' entries in the multi-task panel are unexplained; add a note describing why certain cells are not applicable.

Circularity Check

1 steps flagged

Flywheel tunes training data to the reported benchmarks; the headline SOTA is the fitted target rather than an independent prediction.

specific steps
  1. fitted input called prediction [Sec. 3.1 (Data Iteration Flywheel), Stages 1–4; Sec. 4.3.2 (metric-as-reward)]
    "At each flywheel iteration, the current parsing model is evaluated on a multi-task benchmark suite covering end-to-end document parsing, layout analysis, and element-level parsing. ... the accumulated weakness tags act as a demand signal that flows into Stage 2, where targeted acquisition is performed on disjoint, training-safe sources matching the same weakness profile. ... The synthesis engine (whose corpus selection, layout templates, and rendering parameters are conditioned directly on the Stage 1 tags) ... Iterations continue until benchmark gains across consecutive rounds become negligib"

    The headline olmOCR-Bench/ParseBench scores are the same suite whose per-sample failures are mined in Stage 1 and converted into weakness tags; those tags drive data mining and synthesis-conditioning in Stages 2–3, and the loop stops when gains on this fixed suite saturate (Stage 4). Thus the reported SOTA numbers are the optimization target of the flywheel, not an independent evaluation. Even if individual benchmark pages are 'never reused,' the benchmark's weakness profile is the generative prior for the training data, so the benchmark score is fitted to the benchmark by construction. The RL stage compounds this: 'metric-as-reward' optimizes the exact metrics on which tasks are judged, further reducing the 'prediction' to the fitted objective.

full rationale

The principal circularity is the flywheel's use of the reported benchmark suite as both the diagnostic signal and the convergence target. Stage 1 mines weaknesses from olmOCR-Bench/ParseBench/OmniDocBench; Stages 2–3 acquire and synthesize data 'matching the same weakness profile' with synthesis 'conditioned directly on the Stage 1 tags'; Stage 4 trains until 'benchmark gains ... become negligible.' The RL stage uses 'metric-as-reward,' so the model is explicitly trained to maximize the same metrics by which it is judged. This makes the headline SOTA numbers fitted outputs of an optimization loop rather than independent predictions of general parsing ability. This is partial circularity, not total: the synthesis engine, multi-task RL recipe, and held-out benchmarks for elements (PubTabNet, UniMERNet, ChartQA, DocVQA, etc.) provide substantial independent content, and the cited prior work (Infinity-Parser, Infinity-Doc-400K) is used as a data/teacher source rather than as a uniqueness argument. The 'never reused' assertion is unverified, and even if true, the weakness-profile conditioning is a distribution-level form of fitting. Score 6 rather than 8 because the exact score still depends on model capacity, data quality, and independent benchmarks; but for the headline numbers, the derivation chain is circular.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claims are empirical and systems-level. The paper imports the Qwen3.5 backbone and expert-model pseudo-labels from prior work; it hand-sets λ, resolution/context ceilings, task mixing ratios, and flywheel budgets. No new physical or mathematical entities are postulated.

free parameters (4)
  • λ (spatial reward weight) = 0.3
    Eq. 3 and Sec 5.1: spatial and textual terms weighted by λ=0.3 and 1−λ=0.7 in the doc2json reward. Chosen by hand; no ablation sweep is shown for λ.
  • Training image resolution ceiling and context length = 4096×4096, 32768 tokens
    Sec 5.5.1 (Table 8): selected after ablations on olmOCR-Bench and OmniDocBench, i.e., tuned against the evaluation benchmarks.
  • RL data sampling ratio = 5% of each task (~220K examples)
    Sec 4.3.3: 'randomly sampling 5% of each task from Infinity-Doc2-5M' — a hand-set ratio affecting the RL mix.
  • Flywheel per-tag initialization budget = ~2,000 samples per tag
    Sec 3.1 Stage 2: 'roughly 2,000 samples are collected per tag' in the initialization round; affects data balance and downstream gains.
axioms (4)
  • domain assumption Browser layout engines render DOM content faithfully enough that read-back coordinates and labels are exact by construction
    Sec 3.2 'DOM File Generation' claims labels are 'correct by construction'; this depends on the rendering engine's fidelity, which is not formally verified.
  • domain assumption Pseudo-labels from expert models (dots.ocr, PaddleOCR-VL, MinerU2.5, Infinity-Parser) are accurate after filtering
    Sec 3.1 Stage 3 and Sec 3.3 rely on expert-model annotation; residual noise is acknowledged in Limitations, but the dominant Round-2 gain (olmOCR-Bench +54.9) depends on this premise.
  • domain assumption Re-mapping layout categories to five broad classes (textual, figure, table, formula, other) is a fair protocol across all methods
    Sec 5.2, Table 3: the mapping and re-scoring are done by the authors, not benchmark owners; fairness across methods is assumed.
  • domain assumption Qwen3.5 provides a competent multimodal backbone; gains are attributable to the training recipe
    Sec 4.2 'adopt Qwen3.5 as our foundation model'; all measured improvements are relative to this base.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Infinity-Parser2 Technical Report." pith.science (2026). https://pith.science/paper/FZIPTAFU

@misc{pith2026260707836,
  author       = {Pith},
  title        = {Pith review of: Infinity-Parser2 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FZIPTAFU}},
  note         = {Machine review of arXiv:2607.07836}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end-to-end document parsing, addressing the persistent scarcity of faithfully annotated parsing corpora. Our contributions are threefold. First, we build a scalable synthesis engine, pairing a controllable rendering framework with an iterative refinement loop, and use it to construct and open-source Infinity-Doc2-5M: a 5-million-sample bilingual (Chinese/English) corpus spanning diverse document types, annotated with element bounding boxes, canonical content forms (Markdown, HTML, LaTeX, SMILES, structured charts), and full-page reading order. Second, we introduce a verifiable, multi-task reward system that enables Joint Reinforcement Learning across eight co-trained objectives (document parsing, layout analysis, table parsing, math formula parsing, chart parsing, chemical formula parsing, document VQA, and general multimodal understanding), unifying perception, structure, and reasoning in a single optimization signal. Third, we release two variants under a shared architecture: Infinity-Parser2-Flash, optimized for low-latency inference with a 3.68x throughput gain over Infinity-Parser-7B, and Infinity-Parser2-Pro, engineered for precision-critical settings. Infinity-Parser2-Pro reaches state-of-the-art 87.6% on olmOCR-Bench and 74.3% on ParseBench, surpassing DeepSeek-OCR-2, PaddleOCR-VL-1.5, and MinerU2.5, with strong generalization to charts, chemical formulas, and document VQA.

Figures

Figures reproduced from arXiv: 2607.07836 by Baode Wang, Cheng Peng, Jianming Feng, Jun Huang, Kexuan Ren, Shijun Lin, Wei Chu, Weidi Xu, Weizhen Li, Yichen Yao, Yige Tang, Yinghui Xu, Yuan Qi, Yu Wang, Zuming Huang.

Figure 1
Figure 1. Figure 1: Performance evaluation. Top: document parsing benchmarks. Bottom: cross-domain multi-task capabilities. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overall pipeline of our data iteration flywheel, a closed loop of four stages. Stage 1 evaluates the current model [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overall framework of our DOM-based document data synthesis engine. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overall framework of Infinity-Parser2. We cast document parsing as a unified image-to-sequence problem: a [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Image resolution distribution across benchmark datasets and training dataset. Across all datasets, roughly [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization results of the A-stock image. The challenge is accurately determining column spans (colspans) [PITH_FULL_IMAGE:figures/full_fig_p036_6.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization results of the A-stock image. The challenge is accurately determining column spans (colspans) [PITH_FULL_IMAGE:figures/full_fig_p037_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization results of the multi-column image. The difficulty lies in executing complex layout analysis and [PITH_FULL_IMAGE:figures/full_fig_p036_7.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization results of the multi-column image. The difficulty lies in executing complex layout analysis and [PITH_FULL_IMAGE:figures/full_fig_p037_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization results of the newspaper image. The primary challenge is avoiding bounding box omissions [PITH_FULL_IMAGE:figures/full_fig_p037_8.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization results of the newspaper image. The primary challenge is avoiding bounding box omissions [PITH_FULL_IMAGE:figures/full_fig_p038_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization results of the US-stock image. The difficulty involves achieving precise row alignment across [PITH_FULL_IMAGE:figures/full_fig_p037_9.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization results of the US-stock image. The difficulty involves achieving precise row alignment across [PITH_FULL_IMAGE:figures/full_fig_p038_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization results of the arXiv paper image. The challenge is accurately preserving the structure of [PITH_FULL_IMAGE:figures/full_fig_p038_10.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization results of the arXiv paper image. The challenge is accurately preserving the structure of [PITH_FULL_IMAGE:figures/full_fig_p039_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization results of the magazine image. The main difficulty is recovering the correct reading order [PITH_FULL_IMAGE:figures/full_fig_p038_11.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization results of the magazine image. The main difficulty is recovering the correct reading order [PITH_FULL_IMAGE:figures/full_fig_p039_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Visualization results of the old scanned math image. The challenge lies in recognizing text and symbols [PITH_FULL_IMAGE:figures/full_fig_p039_12.png] view at source ↗
Figure 12
Figure 12. Figure 12: Visualization results of the old scanned math image. The challenge lies in recognizing text and symbols [PITH_FULL_IMAGE:figures/full_fig_p040_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Visualization results of the text parsing task. The challenge lies in faithfully transcribing densely written text [PITH_FULL_IMAGE:figures/full_fig_p039_13.png] view at source ↗
Figure 13
Figure 13. Figure 13: Visualization results of the text parsing task. The challenge lies in faithfully transcribing densely written text [PITH_FULL_IMAGE:figures/full_fig_p040_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Visualization results of the table parsing task. The difficulty involves reconstructing multi-level grouped [PITH_FULL_IMAGE:figures/full_fig_p040_14.png] view at source ↗
Figure 14
Figure 14. Figure 14: Visualization results of the table parsing task. The difficulty involves reconstructing multi-level grouped [PITH_FULL_IMAGE:figures/full_fig_p041_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Visualization results of the formula parsing task. The challenge is accurately transcribing a multi-line [PITH_FULL_IMAGE:figures/full_fig_p040_15.png] view at source ↗
Figure 15
Figure 15. Figure 15: Visualization results of the formula parsing task. The challenge is accurately transcribing a multi-line [PITH_FULL_IMAGE:figures/full_fig_p041_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Visualization results of the Chart-to-Table task. The difficulty lies in reading the numeric value of each [PITH_FULL_IMAGE:figures/full_fig_p040_16.png] view at source ↗
Figure 16
Figure 16. Figure 16: Visualization results of the Chart-to-Table task. The difficulty lies in reading the numeric value of each [PITH_FULL_IMAGE:figures/full_fig_p041_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Visualization results of the Chart-to-JSON task. The challenge is extracting the title, axis labels, and [PITH_FULL_IMAGE:figures/full_fig_p041_17.png] view at source ↗
Figure 17
Figure 17. Figure 17: Visualization results of the Chart-to-JSON task. The challenge is extracting the title, axis labels, and [PITH_FULL_IMAGE:figures/full_fig_p042_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Visualization results of the Chart-to-Code task. The difficulty involves inferring the chart type, underlying [PITH_FULL_IMAGE:figures/full_fig_p041_18.png] view at source ↗
Figure 18
Figure 18. Figure 18: Visualization results of the Chart-to-Code task. The difficulty involves inferring the chart type, underlying [PITH_FULL_IMAGE:figures/full_fig_p042_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Visualization results of the Chemical-to-SMILES task. The challenge is recognizing atoms, bonds, and [PITH_FULL_IMAGE:figures/full_fig_p041_19.png] view at source ↗
Figure 19
Figure 19. Figure 19: Visualization results of the Chemical-to-SMILES task. The challenge is recognizing atoms, bonds, and [PITH_FULL_IMAGE:figures/full_fig_p042_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Visualization results of the DocVQA task. The challenge lies in locating the relevant table within a dense [PITH_FULL_IMAGE:figures/full_fig_p042_20.png] view at source ↗
Figure 20
Figure 20. Figure 20: Visualization results of the DocVQA task. The challenge lies in locating the relevant table within a dense [PITH_FULL_IMAGE:figures/full_fig_p043_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Visualization results of the InfoVQA task. The difficulty lies in reasoning over a visually complex infographic [PITH_FULL_IMAGE:figures/full_fig_p043_21.png] view at source ↗
Figure 21
Figure 21. Figure 21: Visualization results of the InfoVQA task. The difficulty lies in reasoning over a visually complex infographic [PITH_FULL_IMAGE:figures/full_fig_p044_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Visualization results of the MathVista task. The challenge is jointly interpreting the geometric figure and [PITH_FULL_IMAGE:figures/full_fig_p044_22.png] view at source ↗
Figure 22
Figure 22. Figure 22: Visualization results of the MathVista task. The challenge is jointly interpreting the geometric figure and [PITH_FULL_IMAGE:figures/full_fig_p045_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Visualization results of the MMBench task. The difficulty lies in fine-grained spatial perception and [PITH_FULL_IMAGE:figures/full_fig_p044_23.png] view at source ↗
Figure 23
Figure 23. Figure 23: Visualization results of the MMBench task. The difficulty lies in fine-grained spatial perception and [PITH_FULL_IMAGE:figures/full_fig_p045_23.png] view at source ↗

Review history (3 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

138 extracted references · 15 linked inside Pith

  1. [1]

    Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, et al. Mineru2. 5: A decoupled vision-language model for efficient high-resolution document parsing.arXiv preprint arXiv:2509.22186, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Paddleocr-vl: Boosting multilingual document parsing via a 0.9 b ultra-compact vision-language model.arXiv preprint arXiv:2510.14528, 2025

    Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, et al. Paddleocr-vl: Boosting multilingual document parsing via a 0.9 b ultra-compact vision-language model.arXiv preprint arXiv:2510.14528, 2025

    arXiv 2025

  3. [3]

    Deepseek-ocr: Contexts optical compression.arXiv preprint arXiv:2510.18234, 2025

    Haoran Wei, Yaofeng Sun, and Yukun Li. Deepseek-ocr: Contexts optical compression.arXiv preprint arXiv:2510.18234, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Infinity parser: Layout aware reinforcement learning for scanned document parsing.arXiv preprint arXiv:2506.03197, 2025

    Baode Wang, Biao Wu, Weizhen Li, Meng Fang, Yanjie Liang, Zuming Huang, Haozhe Wang, Jun Huang, Ling Chen, Wei Chu, et al. Infinity parser: Layout aware reinforcement learning for scanned document parsing.arXiv preprint arXiv:2506.03197, 2025

    arXiv 2025

  5. [5]

    Logics-parsing technical report.arXiv preprint arXiv:2509.19760, 2025

    Xiangyang Chen, Shuzhao Li, Xiuwen Zhu, Yongfan Chen, Fan Yang, Cheng Fang, Lin Qu, Xiaoxiao Xu, Hu Wei, and Minggang Wu. Logics-parsing technical report.arXiv preprint arXiv:2509.19760, 2025

    arXiv 2025

  6. [6]

    olmocr 2: Unit test rewards for document ocr.arXiv preprint arXiv:2510.19817, 2025

    Jake Poznanski, Luca Soldaini, and Kyle Lo. olmocr 2: Unit test rewards for document ocr.arXiv preprint arXiv:2510.19817, 2025

    arXiv 2025

  7. [7]

    dots.ocr: Multilingual document layout parsing in a single vision-language model

    Rednote HiLab. dots.ocr: Multilingual document layout parsing in a single vision-language model. https: //github.com/rednote-hilab/dots.ocr, 2025

    2025

  8. [8]

    Multimodal ocr: Parse anything from documents, 2026

    Handong Zheng, Yumeng Li, Kaile Zhang, Liang Xin, Guangwei Zhao, Hao Liu, Jiayu Chen, Jie Lou, Jiyu Qiu, Qi Fu, Rui Yang, Shuo Jiang, Weijian Luo, Weijie Su, Weijun Zhang, Xingyu Zhu, Yabin Li, Yiwei ma, Yu Chen, Zhaohui Yu, Guang Yang, Colin Zhang, Lei Zhang, Yuliang Liu, and Xiang Bai. Multimodal ocr: Parse anything from documents, 2026. 24

    2026

  9. [9]

    Deepseek-ocr 2: Visual causal flow.arXiv preprint arXiv:2601.20552, 2026

    Haoran Wei, Yaofeng Sun, and Yukun Li. Deepseek-ocr 2: Visual causal flow.arXiv preprint arXiv:2601.20552, 2026

    arXiv 2026

  10. [10]

    Table2latex-rl: High-fidelity latex code generation from table images via reinforced multimodal language models.arXiv preprint arXiv:2509.17589, 2025

    Jun Ling, Yao Qi, Tao Huang, Shibo Zhou, Yanqin Huang, Jiang Yang, Ziqi Song, Ying Zhou, Yang Yang, Heng Tao Shen, et al. Table2latex-rl: High-fidelity latex code generation from table images via reinforced multimodal language models.arXiv preprint arXiv:2509.17589, 2025

    arXiv 2025

  11. [11]

    TextDiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. TextDiffuser: Diffusion models as text painters. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  12. [12]

    AnyText: Multilingual visual text generation and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. AnyText: Multilingual visual text generation and editing. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  13. [13]

    TableFormer: Table structure understanding with transformers

    Ahmed Nassar, Nikolaos Livathinos, Maksym Lysak, and Peter Staar. TableFormer: Table structure understanding with transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  14. [14]

    Scaling text-rich image understanding via code-guided synthetic multimodal data generation

    Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison- Burch, Ranjay Krishna, Aniruddha Kembhavi, and Christopher Clark. Scaling text-rich image understanding via code-guided synthetic multimodal data generation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL)...

    2025

  15. [15]

    DECIMER: Towards deep learning for chemical image recognition.Journal of Cheminformatics, 12(1):65, 2020

    Kohulan Rajan, Achim Zielesny, and Christoph Steinbeck. DECIMER: Towards deep learning for chemical image recognition.Journal of Cheminformatics, 12(1):65, 2020

    2020

  16. [16]

    Nougat: Neural optical understanding for academic documents

    Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  17. [17]

    Vary: Scaling up the vision vocabulary for large vision-language models

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Vary: Scaling up the vision vocabulary for large vision-language models. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  18. [18]

    Wikimedia downloads

    Wikimedia Foundation. Wikimedia downloads. https://dumps.wikimedia.org, 2026. [Online; accessed 30-June-2026]

    2026

  19. [19]

    Image-based table recognition: Data, model, and evaluation

    Xu Zhong, Elaheh ShafieiBavani, and Antonio Jimeno Yepes. Image-based table recognition: Data, model, and evaluation. InComputer Vision – ECCV 2020, volume 12366 ofLecture Notes in Computer Science, pages 564–580. Springer, 2020

    2020

  20. [21]

    Ng, Bo Pang, Piyush Sharma, and Radu Soricut

    Edwin G. Ng, Bo Pang, Piyush Sharma, and Radu Soricut. Understanding guided image captioning performance across domains, 2021

    2021

  21. [22]

    M 6doc: A large-scale multi-format, multi-type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis, 2023

    Hiuyi Cheng, Peirong Zhang, Sihang Wu, Jiaxin Zhang, Qiyuan Zhu, Zecheng Xie, Jing Li, Kai Ding, and Lianwen Jin. M 6doc: A large-scale multi-format, multi-type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis, 2023

    2023

  22. [23]

    Global table extractor (GTE): A framework for joint table identification and cell structure recognition using visual context

    Xinyi Zheng, Douglas Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. Global table extractor (GTE): A framework for joint table identification and cell structure recognition using visual context. In2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 697–706. IEEE, 2021

    2021

  23. [24]

    Multimodal table understanding

    Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. Multimodal table understanding. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9102–9124, Bangkok, Thailand, 2024. Association for Computational Linguistics

    2024

  24. [25]

    Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M. Rush. Image-to-markup generation with coarse-to-fine attention. InProceedings of the 34th International Conference on Machine Learning (ICML), volume 70 ofProceedings of Machine Learning Research, pages 980–989. PMLR, 2017

    2017

  25. [26]

    When counting meets HMER: Counting-aware network for handwritten mathematical expression recognition

    Bohan Li, Ye Yuan, Dingkang Liang, Xiao Liu, Zhilong Ji, Jinfeng Bai, Wenyu Liu, and Xiang Bai. When counting meets HMER: Counting-aware network for handwritten mathematical expression recognition. InComputer Vision – ECCV 2022, volume 13688 ofLecture Notes in Computer Science, pages 197–214. Springer, 2022

    2022

  26. [27]

    ICDAR 2019 CROHME + TFD: Competition on recognition of handwritten mathematical expressions and typeset formula detection

    Mahshad Mahdavi, Richard Zanibbi, Harold Mouchère, Christian Viard-Gaudin, and Utpal Garain. ICDAR 2019 CROHME + TFD: Competition on recognition of handwritten mathematical expressions and typeset formula detection. In2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1533–1538. IEEE, 2019. 25

    2019

  27. [28]

    ChartAssistant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning

    Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. ChartAssistant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. In Findings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, 2024

    2024

  28. [29]

    ChartMoE: Mixture of diversely aligned expert connector for chart understanding

    Zhengzhuo Xu, Bowen Qu, Yiyan Qi, Sinan Du, Chengjin Xu, Chun Yuan, and Jian Guo. ChartMoE: Mixture of diversely aligned expert connector for chart understanding. InThe Thirteenth International Conference on Learning Representations (ICLR), 2025

    2025

  29. [30]

    UniChart: A universal vision- language pretrained model for chart comprehension and reasoning

    Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. UniChart: A universal vision- language pretrained model for chart comprehension and reasoning. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 14662–14684, Singapore, 2023. Association for Computational Linguistics

    2023

  30. [31]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. InFindings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, Dublin, Ireland, May 2022. Association for Computational Linguistics

    2022

  31. [32]

    ChartGen: Scaling chart understanding via code-guided synthetic chart generation.arXiv preprint arXiv:2507.19492, 2025

    Jovana Kondic, Pengyuan Li, Dhiraj Joshi, Zexue He, Shafiq Abedin, Jennifer Sun, Ben Wiesel, Eli Schwartz, Ahmed Nassar, Bo Wu, Assaf Arbelle, Aude Oliva, Dan Gutfreund, Leonid Karlinsky, and Rogerio Feris. ChartGen: Scaling chart understanding via code-guided synthetic chart generation.arXiv preprint arXiv:2507.19492, 2025

    Pith/arXiv arXiv 2025

  32. [33]

    ChartCoder: Advancing multimodal large language model for chart-to-code generation

    Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Wanxiang Che, Zhiyuan Liu, and Maosong Sun. ChartCoder: Advancing multimodal large language model for chart-to-code generation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics, 2025

    2025

  33. [34]

    Minesh Mathew, Dimosthenis Karatzas, and C. V . Jawahar. DocVQA: A dataset for VQA on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 2200–2209, 2021

    2021

  34. [35]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. InComputer Vision – ECCV 2016, volume 9908 ofLecture Notes in Computer Science, pages 235–251. Springer, 2016

    2016

  35. [36]

    mPLUG-DocOwl 1.5: Unified structure learning for OCR-free document understanding.arXiv preprint arXiv:2403.12895, 2024

    Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mPLUG-DocOwl 1.5: Unified structure learning for OCR-free document understanding.arXiv preprint arXiv:2403.12895, 2024

    Pith/arXiv arXiv 2024

  36. [37]

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V . Jawahar. Infograph- icVQA. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1697–1706, 2022

    2022

  37. [38]

    Exploring the capabilities of large multimodal models on dense text.arXiv preprint arXiv:2405.06706, 2024

    Shuo Zhang, Biao Yang, Zhang Li, Zhiyin Ma, Yuliang Liu, and Xiang Bai. Exploring the capabilities of large multimodal models on dense text.arXiv preprint arXiv:2405.06706, 2024

    Pith/arXiv arXiv 2024

  38. [39]

    TinyChart: Efficient chart understanding with visual token merging and program-of-thoughts learning.arXiv preprint arXiv:2404.16635, 2024

    Liang Zhang, Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, and Fei Huang. TinyChart: Efficient chart understanding with visual token merging and program-of-thoughts learning.arXiv preprint arXiv:2404.16635, 2024

    Pith/arXiv arXiv 2024

  39. [40]

    LLaV A-NeXT- Interleave: Tackling multi-image, video, and 3d in large multimodal models.arXiv preprint arXiv:2407.07895, 2024

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. LLaV A-NeXT- Interleave: Tackling multi-image, video, and 3d in large multimodal models.arXiv preprint arXiv:2407.07895, 2024

    Pith/arXiv arXiv 2024

  40. [41]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26296–26306, 2024

    2024

  41. [42]

    ShareGPT4V: Improving large multi-modal models with better captions

    Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. ShareGPT4V: Improving large multi-modal models with better captions. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  42. [43]

    ShareGPT-4o

    OpenGVLab. ShareGPT-4o. https://huggingface.co/datasets/OpenGVLab/ShareGPT-4o, 2024. Hug- ging Face dataset

    2024

  43. [44]

    CogVLM: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. CogVLM: Visual expert for pretrained language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2024. 26

    2024

  44. [45]

    ALLaV A: Harnessing gpt4v-synthesized data for lite vision-language models.arXiv preprint arXiv:2402.11684, 2024

    Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. ALLaV A: Harnessing gpt4v-synthesized data for lite vision-language models.arXiv preprint arXiv:2402.11684, 2024

    Pith/arXiv arXiv 2024

  45. [46]

    To see is to believe: Prompting GPT-4V for better visual instruction tuning.arXiv preprint arXiv:2311.07574, 2023

    Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. To see is to believe: Prompting GPT-4V for better visual instruction tuning.arXiv preprint arXiv:2311.07574, 2023

    Pith/arXiv arXiv 2023

  46. [47]

    Towards VQA models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards VQA models that can read. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8317–8326, 2019

    2019

  47. [48]

    OCR-VQA: Visual question answering by reading text in images

    Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. OCR-VQA: Visual question answering by reading text in images. InInternational Conference on Document Analysis and Recognition (ICDAR), pages 947–952, 2019

    2019

  48. [49]

    The chemistry development kit (CDK): An open-source java library for chemo- and bioinformatics.Journal of Chemical Information and Computer Sciences, 43(2):493–500, 2003

    Christoph Steinbeck, Yongquan Han, Stefan Kuhn, Oliver Horlacher, Edgar Luttmann, and Egon Willighagen. The chemistry development kit (CDK): An open-source java library for chemo- and bioinformatics.Journal of Chemical Information and Computer Sciences, 43(2):493–500, 2003

    2003

  49. [50]

    RDKit: Open-source cheminformatics.https://www.rdkit.org

    Greg Landrum. RDKit: Open-source cheminformatics.https://www.rdkit.org. Accessed: 2026-06-01

    2026

  50. [51]

    OpenChemLib: Open source java-based chemistry library

    Thomas Sander et al. OpenChemLib: Open source java-based chemistry library. https://github.com/ Actelion/openchemlib, 2024. Idorsia Pharmaceuticals Ltd. / Openmolecules.org. Accessed: 2026-06-01

    2024

  51. [52]

    Indigo toolkit: Universal cheminformatics toolkit, utilities and database search tools

    EPAM Systems, Inc. Indigo toolkit: Universal cheminformatics toolkit, utilities and database search tools. https: //lifescience.opensource.epam.com/indigo/. https://github.com/epam/Indigo. Accessed: 2026- 06-01

    2026

  52. [53]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026

    2026

  53. [54]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. InProceedings of the European conference on computer vision (ECCV), pages 801–818, 2018

    2018

  54. [55]

    Image-based table recognition: data, model, and evaluation

    Xu Zhong, Elaheh ShafieiBavani, and Antonio Jimeno Yepes. Image-based table recognition: data, model, and evaluation. InEuropean conference on computer vision, pages 564–580. Springer, 2020

    2020

  55. [56]

    Image over text: Transforming formula recognition evaluation with character detection matching, 2025

    Bin Wang, Fan Wu, Linke Ouyang, Zhuangcheng Gu, Rui Zhang, Renqiu Xia, Bo Zhang, and Conghui He. Image over text: Transforming formula recognition evaluation with character detection matching, 2025

    2025

  56. [57]

    Deplot: One-shot visual language reasoning by plot-to-table translation

    Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual language reasoning by plot-to-table translation. InFindings of the Association for Computational Linguistics: ACL 2023, pages 10381–10399, 2023

    2023

  57. [58]

    Structchart: Perception, structuring, reasoning for visual chart understanding.arXiv preprint arXiv:2309.11268, 2023

    Renqiu Xia, Bo Zhang, Haoyang Peng, Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Yu Qiao, and Junchi Yan. Structchart: Perception, structuring, reasoning for visual chart understanding.arXiv preprint arXiv:2309.11268, 2023

    Pith/arXiv arXiv 2023

  58. [59]

    An elementary mathematical theory of classification and prediction

    Taffee T Tanimoto. An elementary mathematical theory of classification and prediction. 1958

    1958

  59. [60]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    2024

  60. [61]

    Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020

    2020

  61. [62]

    Swift: a scalable lightweight infrastructure for fine-tuning

    Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. Swift: a scalable lightweight infrastructure for fine-tuning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 29733–29735, 2025

    2025

  62. [63]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297, 2025

    2025

  63. [64]

    Lmms-eval: Reality check on the evaluation of large multimodal models

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Reality check on the evaluation of large multimodal models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 881–916, 2025

    2025

  64. [65]

    olmocr: Unlocking trillions of tokens in pdfs with vision language models

    Jake Poznanski, Aman Rangapur, Jon Borchardt, Jason Dunkelberger, Regan Huff, Daniel Lin, Christopher Wilhelm, Kyle Lo, and Luca Soldaini. olmocr: Unlocking trillions of tokens in pdfs with vision language models. arXiv preprint arXiv:2502.18443, 2025. 27

    arXiv 2025

  65. [66]

    Acosta, Preston Carlson, Sacha Bron, Pierre-Loïc Doulcet, Daniel B

    Boyang Zhang, Sebastián G. Acosta, Preston Carlson, Sacha Bron, Pierre-Loïc Doulcet, Daniel B. Ospina, and Simon Suo. Parsebench: A document parsing benchmark for ai agents, 2026

    2026

  66. [67]

    Omnidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations

    Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, et al. Omnidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24838–24848, 2025

    2025

  67. [68]

    Marker: Convert pdf to markdown and json quickly with high accuracy

    Vik Paruchuri. Marker: Convert pdf to markdown and json quickly with high accuracy. https://github.com/ datalab-to/marker, 2024

    2024

  68. [69]

    Nanonets-ocr-s: An image-to-markdown ocr model

    Nanonets. Nanonets-ocr-s: An image-to-markdown ocr model. https://huggingface.co/nanonets/ Nanonets-OCR-s, 2025

    2025

  69. [70]

    Lightonocr: A 1b end-to-end multilingual vision- language model for state-of-the-art ocr.arXiv preprint arXiv:2601.14251, 2026

    Said Taghadouini, Adrien Cavaillès, and Baptiste Aubertin. Lightonocr: A 1b end-to-end multilingual vision- language model for state-of-the-art ocr.arXiv preprint arXiv:2601.14251, 2026

    Pith/arXiv arXiv 2026

  70. [71]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  71. [72]

    Chandra-ocr-2.https://huggingface.co/datalab-to/chandra-ocr-2, 2026

    Datalab. Chandra-ocr-2.https://huggingface.co/datalab-to/chandra-ocr-2, 2026

    2026

  72. [73]

    Glm-ocr technical report.arXiv preprint arXiv:2603.10910, 2026

    Zhipu AI. Glm-ocr technical report.arXiv preprint arXiv:2603.10910, 2026

    arXiv 2026

  73. [74]

    Doclaynet: A large human- annotated dataset for document-layout segmentation

    Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S Nassar, and Peter Staar. Doclaynet: A large human- annotated dataset for document-layout segmentation. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 3743–3751, 2022

    2022

  74. [75]

    Vision grid transformer for document layout analysis

    Cheng Da, Chuwei Luo, Qi Zheng, and Cong Yao. Vision grid transformer for document layout analysis. InICCV, 2023

    2023

  75. [76]

    Pp-doclayoutv2: A document layout detection model

    PaddlePaddle. Pp-doclayoutv2: A document layout detection model. https://huggingface.co/ PaddlePaddle/PP-DocLayoutV2, 2025

    2025

  76. [77]

    Pp-doclayoutv3: A document layout detection model

    PaddlePaddle. Pp-doclayoutv3: A document layout detection model. https://huggingface.co/ PaddlePaddle/PP-DocLayoutV3, 2026

    2026

  77. [78]

    Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception.arXiv preprint arXiv:2410.12628, 2024

    Zhiyuan Zhao, Hengrui Kang, Bin Wang, and Conghui He. Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception.arXiv preprint arXiv:2410.12628, 2024

    Pith/arXiv arXiv 2024

  78. [79]

    Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context

    Xinyi Zheng, Douglas Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 697–706, 2021

    2021

  79. [80]

    Unimernet: A universal network for real-world mathematical expression recognition.arXiv preprint arXiv:2404.15254, 2024

    Bin Wang, Zhuangcheng Gu, Guang Liang, Chao Xu, Bo Zhang, Botian Shi, and Conghui He. Unimernet: A universal network for real-world mathematical expression recognition.arXiv preprint arXiv:2404.15254, 2024

    Pith/arXiv arXiv 2024

  80. [81]

    Chart-to-text: A large-scale benchmark for chart summarization

    Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. Chart-to-text: A large-scale benchmark for chart summarization. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4005–4023, 2022

    2022

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.