REVIEW 5 major objections 7 minor 2 cited by
DocFusion: A Unified Framework for Document Parsing Tasks
T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A single 289M generative model matches specialized document parsers across four core tasks.
desk verdict A genuinely useful unified parsing model and dataset, but the SOTA claim fails its own DLA benchmark and the recognition numbers need a decontamination check before they can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is Gaussian-Kernel Cross-Entropy Loss (GK-CEL), a modified objective for generative models that output quantized coordinates as vocabulary tokens. After computing the softmax distribution, the method zeros out probabilities outside the coordinate-token index range, applies a one-dimensional convolution whose weights follow a Gaussian, reinserts the convolved values inside that range, and then evaluates cross-entropy only at valid target positions. This smoothing redistributes gradient signal from a target coordinate token to nearby location tokens, counteracting the label jitter that makes cross-entropy treat continuous values as unrelated discrete classes. It is supported by two secondary mechanisms: coordinate quantization, which turns bounding boxes into loc tokens so detection and recognition share one decoder, and the DocLatex-1.6M dataset, whose standardized LaTeX targets reduce annotation noise for math and table outputs.
What would settle it
Run exact and fuzzy string overlap between DocLatex-1.6M training samples and the UniMER-1M and DocGenome test samples, then retrain DocFusion with all near-duplicates removed; if the reported ExpRate, CSR, and F1 margins shrink materially, the benchmark gains are not due to the architecture or loss alone.
Extended reading notes
Core claim
The central claim is that continuous coordinate detection and discrete token recognition can be trained together in one autoregressive decoder if the objective treats neighboring coordinate tokens as related rather than as independent classes. Under standard cross-entropy, small labeling noise spreads probability across adjacent location tokens and produces weak or skewed gradients, and in multi-task training the coordinate prediction can dominate the loss. GK-CEL convolves the softmax distribution restricted to the coordinate-token range with a Gaussian kernel, merges the smoothed values back into the distribution, and then computes cross-entropy, so small coordinate shifts still yield informative gradients. On the paper's evidence this change alone raises recognition quality, LaTeX compile-success rates, table F1, and layout-detection F1 relative to plain cross-entropy, and joint training with OCR data improves DLA F1 over detection-only training. The paper concludes that recognition and detection are mutually beneficial, not competing, when the loss no longer forces a false choice between continuous and discrete targets.
Load-bearing premise
The comparisons assume that formulas and tables in the DocLatex-1.6M training set do not substantially reappear in the UniMER-1M and DocGenome test sets, and the paper reports no decontamination analysis for that overlap.
Editorial extensions
If this is right
- One 289M model can replace a four-model parsing pipeline for layout, math, tables, and OCR, cutting integration and maintenance overhead.
- Recognition data strengthens detection: systems trained jointly on OCR plus layout should beat detection-only systems on the same pages.
- Generative detection without confidence scores or NMS becomes practical, since GK-CEL stabilizes coordinate gradients enough for direct token prediction.
- Standardized, shorter LaTeX targets (about 34.2% shorter for complex and 37.5% shorter for simple expressions on the UniMER-1M test set) reduce autoregressive decoding cost for math and table outputs.
- On the paper's benchmarks, a 289M unified model matches or beats much larger OCR and recognition systems, so parameter count alone is not the binding constraint.
Reading between the lines
- If GK-CEL generalizes, the same Gaussian smoothing should transfer to other generative models with quantized continuous outputs—keypoint detection, segmentation masks, time-series forecasting—where coordinate jitter currently weakens cross-entropy gradients; replacing soft-argmax/MSE hybrids with GK-CEL on one such task would test this directly.
- The paper's collaboration result predicts a scaling pattern: adding further recognition tasks (charts, chemical formulas, handwriting) to the same 289M decoder should keep improving layout F1 and LaTeX compile rates, which is an incremental experiment rather than a new architecture.
- The DocLatex-1.6M standardization recipe can be applied to other LaTeX-heavy benchmarks, and measuring token-length reduction and compile-rate change after re-annotation would quantify how much of the reported gain comes from cleaner targets versus the model itself.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DocFusion, a unified generative document parsing model with 289M parameters that jointly handles document layout analysis (DLA), math expression recognition (MER), table recognition (TR), and OCR. The central methodological contribution is Gaussian-Kernel Cross-Entropy Loss (GK-CEL), which smooths coordinate-token probabilities to reconcile continuous coordinates with discrete token generation. The authors also introduce DocLatex-1.6M, a large-scale LaTeX dataset extracted from arXiv sources. Experiments report strong results on four tasks, with ablations supporting the benefits of multi-task training and GK-CEL.
Significance. If validated, the paper makes a practically valuable contribution: a single lightweight model covering four document parsing tasks, with released code and data. The multi-task collaboration result and the GK-CEL ablation are useful empirical findings. However, the SOTA claim is contradicted by the paper's own DLA numbers, and the MER/TR benchmarks face unresolved data-contamination concerns. The correct assessment of the contribution hinges on the benchmark validity and on a precise specification of GK-CEL.
major comments (5)
- [Abstract and Table 3] The abstract claims SOTA performance across four key tasks, but Table 3 reports DocFusion DLA F1 of 88.4 on the full DocLayNet test set versus 89.5 for YOLO-DocLayout and 89.3 for Deformable-DETR. Section 4.5.4 only claims "strong performance" for DLA, so the abstract and Section 1 overstate the results. Please revise the SOTA claim or provide evidence that the comparison is unfair to DocFusion beyond what is already stated.
- [Appendix A.2 and Sections 4.5.1-4.5.2] DocLatex-1.6M is constructed by extracting LaTeX from arXiv source files, and the MER and TR test sets (UniMER-1M and DocGenome) are also derived from arXiv documents. The paper reports no decontamination analysis such as hashing LaTeX strings or matching arXiv identifiers to ensure the training and test sets are disjoint. Because the model is generative and trained on standardized LaTeX, near-duplicate formulas or tables from the same source papers could be memorized, directly inflating ExpRate, CSR, and cell-level F1. Please provide overlap statistics or demonstrate that the test samples were not seen during training.
- [Sections 4.1 and 4.5.3] The paper states that DocLayNet "offers comprehensive annotations for layout elements and their corresponding text," but the publicly available DocLayNet dataset provides layout bounding boxes and category labels, not text transcriptions. The OCR evaluation on 3,000 English samples from DocLayNet therefore lacks a clearly described ground-truth source. Please specify how the OCR text labels were obtained or re-annotated; without this, the reported BLEU and EditDis results are not verifiable.
- [Section 3.3, Eqs. (3)-(6)] The Gaussian kernel K is defined without normalization, and the convolution output C is substituted into P within the coordinate-token range [s,e] without renormalizing the resulting P''. As written, P'' may not sum to 1, and the cross-entropy in Eq. (6) can take negative values for confident predictions, breaking the standard probabilistic interpretation. Please state whether the kernel or the resulting distribution is normalized in the actual implementation and provide the exact forward computation; otherwise the theoretical grounding of GK-CEL is incomplete.
- [Tables 3 and 4] Table 4 reports DLA F1 of 88.9 for the OCR+DLA configuration, while the full model, which includes all four tasks, has DLA F1 of 88.4 in Table 3. The full four-task configuration is not included in Table 4, so the claim that "integrating recognition data significantly enhances detection performance" is not demonstrated for the actual proposed model. Please include the full-model row in Table 4 and discuss the apparent drop in DLA F1 when MER and TR are added.
minor comments (7)
- [Section 4.5.1 and Table 2] The text says DocFusion's ExpRate surpasses UniMERNet by 5.2%, but Table 2 reports 94.2 versus 89.5, a difference of 4.7 percentage points; please reconcile the numbers.
- [Section 4.5.2 and Table 2] The text says the TR F1 surpasses the second-ranked model by 1.6%, but Table 2 shows 92.1 versus 90.6, a difference of 1.5 percentage points; please correct the value.
- [Section 2] The phrase "multi recognition tasks" should be "multiple recognition tasks."
- [Acknowledgements] There is a typo: "TThe authors" should be "The authors."
- [Appendix C.3 and Table 9] The text states that results for the larger 738M variant "will be included in a future version," but Table 9 already reports DocFusion-large results; this is inconsistent and should be clarified.
- [Table 9 caption] The caption reads "Ablation analysis of Gaussian-Kernel Cross-Entropy Loss" but the table compares model sizes; the caption appears to be copied from Table 5 and should be rewritten.
- [Appendix C.1-C.2] The alternative optimization strategies (data-ratio/loss-weight adjustments and soft-argmax) are described only qualitatively without quantitative results; adding the best observed numbers would strengthen the comparison with GK-CEL.
Circularity Check
No significant circularity: the central loss design is an empirically tested objective, and the paper's claims rest on external benchmarks and controlled ablations rather than on self-citation or fitted predictions.
full rationale
The paper's load-bearing contribution, GK-CEL, is defined explicitly in Equations 1-6 and compared empirically against standard cross-entropy in Table 5; it is not derived from the results it is used to explain, and no fitted parameter is renamed as a prediction. The unified coordinate-token representation is attributed to prior external work (Florence-2, Xiao et al., 2023) rather than to an author-specific uniqueness theorem, and no load-bearing claim is justified by self-citation. Multi-task collaboration claims are supported by controlled ablations in Table 4, and the OCR and DLA results use the external DocLayNet benchmark. The SOTA claim for MER and TR does depend on the independence of the self-constructed DocLatex-1.6M training data from the UniMER-1M and DocGenome test sets, and the paper reports no decontamination analysis; this is a genuine benchmark-validity risk, but it is not circularity of the derivation chain because no equation or fitted quantity reduces the reported scores to the training input by construction. Under the hard rules, such data-hygiene concerns belong in a correctness-risk assessment, not in the circularity score.
Assumptions & free parameters
free parameters (4)
- Gaussian kernel size n
- Gaussian standard deviation sigma
- Coordinate quantization resolution =
1000x1000
- Multi-task data mixing ratios and loss weights
assumptions (4)
- domain assumption Coordinate quantization to discrete tokens preserves enough spatial fidelity for layout analysis.
- ad hoc to paper Gaussian smoothing of coordinate-token probabilities improves gradient quality and multi-task balance.
- domain assumption DocLatex-1.6M, extracted from arXiv LaTeX sources and re-rendered, is a valid proxy for real document images and does not overlap with the test sets.
- domain assumption DocLayNet OCR annotations are accurate enough for training and evaluation.
Cite this review
Pith. "Pith review of DocFusion: A Unified Framework for Document Parsing Tasks." pith.science (2026). https://pith.science/paper/UNIXW44T
@misc{pith2026241212505,
author = {Pith},
title = {Pith review of: DocFusion: A Unified Framework for Document Parsing Tasks},
year = {2026},
howpublished = {\url{https://pith.science/paper/UNIXW44T}},
note = {Machine review of arXiv:2412.12505}
}
read the original abstract
Document parsing is essential for analyzing complex document structures and extracting fine-grained information, supporting numerous downstream applications. However, existing methods often require integrating multiple independent models to handle various parsing tasks, leading to high complexity and maintenance overhead. To address this, we propose DocFusion, a lightweight generative model with only 0.28B parameters. It unifies task representations and achieves collaborative training through an improved objective function. Experiments reveal and leverage the mutually beneficial interaction among recognition tasks, and integrating recognition data significantly enhances detection performance. The final results demonstrate that DocFusion achieves state-of-the-art (SOTA) performance across four key tasks.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 2 Pith papers
-
Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting
A two-stage model that analyzes page layout first, then parses text, tables, and formulas in parallel, reports state-of-the-art accuracy and speed on the benchmarks it evaluates.
-
Improving RL Exploration for LLM Reasoning through Retrospective Replay
RRL replays value-model-selected promising intermediate states during LLM RL training, preserving exploration and improving performance on code, math, and RLHF tasks.
Reference graph
Works this paper leans on
-
[3]
Nougat: Neural optical understanding for academic documents. Preprint, arXiv:2308.13418. Michal Bušta, Lukàš Neumann, and Jirí Matas
-
[7]
Davit: Dual attention vision transformers. Preprint, arXiv:2204.03645. Sergey Filimonov
-
[9]
arXiv preprint arXiv:2204.08387
Layoutlmv3: Pre-training for document ai with unified text and image masking (2022). arXiv preprint arXiv:2204.08387. Rahima Khanam and Muhammad Hussain
arXiv 2022
-
[10]
Yolov11: An overview of the key architectural enhancements. Preprint, arXiv:2410.17725. Anh Duc Le, Bipin Indurkhya, and Masaki Nakagawa
-
[11]
Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. Preprint, arXiv:2407.07895. Jerry Liu
-
[12]
Textmonkey: An ocr-free large multimodal model for understand- ing document. Preprint, arXiv:2403.04473. Renqing Luo and Yuhan Xu
-
[13]
Mixtex: Unambigu- ous recognition should not rely solely on real data. Preprint, arXiv:2406.17148. Zhiming Mao, Haoli Bai, Lu Hou, Jiansheng Wei, Xin Jiang, Qun Liu, and Kam-Fai Wong
-
[14]
arXiv preprint arXiv:2403.16516
Visually guided generative text-layout pre-training for docu- ment intelligence. arXiv preprint arXiv:2403.16516. Erik G Miller and Paul A Viola
Show all 27 references
-
[15]
Accessed: 2024-02-29, cited in pages 1, 2, 4, 6,
Texify. Accessed: 2024-02-29, cited in pages 1, 2, 4, 6,
2024
-
[17]
Preprint, arXiv:2110.07367
Rocketqav2: A joint training method for dense passage retrieval and passage re-ranking. Preprint, arXiv:2110.07367. R. Smith
-
[19]
Preprint, arXiv:2409.01704
General ocr theory: Towards ocr- 2.0 via a unified end-to-end model. Preprint, arXiv:2409.01704. Christoph Wick and Frank Puppe
-
[21]
arXiv preprint arXiv:2406.11633
Docgenome: An open large-scale scientific document benchmark for training and testing multi-modal large language models. arXiv preprint arXiv:2406.11633. Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan
-
[22]
URL https://arxiv
Florence-2: Advancing a unified representation for a variety of vision tasks (2023). URL https://arxiv. org/abs/2311.06242. Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, and Lidong Zhou
2023 arXiv
-
[23]
Preprint, arXiv:2012.14740
Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. Preprint, arXiv:2012.14740. Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Alex Lin, and Fei Huang
2012 arXiv
-
[24]
Preprint, arXiv:2310.05126
Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. Preprint, arXiv:2310.05126. Zhichang Yu
-
[25]
arXiv preprint arXiv:2203.01601
Syntax- aware network for handwritten mathematical expres- sion recognition. arXiv preprint arXiv:2203.01601. Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, and Weizhu Chen
-
[26]
Preprint, arXiv:2110.03611
Adversarial retriever-ranker for dense text retrieval. Preprint, arXiv:2110.03611. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ru...
-
[27]
However, Latex has not been the mainstream approach for TR tasks in recent times, resulting in a limited number of TR models available for comparison in the main experiment
B.2 Other Table-to-Sequence Method This study aims to explore multi-task collaboration, and therefore, the TR task also adopts Latex as the output format to maintain consistency with MER. However, Latex has not been the mainstream approach for TR tasks in recent times, resulti...
2022
-
[2007]
Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jungong Han, and Guiguang Ding
, volume 2, pages 629–633. Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jungong Han, and Guiguang Ding. 2024a. Yolov10: Real-time end-to-end object detection. Preprint, arXiv:2405.14458. Bin Wang, Zhuangcheng Gu, Guang Liang, Chao Xu, Bo Zhang, Botian Shi, and Conghui He...
-
[2017]
In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2223–2231
Deep textspotter: An end-to-end trainable scene text localization and recognition framework. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2223–2231. Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko
2017
-
[2018]
In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS)
Fully convolutional neural networks for page segmentation of historical document images. In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). Renqiu Xia, Song Mao, Xiangchao Yan, Hongbin Zhou, Bo Zhang, Haoyang Peng, Jiahao Pi, Daocheng Fu, Wenjie Wu, H...
2018
-
[2019]
In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 142–147
Rethinking table recognition using graph neural networks. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 142–147. IEEE. Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen
2019
-
[2020]
Preprint, arXiv:2005.12872
End-to-end object detection with transformers. Preprint, arXiv:2005.12872. Jingye Chen, Bin Li, and Xiangyang Xue
2005 arXiv
-
[2021]
In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Scene text telescope: Text-focused scene image super- resolution. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Mingyu Ding, Bin Xiao, Noel Codella, Ping Luo, Jingdong Wang, and Lu Yuan
2021
-
[2022]
Accessed: 2024-02-29, cited in pages 1, 2, 3, 7, 10,
pix2tex - latex ocr. Accessed: 2024-02-29, cited in pages 1, 2, 3, 7, 10,
2024
-
[2023]
arXiv preprint arXiv:2308.12966, 1(2):3
Qwen-vl: A versatile vision-language model for understanding, localiza- tion, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1(2):3. Hangbo Bao, Li Dong, and Furu Wei
-
[2024]
Preprint, arXiv:2312.10997
Retrieval-augmented generation for large language models: A survey. Preprint, arXiv:2312.10997. Yongshuai Huang, Ning Lu, Dapeng Chen, Yibo Li, Zecheng Xie, Shenggao Zhu, Liangcai Gao, and Wei Peng
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.