{"id":"3b0f3f98-9730-4e79-9e8d-2c38da58abfd","arxiv_id":"2412.12505","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A 289M-parameter generative model with a Gaussian-kernel cross-entropy loss jointly handles layout analysis, OCR, math expression recognition, and table recognition, with competitive but partially overstated benchmark gains.","lead":"DocFusion is a single 289-million-parameter model that performs document layout detection, OCR, math expression recognition, and table recognition together. Its proposed Gaussian-kernel loss smooths coordinate-token probabilities, but the claimed state-of-the-art results depend on unverified data separation and overstate the layout benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training and test sets share arXiv provenance with no decontamination check; reported MER/TR SOTA gains may be inflated by train/test overlap.","rationale":"The reader identified the same load-bearing concern: data separation between DocLatex-1.6M and the arXiv-derived test sets. I agree that this is the most fragile premise because it directly affects the two strongest empirical claims (MER ExpRate and TR F1) and cannot be repaired by re-evaluating with a different metric. The paper reports no decontamination analysis, and the construction pipeline in Appendix A.2 makes overlap highly plausible. The DLA SOTA claim is already internally contradicted by Table 3 (F1 88.4 vs 89.5), but that is a framing inconsistency. The data overlap threat, if real, invalidates the core empirical contribution. The proposed decontamination test is straightforward and would settle the matter. Since the reader's CONDITIONAL verdict already hinges on this concern, my assessment does not change the verdict. I therefore recommend UNCHANGED, with the condition that the authors provide the overlap analysis or re-evaluate on a decontaminated benchmark.","tokens_in":13545,"tokens_out":4038,"duration_ms":38397,"concrete_test":"Compute overlap between DocLatex-1.6M and the test sets by hashing normalized LaTeX strings. For MER, compare every training formula to the UniMER-1M SPE/CPE test formulas; for TR, extract table LaTeX from DocGenome test samples and compare against training table LaTeX. Also match arXiv source IDs if available. If any exact or near-duplicate (e.g., >90% character similarity) test item appears in training, re-run MER and TR evaluations on the decontaminated subset and report the metrics. If the gains over UniMERNet and StructEqTable disappear, the central SOTA claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DocFusion achieves SOTA on all four tasks rests on the MER and TR results, where DocFusion outperforms UniMERNet by 5.2% ExpRate and StructEqTable by 1.6% F1. These numbers are only trustworthy if the training data and test sets are disjoint. Appendix A.2 states that DocLatex-1.6M is constructed by extracting LaTeX content from arXiv source files, while Section 4.5 identifies UniMER-1M and DocGenome as the test sets. Both UniMER-1M and DocGenome are also derived from arXiv documents. The paper does not report any decontamination analysis, such as hashing LaTeX strings or matching arXiv identifiers, to establish that formulas and tables in the test sets were not seen during training. Since the model is generative and trained on standardized LaTeX, even near-duplicate examples from the same arXiv papers could be memorized, directly inflating ExpRate, CSR, and cell-level F1. This concern is distinct from the DLA metric protocol issue: it does not hinge on evaluation methodology but on the integrity of the benchmark itself. If overlap exists, the claimed SOTA for MER and TR is unsupported, and the load-bearing mechanism GK-CEL cannot be credited with the observed gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DocFusion, a unified generative document parsing model with 289M parameters that jointly handles document layout analysis (DLA), math expression recognition (MER), table recognition (TR), and OCR. The central methodological contribution is Gaussian-Kernel Cross-Entropy Loss (GK-CEL), which smooths coordinate-token probabilities to reconcile continuous coordinates with discrete token generation. The authors also introduce DocLatex-1.6M, a large-scale LaTeX dataset extracted from arXiv sources. Experiments report strong results on four tasks, with ablations supporting the benefits of multi-task training and GK-CEL.","tokens_in":13754,"tokens_out":7253,"duration_ms":60196,"significance":"If validated, the paper makes a practically valuable contribution: a single lightweight model covering four document parsing tasks, with released code and data. The multi-task collaboration result and the GK-CEL ablation are useful empirical findings. However, the SOTA claim is contradicted by the paper's own DLA numbers, and the MER/TR benchmarks face unresolved data-contamination concerns. The correct assessment of the contribution hinges on the benchmark validity and on a precise specification of GK-CEL.","major_comments":[{"comment":"The abstract claims SOTA performance across four key tasks, but Table 3 reports DocFusion DLA F1 of 88.4 on the full DocLayNet test set versus 89.5 for YOLO-DocLayout and 89.3 for Deformable-DETR. Section 4.5.4 only claims \"strong performance\" for DLA, so the abstract and Section 1 overstate the results. Please revise the SOTA claim or provide evidence that the comparison is unfair to DocFusion beyond what is already stated.","section":"Abstract and Table 3"},{"comment":"DocLatex-1.6M is constructed by extracting LaTeX from arXiv source files, and the MER and TR test sets (UniMER-1M and DocGenome) are also derived from arXiv documents. The paper reports no decontamination analysis such as hashing LaTeX strings or matching arXiv identifiers to ensure the training and test sets are disjoint. Because the model is generative and trained on standardized LaTeX, near-duplicate formulas or tables from the same source papers could be memorized, directly inflating ExpRate, CSR, and cell-level F1. Please provide overlap statistics or demonstrate that the test samples were not seen during training.","section":"Appendix A.2 and Sections 4.5.1-4.5.2"},{"comment":"The paper states that DocLayNet \"offers comprehensive annotations for layout elements and their corresponding text,\" but the publicly available DocLayNet dataset provides layout bounding boxes and category labels, not text transcriptions. The OCR evaluation on 3,000 English samples from DocLayNet therefore lacks a clearly described ground-truth source. Please specify how the OCR text labels were obtained or re-annotated; without this, the reported BLEU and EditDis results are not verifiable.","section":"Sections 4.1 and 4.5.3"},{"comment":"The Gaussian kernel K is defined without normalization, and the convolution output C is substituted into P within the coordinate-token range [s,e] without renormalizing the resulting P''. As written, P'' may not sum to 1, and the cross-entropy in Eq. (6) can take negative values for confident predictions, breaking the standard probabilistic interpretation. Please state whether the kernel or the resulting distribution is normalized in the actual implementation and provide the exact forward computation; otherwise the theoretical grounding of GK-CEL is incomplete.","section":"Section 3.3, Eqs. (3)-(6)"},{"comment":"Table 4 reports DLA F1 of 88.9 for the OCR+DLA configuration, while the full model, which includes all four tasks, has DLA F1 of 88.4 in Table 3. The full four-task configuration is not included in Table 4, so the claim that \"integrating recognition data significantly enhances detection performance\" is not demonstrated for the actual proposed model. Please include the full-model row in Table 4 and discuss the apparent drop in DLA F1 when MER and TR are added.","section":"Tables 3 and 4"}],"minor_comments":[{"comment":"The text says DocFusion's ExpRate surpasses UniMERNet by 5.2%, but Table 2 reports 94.2 versus 89.5, a difference of 4.7 percentage points; please reconcile the numbers.","section":"Section 4.5.1 and Table 2"},{"comment":"The text says the TR F1 surpasses the second-ranked model by 1.6%, but Table 2 shows 92.1 versus 90.6, a difference of 1.5 percentage points; please correct the value.","section":"Section 4.5.2 and Table 2"},{"comment":"The phrase \"multi recognition tasks\" should be \"multiple recognition tasks.\"","section":"Section 2"},{"comment":"There is a typo: \"TThe authors\" should be \"The authors.\"","section":"Acknowledgements"},{"comment":"The text states that results for the larger 738M variant \"will be included in a future version,\" but Table 9 already reports DocFusion-large results; this is inconsistent and should be clarified.","section":"Appendix C.3 and Table 9"},{"comment":"The caption reads \"Ablation analysis of Gaussian-Kernel Cross-Entropy Loss\" but the table compares model sizes; the caption appears to be copied from Table 5 and should be rewritten.","section":"Table 9 caption"},{"comment":"The alternative optimization strategies (data-ratio/loss-weight adjustments and soft-argmax) are described only qualitatively without quantitative results; adding the best observed numbers would strengthen the comparison with GK-CEL.","section":"Appendix C.1-C.2"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the absence of decontamination between DocLatex-1.6M and the arXiv-derived test sets for MER and TR. If overlap exists, the reported SOTA gains for those tasks are unsupported. The abstract's blanket SOTA claim is also contradicted by the DLA results in Table 3. The paper has a useful contribution if these benchmark-validity questions can be resolved and if GK-CEL is specified precisely. I recommend major revision rather than rejection because the issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DocFusion is worth a look if you work on document parsing or RAG pipelines. It is a single 289M generative model that covers layout analysis, math expression recognition, table recognition, and OCR, together with a cleaned DocLatex-1.6M dataset built from arXiv LaTeX. The model itself is Florence-2-style, and GK-CEL is a modest tweak: Gaussian-smooth the cross-entropy over coordinate tokens. But the package is practical, and the multi-task collaboration results (Table 4) suggest recognition data genuinely helps detection. That is the most useful finding in the paper.\n\nThe main claims, though, are overstated. The abstract says SOTA across four tasks, but Table 3 shows DocFusion's DLA F1 on full DocLayNet is 88.4, behind YOLO-DocLayout's 89.5 and Deformable-DETR's 89.3. The authors try to reframe the DLA comparison using threshold-maximized F1 instead of AP, and they disclose it, but it does not rescue the SOTA statement. If you read carefully, the real selling point is that a single small model can approximate specialized models without NMS or confidence tuning.\n\nThe bigger problem is data separation. DocLatex-1.6M is extracted from arXiv LaTeX sources, and the MER and TR test sets (UniMER-1M, DocGenome) are also arXiv-derived. The paper reports no decontamination—no hashing of LaTeX strings, no arXiv ID matching. A generative model trained on standardized LaTeX can memorize near-duplicates, which would inflate ExpRate, CSR, and cell F1. The MER gain of 5.2% ExpRate over UniMERNet and the TR gain of 1.6% F1 over StructEqTable are exactly the numbers that need this check. Even without overlap, GK-CEL's hyperparameters (kernel size n, sigma) are never given, and no error bars appear anywhere. These are fixable, but they are load-bearing for the recognition claims.\n\nI'd send this to peer review. The model and dataset are reproducible, the idea is sensible, and a rigorous referee could push the authors to run decontamination and report the missing hyperparameters. It is not a groundbreaking paper, but it is a solid engineering contribution that belongs in the literature once the evaluation is made honest. For a reader building document automation systems, the dataset alone is worth citing.","headline":"A genuinely useful unified parsing model and dataset, but the SOTA claim fails its own DLA benchmark and the recognition numbers need a decontamination check before they can be trusted.","tokens_in":14356,"tokens_out":2470,"would_cite":true,"duration_ms":20998,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single 289M generative model matches specialized document parsers across four core tasks.","keywords":["document parsing","unified model","layout analysis","mathematical expression recognition","table recognition","OCR","Gaussian-kernel cross-entropy loss","multi-task learning"],"falsifier":"Run exact and fuzzy string overlap between DocLatex-1.6M training samples and the UniMER-1M and DocGenome test samples, then retrain DocFusion with all near-duplicates removed; if the reported ExpRate, CSR, and F1 margins shrink materially, the benchmark gains are not due to the architecture or loss alone.","tokens_in":13272,"feed_emoji":"📄","tokens_out":9873,"duration_ms":79314,"temperature":0.7,"pith_summary":"DocFusion aims to prove that a single lightweight generative model—289M parameters—can do the work of a four-module document parsing pipeline, covering layout analysis, math expression recognition, table recognition, and OCR, and can match or beat specialized systems on standard benchmarks. The paper identifies the key obstacle as representational: detection requires continuous coordinates while generative decoding is discrete, and ordinary cross-entropy gives poor gradients on quantized coordinate tokens. Its proposed solution, Gaussian-Kernel Cross-Entropy Loss (GK-CEL), smooths the model's probability distribution over neighboring coordinate tokens before applying cross-entropy, which the paper shows improves all four tasks in ablations. The work also contributes DocLatex-1.6M, a standardized LaTeX dataset for math and tables, and reports state-of-the-art numbers on the UniMER-1M, DocGenome, and DocLayNet benchmarks. The claim, read sympathetically, is that a carefully chosen objective makes multi-task generative document parsing practical at a small scale.","feed_headline":"One 289M model parses layouts, math, tables, and text","feed_subtitle":"A Gaussian-kernel loss lets one decoder handle detection and recognition at once, matching or beating much larger systems.","key_machinery":"The load-bearing object is Gaussian-Kernel Cross-Entropy Loss (GK-CEL), a modified objective for generative models that output quantized coordinates as vocabulary tokens. After computing the softmax distribution, the method zeros out probabilities outside the coordinate-token index range, applies a one-dimensional convolution whose weights follow a Gaussian, reinserts the convolved values inside that range, and then evaluates cross-entropy only at valid target positions. This smoothing redistributes gradient signal from a target coordinate token to nearby location tokens, counteracting the label jitter that makes cross-entropy treat continuous values as unrelated discrete classes. It is supported by two secondary mechanisms: coordinate quantization, which turns bounding boxes into loc tokens so detection and recognition share one decoder, and the DocLatex-1.6M dataset, whose standardized LaTeX targets reduce annotation noise for math and table outputs.","core_discovery":"The central claim is that continuous coordinate detection and discrete token recognition can be trained together in one autoregressive decoder if the objective treats neighboring coordinate tokens as related rather than as independent classes. Under standard cross-entropy, small labeling noise spreads probability across adjacent location tokens and produces weak or skewed gradients, and in multi-task training the coordinate prediction can dominate the loss. GK-CEL convolves the softmax distribution restricted to the coordinate-token range with a Gaussian kernel, merges the smoothed values back into the distribution, and then computes cross-entropy, so small coordinate shifts still yield informative gradients. On the paper's evidence this change alone raises recognition quality, LaTeX compile-success rates, table F1, and layout-detection F1 relative to plain cross-entropy, and joint training with OCR data improves DLA F1 over detection-only training. The paper concludes that recognition and detection are mutually beneficial, not competing, when the loss no longer forces a false choice between continuous and discrete targets.","pith_inferences":["If GK-CEL generalizes, the same Gaussian smoothing should transfer to other generative models with quantized continuous outputs—keypoint detection, segmentation masks, time-series forecasting—where coordinate jitter currently weakens cross-entropy gradients; replacing soft-argmax/MSE hybrids with GK-CEL on one such task would test this directly.","The paper's collaboration result predicts a scaling pattern: adding further recognition tasks (charts, chemical formulas, handwriting) to the same 289M decoder should keep improving layout F1 and LaTeX compile rates, which is an incremental experiment rather than a new architecture.","The DocLatex-1.6M standardization recipe can be applied to other LaTeX-heavy benchmarks, and measuring token-length reduction and compile-rate change after re-annotation would quantify how much of the reported gain comes from cleaner targets versus the model itself."],"forward_implications":["One 289M model can replace a four-model parsing pipeline for layout, math, tables, and OCR, cutting integration and maintenance overhead.","Recognition data strengthens detection: systems trained jointly on OCR plus layout should beat detection-only systems on the same pages.","Generative detection without confidence scores or NMS becomes practical, since GK-CEL stabilizes coordinate gradients enough for direct token prediction.","Standardized, shorter LaTeX targets (about 34.2% shorter for complex and 37.5% shorter for simple expressions on the UniMER-1M test set) reduce autoregressive decoding cost for math and table outputs.","On the paper's benchmarks, a 289M unified model matches or beats much larger OCR and recognition systems, so parameter count alone is not the binding constraint."],"supporting_citations":[{"why":"Defines the coordinate-quantization representation and generative detection-plus-recognition paradigm that DocFusion inherits.","marker":"Xiao et al., 2023"},{"why":"Supplies DocLayNet, the training and evaluation source for both the DLA and OCR experiments.","marker":"Pfitzmann et al., 2022"},{"why":"Provides UniMERNet, the main MER baseline, and the UniMER-1M test sets used for MER evaluation.","marker":"Wang et al., 2024b"},{"why":"Provides DocGenome, the source of the 3,000-table TR test set used for table-recognition evaluation.","marker":"Xia et al., 2024"},{"why":"Supplies the Dual Attention vision encoder used to extract features from dense document layouts.","marker":"Ding et al., 2022"},{"why":"Supplies HME100K handwritten math expressions, added to training to broaden MER generalization.","marker":"Yuan et al., 2022"},{"why":"GOT is the end-to-end page-level OCR baseline that DocFusion compares against on OCR, MER, and TR.","marker":"Wei et al., 2024"},{"why":"DocLayout-YOLO is the main layout-detection baseline whose F1 performance DocFusion must match.","marker":"Zhao et al., 2024c"}],"fun_headline_variants":["DocFusion: 0.28B model unifies parsing with one loss","Gaussian-kernel loss makes one tiny model parse all docs","One 289M model beats larger systems on four parsing tasks","Unified doc parsing: detection and recognition in one decoder","Small model, big win: 289M unifies four parsing tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparisons assume that formulas and tables in the DocLatex-1.6M training set do not substantially reappear in the UniMER-1M and DocGenome test sets, and the paper reports no decontamination analysis for that overlap.","fun_headline_variants_meta":{"raw":{"variants":["DocFusion: 0.28B model unifies parsing with one loss","Gaussian-kernel loss makes one tiny model parse all docs","One 289M model beats larger systems on four parsing tasks","Unified doc parsing: detection and recognition in one decoder","Small model, big win: 289M unifies four parsing tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000535,"raw_usage":{"total_tokens":2516,"prompt_tokens":837,"completion_tokens":1679,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":1570}},"tokens_in":453,"tokens_out":1679,"duration_ms":10614,"temperature":1.0,"reasoning_tokens":1570,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:00:36.234723+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run exact and fuzzy string overlap between DocLatex-1.6M training samples and the UniMER-1M and DocGenome test samples, then retrain DocFusion with all near-duplicates removed; if the reported ExpRate, CSR, and F1 margins shrink materially, the benchmark gains are not due to the architecture or loss alone.","supporting_citations":[],"review_version":1}