Pith. sign in

REVIEW 4 major objections 6 minor 47 references

This paper argues that document-to-table extraction needs capability-aware evaluation, and backs it with a synthetic benchmark showing LLMs consistently fail hardest on reasoning, faithfulness, and conflict resolution.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 23:24 UTC pith:E6GED7ZF

load-bearing objection A useful, transparent synthetic benchmark for Doc2Table, whose headline finding is robust but whose capability labels and uniqueness guarantees rest on LLM self-verification. the 4 major comments →

arxiv 2602.13812 v3 pith:E6GED7ZF submitted 2026-02-14 cs.DB cs.AIcs.MA

DTBench: A Synthetic Benchmark for Document-to-Table Extraction

Yuxiang Guo , Zhuoran Du , Nan Tang , Kezheng Tang , Congcong Ge , Yunjun Gao This is my paper
classification cs.DB cs.AIcs.MA
keywords document-to-table extractionsynthetic benchmarkcapability taxonomyLLM evaluationmulti-hop reasoningevidence faithfulnessconflict resolutionTable2Doc synthesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that Doc2Table extraction — turning documents into structured tables under a target schema — is not a single skill, and that benchmarks must be evaluated cell by cell, capability by capability. To enable that, it builds DTBench by reversing the usual direction: start with curated ground-truth tables and synthesize documents designed so that each cell requires a known extraction capability. The benchmark covers five capability families and 13 sub-capabilities across 120 cases and 8,811 cell-level instances. Evaluating eight mainstream LLMs, the paper finds a consistent and large gap between direct extraction and indirect extraction, with multi-hop reasoning, evidence faithfulness, and implicit conflict resolution as the persistent bottlenecks. If DTBench's validity holds, it provides a reusable, low-cost way to measure where LLM-based extraction fails — the first step toward reliable SQL-ready tables from unstructured documents.

Core claim

The central claim is that a reverse Table2Doc synthesis workflow can produce a benchmark whose ground-truth tables are known by construction, and whose documents can be engineered so that recovering each cell value exercises one specific capability. Applied to 120 curated tables, the workflow yields DTBench, and the evaluation across eight LLMs of different families and scales shows that recall on capability-demanding cells trails recall on directly extractable cells by roughly 16 to 45 percentage points depending on the model. The author's diagnosis: transformation-alignment and rule-based conflict resolution are largely internalized by current LLMs, but multi-hop reasoning, outputting NULL

What carries the argument

The load-bearing mechanism is the multi-agent Table2Doc synthesis workflow with capability treated as a latent variable. An annotator assigns each table cell a first-level capability label; a refiner picks a sub-capability and generates inverse evidence sets whose recovery supposedly requires exactly that sub-capability; a planner organizes the evidence into a document outline; a writer generates section-by-section text; and deterministic checkers plus LLM checklist verifiers enforce completeness ('every cell value is recoverable'), exclusiveness ('no extra schema-conformable facts'), faithful grounding, and no-conflicts. This controlled inversion is what allows cell-level capability labels

Load-bearing premise

The benchmark's validity rests on the assumption that the synthesis/verification pipeline actually produces documents whose only schema-conformable table is the ground truth and whose cells require exactly the assigned capabilities, a property checked by LLM-based checklists rather than formal proof or independent human annotation.

What would settle it

Independently re-extract a random sample of DTBench documents with human annotators or rule-based matchers. If any cell labeled 'multi-hop reasoning' can be recovered by direct string matching, or any document contains a schema-conformable value absent from the ground-truth table, then Completeness/Exclusiveness fails for that case and its CSSR/SCSSR scores are not measuring the intended capability.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Capability-specific success rates (CSSR/SCSSR) become the natural reporting unit for Doc2Table systems; a single F1 score hides the direct/indirect gap.
  • Because scaling model size within a family shrinks the direct-extraction gap but barely narrows the indirect gap, larger models alone are not a sufficient path to reliable table extraction.
  • The near-saturated rule-based resolution scores imply that explicit deconfliction rules are not the hard part; future systems should focus on representing implicit constraints and source reliability.
  • Low evidence-faithfulness scores imply that production pipelines should expect LLMs to hallucinate table cells and should treat unfilled cells (NULL) as a first-class requirement during evaluation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: the same reverse-generation machinery could be turned around to produce training data, not just evaluation data, since it yields documents with guaranteed ground truth and targeted difficulty; the authors do not draw this implication.
  • Going beyond the paper: the evidence-faithfulness findings suggest a concrete deployment rule — for every cell an LLM fills, require a retrievable source span in the document and force NULL when none exists; this deterministic backstop is a testable extension the paper leaves implicit.
  • Going beyond the paper: if the consistency gap persists across models, one can test whether injecting the benchmark's implicit cross-attribute constraints (e.g., discharge date cannot precede admission date) directly into the decoding or verification layer closes most of the conflict-resolution gap.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. DTBench is a synthetic benchmark for document-to-table (Doc2Table) extraction, built by reversing the extraction direction: the authors start from 120 curated ground-truth tables, propose a two-level capability taxonomy (5 categories, 13 subcategories), and use a multi-agent LLM workflow to synthesize documents whose cells are intended to require those capabilities. The benchmark contains 8,811 cell-level instances, and the paper evaluates eight LLMs with overall precision/recall/F1 plus capability-specific success rates (CSSR/SCSSR). The central claim is that the benchmark validly measures capability-specific Doc2Table performance and that the experiments reveal persistent gaps in indirect extraction, especially multi-hop reasoning, evidence faithfulness, and implicit conflict resolution.

Significance. The paper makes a useful and timely contribution: it formalizes a capability taxonomy, proposes a scalable reverse-generation scheme, and releases a public benchmark with clearly defined metrics. If the synthesized documents truly satisfy Definition 3 (Completeness, Exclusiveness, Capability Awareness) and the LLM-assigned labels are accurate, the CSSR/SCSSR results provide a valuable capability-aware view of Doc2Table extraction that existing benchmarks lack. The deployment of multiple agents with iterative checklist verification is a reasonable engineering approach, and the public release lowers the barrier to future work. The main risk is validity: the load-bearing properties and labels are verified only by LLM prompts, not by independent human annotation or formal/document-level checks, so the capability-specific conclusions inherit the biases of the generator and annotator.

major comments (4)
  1. [§3.1 / Definition 3, §3.3.2, §3.3.4, Appendix D.3/D.6] Exclusiveness (Definition 3(iii)) is an existential property over all possible extraction functions, but the verification pipeline checks individual evidence sets and sections with LLM prompts; Step 5 merely concatenates verified sections without any document-level audit. Even if each section passes 'schema leakage', an additional tuple can arise from combinations of facts across sections (e.g., an entity name in one section and attribute values in another). No evidence is provided that no table T⊃T* is extractable from the assembled document, so the benchmark's claimed uniqueness of T* is not established. This is load-bearing because all P/R/CSSR scores are computed against T*.
  2. [§3.3.4, Appendix D.6] Completeness (Definition 3(ii)) is checked by verifying that each planned evidence is 'present' in a section, not by testing whether the target cell value is recoverable from the final text via the assigned capability. The checklist asks whether the evidence is 'extractable' but does not actually run extraction; a writer can include evidence in a form that is unrecoverable after paraphrase, or that requires world knowledge, and still pass 'faithful grounding'. The verifier also sees the ground-truth table and required evidences, making this a self-consistency check rather than an extraction-based check. Consequently, Rdir/Rind and CSSR/SCSSR may not measure what they claim.
  3. [§3.3.1, §3.4, Eq. (4)/(5)] The capability labels that define the ground-truth partition in CSSR/SCSSR are assigned by Grok-4-fast and are not human-validated. Appendix B reports human evaluation only for document quality on 50 documents, not for capability labels, evidence sets, or schema-leakage judgments. If a cell labeled 'Multi-hop Reasoning' is actually recoverable by direct matching, or a direct cell is labeled 'Arithmetic Reasoning', all capability-specific conclusions (e.g., multi-hop being a bottleneck, EF being low) are artifacts of label noise. The authors should provide inter-annotator agreement or at least an independent audit on a random sample.
  4. [§4.2 / Table 3] The direct-vs-indirect analysis is based solely on recall (Rdir vs Rind); precision is computed globally and cannot be attributed to direct/indirect categories. Since precision and recall interact through the F1 metric, the reported Δind drop may be driven partly by precision differences across models. The authors should report precision for direct and indirect cells separately, or at least discuss how non-aligned predicted cells are categorized when computing unified precision.
minor comments (6)
  1. [Figure 1(a)] Typo: 'Asserts' should be 'Assets' (and 'Current asserts' → 'Current assets').
  2. [§4.3] The acronym SCSSR is misspelled as 'SCCSR' in 'Table 4 and Figure 5 report CSSR and SCCSR results'. Please make the abbreviation consistent.
  3. [Figure 3] 'Cheker' is a typo for 'Checker'.
  4. [Appendix C] The row-alignment threshold τ is a free parameter; its value is not given, and no sensitivity analysis is reported. Since misalignment directly affects cell-level scores, a brief robustness check would strengthen the evaluation.
  5. [§4.3 / Table 4] Constraint-based Resolution for Llama3.1-70B is 10.81 while the smaller Llama3.1-8B achieves 64.86; this surprising non-monotonic pattern is not discussed. A short explanation would help readers judge whether this is a data artifact or a genuine behavioral difference.
  6. [§2.2 / Definition 2] Evidence sets are defined abstractly but the evaluation never uses them; the connection between evidence sets and the actual document text is only implicit. The authors should clarify whether evidence sets are used at evaluation time or only during synthesis.

Circularity Check

0 steps flagged

No circularity: DTBench's measurements are not reduced to their inputs.

full rationale

DTBench's construction chain—external gold tables, schema augmentation, LLM-based evidence generation, section-wise writing, and cell-level evaluation—does not fit any parameter to the quantities it later reports. The capability labels are produced by an annotator and checked by verifiers, and CSSR/SCSSR are then computed on independent external LLMs (Qwen3, Llama3.1, DeepSeek, GPT-5, Gemini). The paper explicitly identifies the inverse-generation circularity risk ('leading to a circular dependency between capability selection and evidence generation', Section 3.2, C1) and addresses it by treating capability as a latent variable, so the design does not define capabilities in terms of evaluation outcomes. The reliance on LLM checklist verifiers for Completeness/Exclusiveness and label alignment is a validity concern, but it is not a circular reduction in the paper's own equations: no Eq. X equals Eq. Y by construction, and no fitted parameter is renamed as a prediction. No load-bearing self-citation chain or imported uniqueness theorem appears; citations to the authors' earlier work are background motivation only. Therefore no significant circularity is present.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The central claim is not a mathematical derivation, so the ledger mainly records design assumptions. The row-alignment threshold τ is the main evaluation free parameter. The substantive risks are the domain assumptions about LLM-generated documents and labels.

free parameters (1)
  • row-alignment similarity threshold τ = not stated
    Used in the bipartite matching evaluation to decide which predicted rows align with gold rows (Appendix C); directly affects P/R/F1 and no sensitivity analysis is reported.
axioms (4)
  • ad hoc to paper Synthesized documents satisfy Exclusiveness: no table T⊃T* can be extracted from D.
    Central to benchmark validity; verified only by an LLM checklist verifier (Sections 3.3.2 and 3.3.4), not by formal proof or independent human audit.
  • domain assumption LLM-assigned capability labels correctly reflect the capabilities needed to extract each cell.
    Section 3.3.1 uses an LLM Annotator; no inter-annotator agreement or human validation of the resulting capability matrix is reported.
  • domain assumption Synthetic LLM-generated documents are representative of real-world Doc2Table documents.
    Appendix B evaluates lexical richness, consistency, and coherence, but not whether extraction difficulty transfers to real contracts, reports, or clinical notes.
  • domain assumption The five-category taxonomy H is comprehensive and its categories are orthogonal.
    Section 2.2 proposes the taxonomy from authors' analysis of prior work; there is no empirical validation against a real corpus of extraction errors.

pith-pipeline@v1.3.0-alltime-deepseek · 18359 in / 12792 out tokens · 114933 ms · 2026-08-02T23:24:04.276889+00:00 · methodology

0 comments
read the original abstract

Document-to-table (Doc2Table) extraction derives structured tables from unstructured documents under a target schema, enabling reliable and verifiable SQL-based data analytics. Although large language models (LLMs) have shown promise in flexible information extraction, their ability to produce precisely structured tables remains insufficiently understood, particularly for indirect extraction that requires complex capabilities such as reasoning and conflict resolution. Existing benchmarks neither explicitly distinguish nor comprehensively cover the diverse capabilities required in Doc2Table extraction. We argue that a capability-aware benchmark is essential for systematic evaluation. However, constructing such benchmarks using human-annotated document-table pairs is costly, difficult to scale, and limited in capability coverage. To address this, we adopt a reverse Table2Doc paradigm and design a multi-agent synthesis workflow to generate documents from ground-truth tables. Based on this approach, we present DTBench, a synthetic benchmark that adopts a proposed two-level taxonomy of Doc2Table capabilities, covering 5 major categories and 13 subcategories. We evaluate several mainstream LLMs on DTBench, and demonstrate substantial performance gaps across models, as well as persistent challenges in reasoning, faithfulness, and conflict resolution. DTBench provides a comprehensive testbed for data generation and evaluation, facilitating future research on Doc2Table extraction. The benchmark is publicly available at https://github.com/ZJU-DAILY/DTBench.

Figures

Figures reproduced from arXiv: 2602.13812 by Congcong Ge, Kezheng Tang, Nan Tang, Yunjun Gao, Yuxiang Guo, Zhuoran Du.

Figure 1
Figure 1. Figure 1: Challenging examples of Doc2Table extraction that require different capabilities. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A Two-level Taxonomy of Doc2Table Extraction Capabilities. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed multi-agent workflow for Table2Doc synthesis. on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Proportion of cells annotated with different capa [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance of LLMs across five capabilities (CSSR). [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 8 linked inside Pith

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Flo- rencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, et al . 2024. GPT-4 Technical Report.CoRR(2024). arXiv:2303.08774

    Pith/arXiv arXiv 2024

  2. [2]

    Appelt, Jerry R

    Douglas E. Appelt, Jerry R. Hobbs, John Bear, David J. Israel, and Mabry Tyson

  3. [3]

    Simran Arora, Brandon Yang, Sabri Eyuboglu, Avanika Narayan, Andrew Hojel, Immanuel Trummer, and Christopher Ré. 2023. Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes.Proc. VLDB Endow.17, 2 (2023), 92–105

    2023

  4. [4]

    Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2025. LongWriter: Unleashing 10, 000+ Word Generation from Long Context LLMs. InICLR

    2025

  5. [5]

    Chengliang Chai, Jiajun Li, Yuhao Deng, Yuanhao Zhong, Ye Yuan, Guoren Wang, and Lei Cao. 2025. Doctopus: Budget-aware Structural Table Extraction from Unstructured Documents.Proc. VLDB Endow.18, 11 (2025), 3695–3707

    2025

  6. [6]

    Google DeepMind. 2025. Gemini 3 Flash. https://deepmind.google/models/ gemini/flash

    2025

  7. [7]

    Google DeepMind and Google Brain. 2023. Gemini: a family of highly capable multimodal models.CoRR(2023). arXiv:2312.11805

    Pith/arXiv arXiv 2023

  8. [8]

    DeepSeek-AI. 2025. DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models.CoRR(2025). arXiv:2512.02556

    Pith/arXiv arXiv 2025

  9. [9]

    Qiyan Deng, Jianhui Li, Chengliang Chai, Jinqi Liu, Junzhi She, Kaisen Jin, Zhaoze Sun, Yuhao Deng, Jia Yuan, Ye Yuan, et al. 2025. Unstructured Data Analysis using LLMs: A Comprehensive Benchmark.CoRR(2025). arXiv:2510.27119

    arXiv 2025

  10. [10]

    Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song. 2024. Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction. InEMNLP. 9300–9322

    2024

  11. [11]

    Xin Luna Dong, Laure Berti-Équille, and Divesh Srivastava. 2009. Integrating Conflicting Data: The Role of Source Dependence.Proc. VLDB Endow.2, 1 (2009), 550–561

    2009

  12. [12]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. InSIGKDD. 6491–6501

    2024

  13. [13]

    Hamed Babaei Giglou, Jennifer D’Souza, and Sören Auer. 2024. LLMs4Synthesis: Leveraging Large Language Models for Scientific Synthesis.CoRR(2024). arXiv:2409.18812

    Pith/arXiv arXiv 2024

  14. [14]

    Mandy Guo, Zihang Dai, Denny Vrandecic, and Rami Al-Rfou. 2020. Wiki-40B: Multilingual Language Model Dataset. InLREC. 2440–2452

    2020

  15. [15]

    Yuxiang Guo, Lu Chen, Zhengjie Zhou, Baihua Zheng, Ziquan Fang, Zhikun Zhang, Yuren Mao, and Yunjun Gao. 2023. CampER: An Effective Framework for Privacy-Aware Deep Entity Resolution. InSIGKDD. 626–637

    2023

  16. [16]

    Narasayya, and Surajit Chaudhuri

    Yeye He, Xu Chu, Kris Ganjam, Yudian Zheng, Vivek R. Narasayya, and Surajit Chaudhuri. 2018. Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations.Proc. VLDB Endow.11, 10 (2018), 1165–1177

    2018

  17. [17]

    Yeye He, Zhongjun Jin, and Surajit Chaudhuri. 2020. Auto-Transform: Learning- to-Transform by Patterns.Proc. VLDB Endow.13, 11 (2020), 2368–2381

    2020

  18. [18]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions.ACM Trans. Inf. Syst.43, 2 (2025), 42:1–42:55

    2025

  19. [19]

    Yizhu Jiao, Sha Li, Sizhe Zhou, Heng Ji, and Jiawei Han. 2024. Text2DB: Integration-Aware Information Extraction with Large Language Model Agents. InACL Findings 2024. 185–205

    2024

  20. [20]

    Yizhu Jiao, Ming Zhong, Sha Li, Ruining Zhao, Siru Ouyang, Heng Ji, and Jiawei Han. 2023. Instruct and Extract: Instruction Tuning for On-Demand Information Extraction. InEMNLP. 10030–10051

    2023

  21. [21]

    Kaggle, Inc. 2024. Kaggle: Your Machine Learning and Data Science Community. https://www.kaggle.com

    2024

  22. [22]

    K. V. Kanimozhi and M. Venkatesan. 2015. Unstructured Data Analysis—A Survey. International Journal of Advanced Research in Computer and Communication Engineering4, 3 (2015), 223–225

    2015

  23. [23]

    Satyananda Kashyap, Sola Shirai, Nandana Mihindukulasooriya, and Horst Samu- lowitz. 2025. StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation.CoRR(2025). arXiv:2507.21340

    Pith/arXiv arXiv 2025

  24. [24]

    Jiale Lao, Andreas Zimmerer, Olga Ovcharenko, Tianji Cong, Matthew Russo, Gerardo Vitagliano, Michael Cochez, Fatma Özcan, Gautam Gupta, Thibaud Hottelier, et al. 2025. SemBench: A Benchmark for Semantic Query Processing Engines.CoRR(2025). arXiv:2511.01716

    arXiv 2025

  25. [25]

    Taesung Lee, Zhongyuan Wang, Haixun Wang, and Seung-won Hwang. 2013. Attribute extraction and scoring: A probabilistic approach. InICDE. 194–205

    2013

  26. [26]

    Huang Lei, Jiaming Guo, Guanhua He, Xishan Zhang, Rui Zhang, Shaohui Peng, Shaoli Liu, and Tianshi Chen. 2024. Ex3: Automatic novel writing by extracting, excelsior and expanding. InACL. 9125–9146

    2024

  27. [27]

    Changlun Li, Chenyu Yang, Yuyu Luo, Ju Fan, and Nan Tang. 2025. Weak-to- Strong Prompts with Lightweight-to-Powerful LLMs for High-Accuracy, Low- Cost, and Explainable Data Transformation.Proc. VLDB Endow.18, 8 (2025), 2371–2384

    2025

  28. [28]

    Yinghao Li, Le Song, and Chao Zhang. 2022. Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition. InSIGKDD. 978–988

    2022

  29. [29]

    Cafarella, Lei Cao, Peter Baile Chen, Zui Chen, Michael J

    Chunwei Liu, Matthew Russo, Michael J. Cafarella, Lei Cao, Peter Baile Chen, Zui Chen, Michael J. Franklin, Tim Kraska, Samuel Madden, Rana Shahout, and Gerardo Vitagliano. 2025. Palimpzest: Optimizing AI-Powered Analytics with Declarative Query Processing. InCIDR

    2025

  30. [30]

    Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017. The E2E Dataset: New Challenges for End-to-End Generation. InSIGDIAL 2017. 201–206

    2017

  31. [31]

    OpenAI. 2025. GPT-5 mini Model Documentation. https://platform.openai.com/ docs/models/gpt-5-mini

    2025

  32. [32]

    OpenAI. 2025. Introducing GPT-5. https://openai.com/index/introducing-gpt-5

    2025

  33. [33]

    Liana Patel, Siddharth Jha, Melissa Pan, Harshit Gupta, Parth Asawa, Carlos Guestrin, and Matei Zaharia. 2025. Semantic Operators and Their Optimization: Enabling LLM-Based Data Processing with Accuracy Guarantees in LOTUS.Proc. VLDB Endow.18, 11 (2025), 4171–4184

    2025

  34. [34]

    Parameswaran, and Eugene Wu

    Shreya Shankar, Tristan Chambers, Tarak Shah, Aditya G. Parameswaran, and Eugene Wu. 2025. DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing.Proc. VLDB Endow.18, 9 (2025), 3035–3048

    2025

  35. [35]

    Baoguang Shi, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai. 2016. Robust Scene Text Recognition With Automatic Rectification. InCVPR. 4168– 4176

    2016

  36. [36]

    Llama Team. 2024. The Llama 3 Herd of Models.CoRR(2024). arXiv:2407.21783

    Pith/arXiv arXiv 2024

  37. [37]

    Qwen Team. 2025. Qwen3 Technical Report.CoRR(2025). arXiv:2505.09388

    Pith/arXiv arXiv 2025

  38. [38]

    Wikimedia Foundation. 2023. Wikipedia Database Download. https://dumps. wikimedia.org

    2023

  39. [39]

    Shieber, and Alexander M

    Sam Wiseman, Stuart M. Shieber, and Alexander M. Rush. 2017. Challenges in Data-to-Document Generation. InEMNLP. 2253–2263

    2017

  40. [40]

    Fei Wu, Raphael Hoffmann, and Daniel S. Weld. 2008. Information extraction from Wikipedia: moving down the long tail. InSIGKDD. 731–739

    2008

  41. [41]

    xAI. 2025. Grok 4 Fast. https://x.ai/news/grok-4-fast Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Yuxiang Guo, et al

    2025

  42. [42]

    Xin Luna Dong. 2020. Data Sets for Data Fusion Experiments. https://lunadong. com/fusiondatasets

    2020

  43. [43]

    Zhengxuan Zhang, Zhuowen Liang, Yin Wu, Teng Lin, Yuyu Luo, and Nan Tang

  44. [44]

    Guineng Zheng, Subhabrata Mukherjee, Xin Luna Dong, and Feifei Li. 2018. OpenTag: Open Attribute Value Extraction from Product Profiles. InSIGKDD. 1049–1058

    2018

  45. [45]

    assignments

    Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, and Mrinmaya Sachan. 2023. Recurrentgpt: Interactive generation of (arbitrarily) long text.CoRR(2023). arXiv:2305.13304 DTBench: A Synthetic Benchmark for Document-to-Table Extraction Conference acronym ’XX, June 03–05, 2018, Woodstock, NY A Table Coll...

    Pith/arXiv arXiv 2023

  46. [1993]

    FASTUS: A Finite-state Processor for Information Extraction from Real- world Text. InIJCAI. 1172–1178

  47. [2025]

    arXiv:2504.10036

    DataPuzzle: Breaking Free from the Hallucinated Promise of LLMs in Data Analysis.CoRR(2025). arXiv:2504.10036

    arXiv 2025