Pith. sign in

REVIEW 6 major objections 6 minor 41 references

Structure then Query: Enabling Precise Analytical Queries over Unstructured Documents

T0 review · 6 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AnnoIndex claims that pre-building a hierarchical annotation index over documents and then executing SQL-like extraction plans yields precise analytical queries at higher F1 and lower LLM cost than prior retrieval-based systems.

desk verdict Solid architecture and a genuinely useful cost model, but the evaluation needs artifacts and a sensitivity analysis before the headline F1 claims are trustworthy. read the letter →

arxiv 2608.13384 v1 pith:57XSARXI submitted 2026-08-13 cs.IR cs.DB

Teng Lin , Yuyu Luo , Nan Tang This is my paper · ORCID
classification cs.IRcs.DB
keywords unstructureddocumentanalysisannotationindexschemainductionstructuredqueryengineLLMcostamortizationattributereuseEXTRACToperatoranalyticalqueries
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AnnoIndex asks whether unstructured document collections can be queried as precisely as relational databases, and argues that they can if you first materialize an annotation index over the corpus instead of relying on vector similarity. The system's SchemaLoop automatically induces a three-layer schema (dataset, table, document), uses a lightweight extractor to populate field values once offline, and then answers queries with a SQL-like plan that filters on indexed fields first and applies extraction only to the dwindling remainder. The paper reports an average F1 of 0.87 across three real-world datasets, beating the strongest prior cost-optimized system by 7 points and a direct LLM prompt baseline by 14 points, while using fewer amortized tokens per query. If this holds, the practical consequence is that precise analytical workloads over text—range filters, joins, progressive reasoning—can be served without paying LLM cost on every document for every query.

What carries the argument

The two load-bearing mechanisms are the Annotation Index, a materialized store of per-document attribute values governed by a three-layer schema, and the EXTRACT operator, a user-defined predicate in a SQL-like execution plan that can be backed by regex, a lightweight model, or an LLM. SchemaLoop produces the schema by a hypothesize-verify-refine loop: it groups documents by upper-layer annotations, generates candidate schemas with an LLM, verifies them by extracting from a random sample of 5 documents per group using a lightweight model, and keeps a schema only if extraction success is at least 0.6 and filtering efficiency at least 0.3, where filtering efficiency is one minus the maximum fraction of documents sharing the same value assignment. The query engine then evaluates predicates progressively: index filters first, then extraction in ascending cost, with short-circuiting and join-to-filter rewriting, and it persists newly extracted values back into the index for reuse.

What would settle it

Run AnnoIndex on the WikiText corpus while withholding from the schema-verification sample every document containing a queried attribute that occurs in fewer than 5% of documents, then measure recall on the join and progressive-reasoning queries that reference that attribute; a large recall drop would show that the 5-document verification is the cause rather than the extraction model itself.

Watch

Extended reading notes

Core claim

The central discovery is that the two obstacles named in the paper—fuzzy information location and unreliable algebraic reasoning—can be overcome by decoupling extraction from querying. AnnoIndex precomputes a hierarchical, queryable annotation index whose schemas are discovered automatically from the raw corpus, so that attribute-level filters such as birth year before 1985 become plain numerical comparisons that require no LLM call. During querying, the engine compiles the user question into a SQL-extended plan, filters using the index first, and then runs extraction in ascending order of cost—regex, then a lightweight model, then an LLM—calling the expensive model only on the minimal set of documents that survive the earlier filters. Extracted values are written back into the index, so repeated queries become cheaper. The paper claims this design yields state-of-the-art accuracy, with average F1 0.87, while consuming roughly 11 times fewer LLM tokens than the direct LLM baseline.

Load-bearing premise

The load-bearing operational premise is that a schema accepted on the evidence of a random sample of only 5 documents per group, with success-rate threshold 0.6 and filtering-efficiency threshold 0.3, will still extract the attributes users actually query across the full corpus; if the sample misses rare but queried attributes, the index filters out the right documents and the reported accuracy and cost gains do not transfer.

Editorial extensions

If this is right

  • Attribute extraction cost shifts from per-query to one-time build, so workloads with many repeated or related queries see marginal LLM cost approach zero.
  • Queries expressible as index filters, such as equality or range conditions, need no LLM at all, making high-throughput interactive analytics over text feasible.
  • Deep semantic predicates, such as determining whether a legal provision conflicts with superior law, remain answerable because the engine reserves LLM calls for the small candidate set that survives structured filtering.
  • Cross-table joins and multi-step reasoning become executable as SQL-like plans rather than opaque LLM reasoning, a capability the paper reports as a 0.86 F1 on three-way join queries.
  • Attribute reuse makes the system continuously cheaper: a virtual field promoted to the schema after repeated EXTRACT usage reduces the cost of future queries that reference it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the schema verification thresholds themselves could be learned from the query workload rather than fixed in advance; adapting acceptance criteria to observed query attributes would likely improve recall on rare-field questions.
  • If the cost-ascending extraction principle transfers beyond text, the same materialize-cheap-labels-first architecture could be applied to image or multimodal collections, provided a schema induction loop can define per-domain attributes.
  • The paper's amortized token comparison credits the offline build over a fixed 500-query evaluation; a deployment planner would need the break-even point—the workload size at which the one-time indexing cost is recovered—which the paper does not isolate as a single number.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper presents AnnoIndex, a system for precise analytical queries over unstructured documents. It consists of SchemaLoop, an offline closed-loop schema induction framework that derives three-layer hierarchical annotation schemas (dataset, table, document) and materializes an annotation index, and a Structured Query Engine that compiles natural-language queries into SQL-like execution plans, applies structured filtering against the index, and invokes an EXTRACT operator with progressively expensive extractors (regex, Mistral-7B, GPT-4o). Extracted values are written back into the index for reuse. The evaluation uses three datasets (LCR, WikiText, SWDE) with 500 queries each, reporting average F1 of 0.87 (Performance mode) and 0.83 (Economical mode) versus baselines such as QUEST (0.80) and a GPT-4o LLM baseline (0.73), while claiming substantially lower amortized LLM token costs. The paper argues that the offline-materialization and attribute-reuse design amortizes extraction costs and enables cheaper, precise analytic queries.

Significance. If the results hold up, AnnoIndex represents a useful step toward structured analytical querying over unstructured text, combining automatic schema induction with cost-aware execution. The strengths are the closed-loop verification of candidate schemas, the progressive EXTRACT strategy, and the persistence and reuse of extracted attributes, which together offer a plausible path to reducing LLM costs while preserving precision. However, the empirical evidence as presented is insufficient to support the strongest claims: Table I and Figure 7 report F1 without variance or significance tests; the schema verification gate is based on a statistically fragile five-document sample with hand-set thresholds; the efficiency comparison appears not to include baseline offline costs; and no code, query sets, or seeds are provided. These issues are load-bearing for the stated state-of-the-art conclusion and require additional experiments and reporting rather than a simple rewrite.

major comments (6)
  1. [§IV-B.d, Eq. (1)] The verification step samples only 5 random documents per group, but the extraction success rate formula in Eq. (1) is written over the full group D_g, not the sample actually used. With n=5, a true success rate near the 0.6 threshold has a 95% Clopper-Pearson interval of roughly [0.17, 0.93], so accept/reject decisions are dominated by sampling noise. The paper reports no random seeds, no repeated verification runs, and no sensitivity analysis of the thresholds (SR≥0.6, FE≥0.3). Because schema quality underpins the index-based filtering and the claimed cost reductions, a systematic sensitivity study varying sample size and thresholds, with endpoint F1 and token cost as outputs, is needed to establish that the gate is robust.
  2. [Table I, §VI-C] Table I reports per-dataset F1 without standard deviations, confidence intervals, or significance tests. Several differences between AnnoIndex and the strongest baselines are small (e.g., WikiText 0.91 vs Lotus 0.89; SWDE 0.96 vs Lotus 0.95), and the claim that AnnoIndex "consistently outperform[s] or match[es] all competing systems" is not supported without evidence that these gaps exceed noise. Please report variance over repeated runs (e.g., multiple seeds or bootstrap resampling) and apply an appropriate paired significance test to the fixed query sets for both F1 and token costs.
  3. [§VI-D, Figure 4] The "amortized LLM token consumption per query" metric divides total offline plus online tokens by 500 queries. Although AnnoIndex's offline SchemaLoop and index population costs are included, it is not stated whether comparable offline costs for baselines (e.g., GraphRAG's graph construction, ZenDB's index/tree construction, Lotus's indexing, or QUEST's index build) are included. If these are excluded, the comparison is not apples-to-apples and the claim that AnnoIndex achieves the lowest amortized cost is unsupported. The paper should either include all systems' total (offline+online) costs or clearly justify any exclusion.
  4. [Abstract, §I Contributions, §IV-A, §VI-B] The abstract and contribution list state that SchemaLoop "automatically" creates hierarchical annotation schemas, but the dataset-level schema is explicitly predefined by humans in Section VI-B and Section IV-A says it is "pre-defined by humans." This overstates the automation in the central contribution. Please either automate or generate this layer as well, or revise the claims to state precisely that table-level and document-level schemas are automatically induced given a manually specified dataset-level root schema.
  5. [§VI-A.b, §VI-B] Ground truth is initialized by GPT-4o extractions and then student-corrected, while Performance-mode EXTRACT operations also use GPT-4o. This creates a circularity-adjacent risk: the gold standard may be biased toward what GPT-4o can extract, favoring systems that use GPT-4o as their backend. Although students corrected and supplemented the initial output, the paper provides no inter-annotator agreement statistics and no measure of how much the gold standard differs from the raw GPT-4o output. Please report these quantities, and ideally include a gold standard constructed independently of GPT-4o (e.g., existing structured metadata or expert-built annotations) as a robustness check.
  6. [§VI (Experimental Setup)] No code, datasets, query sets, or random seeds are made available, which severely limits the reproducibility of a systems paper. The 500-query sets per dataset, the sampling procedure for verification documents, and the seed/settings for LLM calls are essential to independently verify the F1 and cost numbers. The authors should release the implementation and data artifacts, or at a minimum provide detailed query sets and seed settings in an appendix.
minor comments (6)
  1. [§IV-B.d, Eqs. (1)-(2)] The text says verification uses a random subset of 5 documents, but the SR and FE formulas are defined over all D_g. Please align the notation with the actual sample size used in the verification procedure.
  2. [§IV-C, Eq. (3)] The term PR(S_row) appears without definition; presumably it is the filtering efficiency or precision of the document-level schema, but it is never introduced. Define the notation or replace it with the appropriate previously defined quantity.
  3. [§VI-A.c] There is a typo in the VectorDB+RAG description: "We implement this into a RAG pipelin." should be a complete sentence. Also, Section I contains "More worse," which should be "What is worse" or "More importantly."
  4. [§IV-D, §VI-G] The system name is inconsistently spelled: "AnnolIndex" in Section IV-D and "AnnoRe" in Section VI-G should be corrected to "AnnoIndex" throughout.
  5. [§VI-C.a] The text states that Lotus has "larger F1 variance" on cross-table joins and aggregation, but no variance numbers are reported anywhere. Either provide quantitative variance data or soften this to a qualitative observation.
  6. [§VI-F] The token counts for the join query example (58.2K vs 44.5K) are reported without specifying the counting convention (input plus output tokens? prompt length only?). Please define the token accounting used for all efficiency figures.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the F1 and cost claims are empirically measured against a human-corrected gold standard, and the paper's self-citations are background pointers rather than load-bearing derivation steps.

full rationale

This is an empirical systems paper, not a derivation, so the classic circularity failure modes do not apply. The central claim (AnnoIndex outperforms baselines in F1 and amortized cost) is established by direct measurement on fixed query sets against a gold standard. The only circularity-adjacent element is that GPT-4o is used both to bootstrap candidate ground-truth attributes and to power the Performance-mode EXTRACT operator; however, the paper states that 'Eight graduate students then manually verify and correct all attribute-value pairs and supplement missing attributes,' which breaks any definitional equivalence between system outputs and ground truth. The SchemaLoop verification gate (random 5 documents, SR >= 0.6, FE >= 0.3) is a load-bearing statistical robustness concern, but it is a validity threat rather than circularity: the gate is not the quantity being predicted, and no equation reduces a claimed result to this gate by construction. The paper contains multiple self-citations (e.g., refs [2], [3], [7], [8], [11], [15], [19], [20]), but they are background and related-work pointers; no load-bearing argument or uniqueness theorem is imported from them. Cost-reduction claims follow from the defined amortization identity and measured invocation counts, not from a fitted parameter renamed as a prediction. Overall: no significant circularity; any concerns belong to correctness risk, not circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central accuracy and cost claims rest on several engineering assumptions: LLM-proposed schemas are extractable by a small model, a 5-document sample is representative, and the parser reliably maps queries to the induced layers. The only fitted constants are hand-set thresholds (SR=0.6, FE=0.3, sample size 5, promotion threshold 10), which are not validated by sensitivity analysis. The dataset-level schema is a human input, so the 'automatic' schema induction claim is only partially true.

free parameters (4)
  • extraction success rate threshold (SR) = 0.6
    Hand-set threshold in Section VI-B; schemas below it are refined. No sensitivity analysis is provided, and the central filter quality depends on it.
  • filtering efficiency threshold (FE) = 0.3
    Hand-set threshold in Section VI-B; no ablation varying it.
  • verification sample size = 5 documents per group
    Section IV-B.d verifies candidate schemas on a random 5 documents; representativeness is assumed.
  • virtual field promotion threshold = 10 references
    Section V-D promotes a virtual field after more than 10 EXTRACT references; the number appears arbitrary and untested.
assumptions (4)
  • domain assumption A random sample of 5 documents represents each document group for schema verification.
    Section IV-B.d uses this sample to compute SR/FE; if unrepresentative, the induced schema can be wrong.
  • domain assumption Mistral-7B can extract the attribute values of LLM-generated schemas well enough to meet the SR threshold.
    Section VI-B uses Mistral-7B for verification and economical mode; the paper does not measure per-attribute extraction quality.
  • domain assumption SchemaLoop converges within 3 to 5 iterations on real corpora.
    Section IV-C states empirical convergence; no convergence curves or failure cases are reported.
  • domain assumption The semantic parser maps natural language queries to the three-layer schema without systematic routing errors.
    Section V-A assumes layer-wise resolution works; no parser accuracy or error analysis is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structure then Query: Enabling Precise Analytical Queries over Unstructured Documents." pith.science (2026). https://pith.science/paper/57XSARXI

@misc{pith2026260813384,
  author       = {Pith},
  title        = {Pith review of: Structure then Query: Enabling Precise Analytical Queries over Unstructured Documents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/57XSARXI}},
  note         = {Machine review of arXiv:2608.13384}
}
read the original abstract

Unstructured documents constitute the majority of enterprise and web data. With the rapid development of large language models(LLMs), researchers have started to build data systems that analyze unstructured textual documents like operating on databases. However, because mainstream retrieval methods still relies on fuzzy matching based on vector similarity, accurately obtaining information and performing structured analysis and reasoning remains a major challenge. To address these limitations, AnnoIndex introduces two core fundamental components. The first is Annotation Index. The system uses a module called SchemaLoop to automatically create hierarchical annotation schemas from the raw corpus, and then uses lightweight language model to extract specific values. It turns scattered unstructured text into a materialized, structured index that enables low-cost filtering and querying. The annotation index avoids the black-box matching of vector similarity and amortizes attribute extraction costs from online queries to a one-time build. The second innovation is a Structured Query Engine. It compiles user questions into execution plans based on SQL extension. It first uses the Annotation Index for precise documents filtering, then gradually applies extraction operations in ascending order of cost, resorting to LLMs only for the remaining minimal fraction of the corpus that require deep semantic understanding. The extracted attributions are merged into the annotation index, reducing the cost of future queries. Experiments on three real-world datasets demonstrate that AnnoIndex consistently outperforms state-of-the-art baselines, achieving the highest average F1 score (0.87) while maintaining robust performance on complex multi-hop join and progressive reasoning queries.

Figures

Figures reproduced from arXiv: 2608.13384 by the authors.

Figure 1
Figure 1. The figure a shows SchemaLoop’s three-layer hierarchical annotation schema induction. The framework automatically [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Multi-loop engineering in SchemaLoop’s closed-loop [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The retrieval framework translates natural language queries into SQL-extended execution plan that first applies fast [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Amortized LLM Token Consumption by Dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: EXTRACT Invocations Across Query Batches. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Ablation Experiment Results on WikiText. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: reports results on WikiText, where joinable tables (Player, Team, City, Owner) exist. AnnoIndex significantly outperforms all baselines, especially on three-way reasoning. For example, the query “List players who are older than 35, play for a team that has won more tha…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 24 canonical work pages

  1. [1]

    80 Percent of Your Data Will Be Unstructured in Five Years,

    T. King, “80 Percent of Your Data Will Be Unstructured in Five Years,” Solutions Review, accessed: 2026-01-22. [Online]. Available: https://solutionsreview.com/data-management/80-percent-of- your-data-will-be-unstructured-in-five-years/

    work page 2026

  2. [2]

    Simplifying data integration: Slm-driven systems for unified semantic queries across heterogeneous databases,

    T. Lin, “Simplifying data integration: Slm-driven systems for unified semantic queries across heterogeneous databases,” in2025 IEEE 41st International Conference on Data Engineering (ICDE), May 2025, pp. 4690–4693

    work page 2025

  3. [3]

    MEBench: Benchmarking large language models for cross-document multi-entity question answering,

    T. Lin, Y . Luo, H. Zhang, J. Zhang, C. Liu, K. Wu, and N. Tang, “MEBench: Benchmarking large language models for cross-document multi-entity question answering,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Peng, Eds. Suzhou, China: Association for Comput...

    work page 2025

  4. [4]

    Palimpzest: Optimizing ai-powered analytics with declarative query processing,

    C. Liu, M. Russo, M. Cafarella, L. Cao, P. B. Chen, Z. Chen, M. Franklin, T. Kraska, S. Madden, R. Shahoutet al., “Palimpzest: Optimizing ai-powered analytics with declarative query processing,” inProceedings of the Conference on Innovative Database Research (CIDR), 2025, p. 2

    work page 2025

  5. [5]

    Querying templatized document collections with large language models,

    Y . Lin, M. Hulsebos, R. Ma, S. Shankar, S. Zeighami, A. G. Parameswaran, and E. Wu, “Querying templatized document collections with large language models,” in2025 IEEE 41st International Confer- ence on Data Engineering (ICDE), 2025, pp. 2422–2435

    work page 2025

  6. [6]

    Quest: Query optimization in unstructured document analysis,

    Z. Sun, C. Chai, Q. Deng, K. Jin, X. Guo, H. Han, Y . Yuan, G. Wang, and L. Cao, “Quest: Query optimization in unstructured document analysis,”Proc. VLDB Endow., vol. 18, no. 11, p. 4560–4573, Jul

    work page

  7. [7]

    Monte Carlo Tree Search for Table-to-Multimodal Report Generation

    T. Lin, Z. Zhang, Y . Luo, and N. Tang, “Monte carlo tree search for table-to-multimodal report generation,” 2026. [Online]. Available: https://arxiv.org/abs/2608.04071

    work page Pith review arXiv 2026

  8. [8]

    AnnoRetrieve: Efficient Structured Retrieval for Unstructured Document Analysis

    T. Lin, Y . Luo, and N. Tang, “Annoretrieve: Efficient structured retrieval for unstructured document analysis,” 2026. [Online]. Available: https://arxiv.org/abs/2604.02690

    work page Pith review arXiv 2026

Show all 41 references
  1. [9]

    A survey on rag meeting llms: Towards retrieval-augmented large language models,

    W. Fan, Y . Ding, L. Ning, S. Wang, H. Li, D. Yin, T. S. Chua, and Q. Li, “A survey on rag meeting llms: Towards retrieval-augmented large language models,” 2024

    2024

  2. [10]

    A comprehensive survey on long context language modeling,

    J. Liu, D. Zhu, Z. Bai, Y . He, H. Liao, H. Que, Z. Wang, C. Zhang, G. Zhang, J. Zhang, Y . Zhang, Z. Chen, H. Guo, S. Li, Z. Liu, Y . Shan, Y . Song, J. Tian, W. Wu, Z. Zhou, R. Zhu, J. Feng, Y . Gao, S. He, Z. Li, T. Liu, F. Meng, W. Su, Y . Tan, Z. Wang, J. Yang, W. Ye, B. ...

    2025

  3. [11]

    Docsage: An information structuring agent for multi-doc multi-entity question answering,

    T. Lin, Y . Zhu, Z. Zhang, Y . Luo, and N. Tang, “Docsage: An information structuring agent for multi-doc multi-entity question answering,” 2026. [Online]. Available: https://arxiv.org/abs/2603.11798

    2026

  4. [12]

    Numerical constraint-aware dense retrieval with two-phase contrastive learning,

    M. Wang, Y . Wang, and F. Wu, “Numerical constraint-aware dense retrieval with two-phase contrastive learning,”Big Data Mining and Analytics, vol. 9, no. 2, pp. 341–359, 2026

    2026

  5. [13]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,”Advances in Neural Information Processing Systems, vol. 33, pp. 9459–9474, 2020

    2020

  6. [14]

    Leave no document behind: Benchmarking long-context LLMs with extended multi-doc QA,

    M. Wang, L. Chen, F. Cheng, S. Liao, X. Zhang, B. Wu, H. Yu, N. Xu, L. Zhang, R. Luo, Y . Li, M. Yang, F. Huang, and Y . Li, “Leave no document behind: Benchmarking long-context LLMs with extended multi-doc QA,” inProceedings of the 2024 Conference on Empirical Methods in Natu...

    2024

  7. [15]

    Structured retrieval-augmented generation for multi-entity question answering over heterogeneous sources,

    T. Lin, “Structured retrieval-augmented generation for multi-entity question answering over heterogeneous sources,” in2025 IEEE 41st International Conference on Data Engineering Workshops (ICDEW), 2025, pp. 253–258

    2025

  8. [16]

    Rq-rag: Learning to refine queries for retrieval augmented generation,

    C.-M. Chan, C. Xu, R. Yuan, H. Luo, W. Xue, Y .-T. Guo, and J. Fu, “Rq-rag: Learning to refine queries for retrieval augmented generation,”ArXiv, vol. abs/2404.00610, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:268819582

    2024 arXiv

  9. [17]

    Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy,

    Z. Shao, Y . Gong, Y . Shen, M. Huang, N. Duan, and W. Chen, “Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy,” 2023. [Online]. Available: https://arxiv.org/abs/2305.15294

    2023 arXiv

  10. [18]

    From local to global: A graph rag approach to query- focused summarization,

    D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson, “From local to global: A graph rag approach to query- focused summarization,”arXiv preprint arXiv:2404.16130, 2024

    2024 arXiv

  11. [19]

    Lightkgg: Simple and efficient knowledge graph generation from textual data,

    T. Lin, “Lightkgg: Simple and efficient knowledge graph generation from textual data,” 2025. [Online]. Available: https://arxiv.org/abs/2510.23341

    2025

  12. [20]

    Srag: Structured retrieval- augmented generation for multi-entity question answering over wikipedia graph,

    T. Lin, Y . Zhu, Y . Luo, and N. Tang, “Srag: Structured retrieval- augmented generation for multi-entity question answering over wikipedia graph,”CoRR, vol. abs/2503.01346, March 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2503.01346

    2025 doi arXiv

  13. [21]

    deepdoctection,

    The Deepdoctection Authors, “deepdoctection,” 2023, accessed: 2026-01-22. [Online]. Available: https://github.com/deepdoctection/deepdoctection

    2023

  14. [22]

    Docetl: Agentic query rewriting and evaluation for complex document processing,

    S. Shankar, T. Chambers, T. Shah, A. G. Parameswaran, and E. Wu, “Docetl: Agentic query rewriting and evaluation for complex document processing,”arXiv preprint arXiv:2410.12189, 2024

    2024 arXiv

  15. [23]

    Unstructured,

    Unstructured Technologies, Inc., “Unstructured,” 2024, accessed: 2026-01-22. [Online]. Available: https://github.com/Unstructured- IO/unstructured

    2024

  16. [24]

    Retrieval-augmented generation for large language models: A survey,

    Y . Gao, Y . Xiong, X. Gao, K. Jia, J. Pan, Y . Bi, Y . Dai, J. Sun, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” 2024. [Online]. Available: https://arxiv.org/abs/2312.10997

    2024 arXiv

  17. [25]

    Doctopus: Budget-aware structural table extraction from unstructured documents,

    C. Chai, J. Li, Y . Deng, Y . Zhong, Y . Yuan, G. Wang, and L. Cao, “Doctopus: Budget-aware structural table extraction from unstructured documents,”Proceedings of the VLDB Endowment, vol. 18, no. 11, pp. 3695–3707, 2025

    2025

  18. [26]

    Docdb: A database for unstructured document analysis,

    Z. Li, Y . Zhong, C. Chai, Z. Sun, Y . Deng, Y . Yuan, G. Wang, and L. Cao, “Docdb: A database for unstructured document analysis,” Proceedings of the VLDB Endowment, vol. 18, no. 12, pp. 5387–5390, 2025

    2025

  19. [27]

    Eleet: Efficient learned query execution over text and tables,

    M. Urban and C. Binnig, “Eleet: Efficient learned query execution over text and tables,”Proceedings of the VLDB Endowment, vol. 17, no. 13, pp. 4867–4880, 2024

    2024

  20. [28]

    Unify: A system for unstructured data analytics,

    J. Wang, Y . Li, J. Wu, S. Xu, and G. Li, “Unify: A system for unstructured data analytics,”Proceedings of the VLDB Endowment, vol. 18, no. 12, pp. 5287–5290, 2025

    2025

  21. [29]

    Acorn: Performant and predicate-agnostic search over vector embeddings and structured data,

    L. Patel, P. Kraft, C. Guestrin, and M. Zaharia, “Acorn: Performant and predicate-agnostic search over vector embeddings and structured data,” Proc. ACM Manag. Data, vol. 2, no. 3, May 2024. [Online]. Available: https://doi.org/10.1145/3654923

    2024 doi

  22. [30]

    Arcade: A real-time data system for hybrid and continuous query processing across diverse data modalities,

    J. Yang, S. Mo, J. Shi, Z. Yu, K. Shi, X. Ding, and G. Cong, “Arcade: A real-time data system for hybrid and continuous query processing across diverse data modalities,”arXiv preprint arXiv:2509.19757, 2025

    2025

  23. [31]

    Autoschemakg: Autonomous knowledge graph construction through dynamic schema induction from web-scale corpora,

    J. Bai, W. Fan, Q. Hu, Q. Zong, C. Li, H. T. Tsang, H. Luo, Y . Yim, H. Huang, X. Zhouet al., “Autoschemakg: Autonomous knowledge graph construction through dynamic schema induction from web-scale corpora,”arXiv preprint arXiv:2505.23628, 2025

    2025 arXiv

  24. [32]

    SQUiD: Synthesizing relational databases from unstructured text,

    M. Sadia, Z. Yang, Y . Xiao, A. Chen, and A. Roy Chowdhury, “SQUiD: Synthesizing relational databases from unstructured text,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Peng...

    2025

  25. [33]

    Lexa: Towards automatic legal citation classification,

    F. Galgani and A. Hoffmann, “Lexa: Towards automatic legal citation classification,” inAI 2010: Advances in Artificial Intelligence, J. Li, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 445–454

    2010

  26. [34]

    From one tree to a forest: a unified solution for structured web data extraction,

    Q. Hao, R. Cai, Y . Pang, and L. Zhang, “From one tree to a forest: a unified solution for structured web data extraction,” inProceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR ’11. New York, NY , USA: As...

    2011

  27. [35]

    Vectordb: A minimal python package for storing and retrieving text using chunking, embeddings, and vector search,

    Kagi Search, “Vectordb: A minimal python package for storing and retrieving text using chunking, embeddings, and vector search,” https://github.com/kagisearch/vectordb, accessed: 2026-04-18

    2026

  28. [36]

    Openai embedding model,

    OpenAI, “Openai embedding model,” https://huggingface.co/Xenova/text-embedding-ada-002, accessed [Date of access]

  29. [37]

    Lotus: Enabling semantic queries with llms over tables of unstructured and structured data,

    L. Patel, S. Jha, C. Guestrin, and M. Zaharia, “Lotus: Enabling semantic queries with llms over tables of unstructured and structured data,”arXiv preprint arXiv:2407.11418, 2024

    2024 arXiv

  30. [38]

    Deberta: Decoding- enhanced bert with disentangled attention,

    P. He, X. Liu, J. Gao, and W. Chen, “Deberta: Decoding- enhanced bert with disentangled attention,” inInternational Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=XPZIaotutsD

    2021

  31. [39]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  32. [40]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnieret al., “Mistral 7b,”arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  33. [2025]

    Available: https://doi.org/10.14778/3749646.3749713

    [Online]. Available: https://doi.org/10.14778/3749646.3749713

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.