Pith. sign in

REVIEW 4 major objections 4 minor 87 references

CoddLLM: Empowering Large Language Models for Data Analytics

T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a 12-billion-parameter language model can be post-trained into a data-analytics specialist that outperforms much larger general-purpose models on average across eight datasets.

desk verdict A well-documented post-training recipe for data analytics, but the headline 'beats GPT-4o' rests on seen benchmarks; on truly unseen data it trails. read the letter →

arxiv 2502.00329 v1 pith:6AJ5U7C7 submitted 2025-02-01 cs.DB cs.AI

Jiani Zhang , Hengrui Zhang , Rishav Chakravarti , Yiqun Hu , Patrick Ng , Asterios Katsifodimos , Huzefa Rangwala , George Karypis
show 1 more author
Alon Halevy
This is my paper · ORCID
classification cs.DBcs.AI
keywords dataanalyticslargelanguagemodelspost-trainingsyntheticgenerationtableselectiontext-to-SQLinstructiontuningbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a 12-billion-parameter language model can be post-trained into a data-analytics specialist that outperforms much larger general-purpose models on average across eight datasets. The authors introduce a three-chapter post-training recipe: filtered analytics knowledge converted into question-answer pairs, two new text-table alignment tasks (Text-to-Schema and Row-to-Text), and task-specific examples for table selection and Text-to-SQL. They also release AnalyticsMMLU, a multiple-choice benchmark, and three table-selection benchmarks, including a human-annotated Wikipedia-based set. If the results hold, a relatively small open model could take over common data-analytics workloads at lower cost than frontier API models.

What carries the argument

The key machinery is the three-chapter post-training corpus plus the extraction-and-synthesis data pipeline. Chapter 1 filters a large educational web corpus for analytics relevance using a trained classifier, converts the filtered documents into question-answer pairs grounded in the reference documents, and filters them with an LLM-as-a-judge. Chapter 2 adds two new tasks: Text-to-Schema, which generates a table schema from a scenario description, and Row-to-Text, which generates a text description for a table row. Chapter 3 adds task-specific examples for table selection and Text-to-SQL, drawn mostly from public datasets and a large synthetic SQL set. The model is instruction-tuned with the standard autoregressive next-token loss on output tokens, first on Chapter 1 for one epoch and then on Chapters 2 and 3 for two epochs while mixing in 10% of Chapter 1 data.

What would settle it

Regenerate the AnalyticsMMLU and WikiPage-TS questions with a different model family and the same human review, then measure CoddLLM's margin over its base model and over GPT-4o; if the margin largely disappears, the shared generator was the source of the reported gains.

Watch

Extended reading notes

Core claim

The central discovery is that a deliberately structured post-training corpus of one billion tokens, organized into knowledge, text-table alignment, and task-specific chapters, can turn a 12B instruct model into a data-analytics specialist. In the paper's evaluation, CoddLLM reaches the highest overall score of 0.697 across eight datasets, a 39.3% relative gain over the base Mistral-NeMo model and a 4.0% edge over the best general-purpose model in the comparison. For table selection it beats the best general model by 12.1%, and for Text-to-SQL it reaches an average execution accuracy of 0.576, a 24.9% relative improvement over the base.

Load-bearing premise

The unseen benchmarks that support the generalization claims were generated by the same family of models that synthesized the training corpus, so the reported gains rest on the assumption that this shared provenance does not bias the questions toward the model's training distribution.

Editorial extensions

If this is right

  • A 12B open-weight model can reach the top average accuracy (0.697) across eight analytics datasets, so cost-sensitive deployments could substitute a small specialized model for a frontier API.
  • The Text-to-Schema and Row-to-Text alignment tasks, not the SQL examples alone, drive the largest gains on table-centric tasks, so these tasks are reusable ingredients for other structured-data models.
  • Table selection, which embedding-based retrieval handles as top-1 ranking, can be solved by an LLM that reasons about multi-table relationships, yielding 91.9% accuracy on the Open-WikiTable variant and outperforming BM25 and dense retrieval baselines.
  • On Text-to-SQL, the performance gains come from improved join detection and use of external knowledge, not from more SQL snippets, so adding paired examples with such annotations is the effective ingredient.
  • The claimed 12.1% lead over GPT-4o in table selection and the 24.9% average improvement over the base in Text-to-SQL suggest that domain-specific post-training can transfer across analytics tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the unseen benchmarks (AnalyticsMMLU and WikiPage-TS) were generated with the same model family that synthesized the training corpus, the generalization margin may partly reflect stylistic overlap; regenerating them with an independent generator is a direct test of that concern.
  • The human-annotated WikiPage-TS benchmark, despite only 104 examples, probes multi-hop reasoning over mixed text and tables; scaling it with the same annotation protocol would provide a stronger and more stable test for future analytics models.
  • The extraction-and-synthesis pipeline, grounding generated QA pairs in reference documents, is a reusable template for other specialized domains where raw corpora exist but instruction data is scarce.
  • If the table-selection margin is confirmed independently, data-lake search pipelines may shift from embedding-based ranking to LLM-based set selection over candidate tables.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents CoddLLM, a 12B model obtained by post-training Mistral-NeMo-Instruct on a curated analytics corpus. The corpus is organized into three chapters: Chapter 1 contains roughly 8.8M instruction-response pairs synthesized from filtered FineWeb-Edu documents using Claude-3.5-Sonnet; Chapter 2 adds two new table-text alignment tasks, Text-to-Schema and Row-to-Text, generated from SchemaPile and Open-WikiTable with the same model family; Chapter 3 adds Table Selection examples derived from BIRD and Open-WikiTable training sets and Text-to-SQL examples from BIRD, Spider, and synthetic data. The authors introduce AnalyticsMMLU and WikiPage-TS as new benchmarks and report that CoddLLM achieves the highest overall score of 0.697 across eight evaluation datasets, outperforming GPT-4o by 4.0% overall and by 12.1% on Table Selection.

Significance. If the headline result were fully supported, this would be a notable contribution: a 12B domain-post-trained model outperforming frontier general models on a suite of data-analytics tasks. The paper has genuine strengths that should be credited: Table 2 explicitly separates seen and unseen benchmarks, Table 6 gives informative ablations over data-format and chapter choices, the data-generation protocol is described in enough detail to reproduce, and the new benchmarks receive human review. However, the central generalization claim is currently not established. The overall score is a task-average mean in which seen data carry more weight than unseen data, and the unseen benchmarks share a generator with the training corpus. The significance of the work therefore depends on whether the authors can either produce independent evaluation evidence or substantially reframe the claims as in-distribution post-training gains.

major comments (4)
  1. [§2.1, Table 3] The headline claim that CoddLLM 'achieves the highest overall score of 0.697' and 'outperforms GPT-4o by 4.0%' is not supported as a claim of general analytics capability. The overall score in Table 3 is the mean of the three task averages, not the mean of the eight dataset accuracies; under this weighting the four unseen datasets contribute only 4/9 of the total score, while the seen datasets contribute 5/9. On a simple average of the four unseen rows alone (MCQ-DA, MCQ-DB, MCQ-ML, WikiPage-TS), CoddLLM scores 71.2 versus GPT-4o's 75.4, and on WikiPage-TS it is below GPT-4o. Similarly, the advertised 12.1% Table-Selection advantage comes entirely from the two seen benchmarks BIRD-TS and Open-WikiTable-TS, whose training sets are used in Section 4.3.1. The abstract and Section 2.1 should either restrict the 'best overall' claim to in-distribution tasks or report an independent-evaluation average as the primary comparison.
  2. [§5.1, §5.2 vs §4.1.2, §4.2] The benchmarks introduced to demonstrate generalization are not distributionally independent of the training data. AnalyticsMMLU questions are partially generated with Claude-3.5-Sonnet (Section 5.1), and WikiPage-TS questions and ground-truth tables are generated with Claude-3.5-Sonnet (Section 5.2, Step 2), while the Chapter 1 QA pairs, the Text-to-Schema descriptions, and the Row-to-Text descriptions are also synthesized with Claude-3.5-Sonnet (Sections 4.1.2, 4.2.1, 4.2.2). This shared provenance means that gains on the 'unseen' benchmarks could reflect stylistic or formatting familiarity rather than general analytics ability, and manual review of answers does not remove that confound. A concrete mitigation would be to split AnalyticsMMLU into textbook-sourced and Claude-generated items and report each separately, and to validate WikiPage-TS against an independently authored question set.
  3. [§5.2, Table 3] WikiPage-TS contains only 104 examples, and the paper reports no confidence intervals or significance tests. With n=104, the standard error of a binary accuracy near 0.5 is about 4.9 percentage points, so the 55.8 versus 58.7 difference between CoddLLM and GPT-4o is well within sampling noise. The large relative improvement over the base model (from 28.8 to 55.8) is suggestive, but the paper should report bootstrap confidence intervals or an exact binomial test and avoid drawing strong conclusions from this dataset in isolation.
  4. [§5.3, Table 3] Formatting-failure recovery is applied unevenly across the compared systems, which makes the per-model comparisons in Table 3 difficult to interpret. For Text-to-SQL, GPT-3.5-Turbo, GPT-4o-mini, GPT-4o, and Mixtral-8x7B receive an additional ```sql``` code-block extraction when the <SQL> tag is not produced, while CoddLLM results are reported without that post-processing step. For Table Selection, the base Mistral-NeMo model receives additional answer extraction on WikiPage-TS, but other baselines are marked as FE or LCE without the same treatment. Because the evaluation metrics are exact-match on format-sensitive outputs, the reported scores conflate generation quality with answer-extraction engineering. The authors should apply the same extraction pipeline to all models and report both raw and post-processed scores for every system.
minor comments (4)
  1. [§7] The section title contains a duplicated word: 'Conclusion and and Future Work' should be 'Conclusion and Future Work'.
  2. [§5.1] In the main-results paragraph, 'MCD-DB' is a typo for 'MCQ-DB'.
  3. [§4.3.1] The text 'Open-The WikiTable dataset' should read 'The Open-WikiTable dataset'.
  4. [Abstract] The abstract states that the paper contributes 'three comprehensive benchmarks,' but the paper actually introduces four new evaluation datasets (three AnalyticsMMLU datasets and WikiPage-TS) in addition to re-purposing two existing ones; the wording should be aligned with the text in Section 5.

Circularity Check

3 steps flagged · score 4.0 of 10

The 'best overall' and 12.1% table-selection claims are computed over benchmarks whose training splits are in CoddLLM's corpus; on the only unseen table-selection benchmark CoddLLM trails GPT-4o, and the 'unseen' benchmarks are generated by the same Claude-3.5-Sonnet family used to synthesize the training data.

  1. fitted input called prediction [Section 2.1 bullet; Section 4.3.1; Table 3]
    "For Table Selection, CoddLLM outperforms GPT-4o by 12.1%, making it the most potent model for this task. ... To create training examples, we use the training set of BIRD and Open-WikiTable and convert these datasets to serve the needs of the Table Selection task."

    The Table Selection average in Table 3 combines BIRD-TS and Open-WikiTable-TS, both explicitly marked 'Seen' because their training splits are used in Chapter 3, with the unseen WikiPage-TS. On the only unseen TS benchmark, CoddLLM scores 55.8 versus GPT-4o's 58.7. The advertised 12.1% relative advantage therefore comes entirely from the two in-distribution benchmarks; it reflects fine-tuning on the same benchmark family rather than an independently measured table-selection capability.

  2. fitted input called prediction [Section 2.1 bullet; Section 4.3.2; Table 3]
    "In the Text-to-SQL evaluation, CoddLLM achieves an average execution accuracy of 0.576, outperforming all other models ... We extract a sample of approximately 9K examples from the BIRD [32] dataset's training set and 7K examples from the Spider [64] dataset's training set."

    Spider-dev and BIRD-dev are marked 'Seen' in Table 2, and Section 4.3.2 confirms that training splits of both datasets are in the Chapter 3 corpus. The reported Text-to-SQL edge over GPT-4o (8.9% on Spider, 7.8% on BIRD) is thus an in-distribution fine-tuning comparison against a zero-shot generalist. It is presented as a general capability claim, but it reduces to the fact that the model was post-trained on the same benchmark's training data.

1 more flagged steps
  1. other [Section 5.1; Section 4.1.2; Section 5.2 Step 2; Table 2]
    "We source some of the questions from textbooks and generated additional questions and answers using Claude-3.5-Sonnet. ... we prompt Claude-3.5-Sonnet to synthesize question-answer pairs of varying difficulty levels ... For each Wikipedia page, we use Claude-3.5-Sonnet to generate questions and identify the ground truth tables needed to answer them."

    The four datasets marked 'Unseen' — MCQ-DB, MCQ-DA, MCQ-ML, and WikiPage-TS — are unseen only in the sense that no examples appear in the training corpus. Their questions are generated with the same Claude-3.5-Sonnet model family used to synthesize the Chapter 1 QA corpus and the WikiPage-TS questions. The 'unseen' label therefore does not mean distributionally independent: gains on these benchmarks may reflect familiarity with Claude-style question formats and phrasing rather than general analytics ability, so the generalization claim is not independently established.

full rationale

There is no equation-level circularity: the loss function, training recipe, and ablations are self-contained, and the paper is transparent about which evaluation datasets are 'Seen' versus 'Unseen.' The central claims, however, rely on benchmark averages that mix in-distribution fine-tuning gains with genuinely new evaluations. The headline 0.697 overall score and the 12.1% table-selection advantage over GPT-4o are obtained by averaging over BIRD-TS, Open-WikiTable-TS, Spider-dev, and BIRD-dev, all of whose training splits are in the CoddLLM corpus; on the only unseen table-selection dataset, WikiPage-TS, CoddLLM is below GPT-4o. Separately, the 'unseen' AnalyticsMMLU and WikiPage-TS benchmarks are generated with the same Claude-3.5-Sonnet model family that synthesized much of the training data, so they do not provide an independent distributional test. These are evaluation-validity concerns rather than derivational circularity, but they do weaken the paper's central generalization claim. The self-citations (e.g., Halevy's data-integration book, NameGuess) are not load-bearing, and no uniqueness theorem or ansatz is smuggled in via citation. Score 4 reflects partial circularity of the headline evaluation claims, not the training pipeline itself.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest primarily on corpus curation choices, automated judge filtering, transfer from synthetic alignment tasks, and the assumption that Claude-generated evaluation questions are independent of the Claude-generated training data. None are formal axioms; they are empirical modeling assumptions that a replication study would need to test.

free parameters (5)
  • Document filter threshold = score >= 4
    Documents from FineWeb-Edu rated below 4 by the classifier are discarded; this threshold determines the size and quality of Chapter 1 and is chosen by the authors.
  • Training epochs and data mix = 1 epoch Chapter 1; 2 epochs Chapters 2-3; 10% Chapter 1 mix
    The final checkpoint configuration was selected after experiments; the paper states 'The final endpoint is selected as CoddLLM' after training with these settings.
  • Prometheus-eval filtering threshold = score > 4
    Text-to-Schema scenario descriptions are retained only if the judge score is above four; this affects Chapter 2 data.
  • Inference sampling hyperparameters = temp 0.0/0.7/1.0; top_p 0.99/0.95/1.0
    The authors tuned these and then fixed them for all reported evaluations.
  • BM25 candidate count and row sample size = 10 candidates; 3 sample rows per table
    Open-WikiTable-TS uses top-10 BM25 retrieval and three sampled rows per table; these choices shape both training and evaluation for that task.
assumptions (4)
  • domain assumption FineWeb-Edu contains sufficient analytics-relevant knowledge for post-training.
    Chapter 1 is built by filtering FineWeb-Edu with a learned classifier; if the source corpus lacks domain knowledge, the pipeline cannot produce it. Invoked in Section 4.1.1.
  • domain assumption LLM-as-a-judge filtering (Prometheus-eval, Claude ratings) yields high-quality instruction data.
    Step 3 of Section 4.1.3 assumes automated judge scores correlate with human quality; the paper validates only the extraction step with 300 samples, not the full synthesis pipeline.
  • ad hoc to paper Text-to-Schema and Row-to-Text training transfers to table selection and SQL generation.
    Section 4.2 states these tasks enhance understanding of schema and table-text translation; the hypothesis is supported only by the ablation in Section 5.4.3, which is mostly on seen benchmarks.
  • domain assumption Claude-3.5-Sonnet-generated benchmark questions, after human review, are valid evaluation items.
    AnalyticsMMLU and WikiPage-TS use questions generated by Claude-3.5-Sonnet; the human review checks answers and labels but does not eliminate stylistic overlap with training data generated by the same model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoddLLM: Empowering Large Language Models for Data Analytics." pith.science (2026). https://pith.science/paper/6AJ5U7C7

@misc{pith2026250200329,
  author       = {Pith},
  title        = {Pith review of: CoddLLM: Empowering Large Language Models for Data Analytics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AJ5U7C7}},
  note         = {Machine review of arXiv:2502.00329}
}
read the original abstract

Large Language Models (LLMs) have the potential to revolutionize data analytics by simplifying tasks such as data discovery and SQL query synthesis through natural language interactions. This work serves as a pivotal first step toward the development of foundation models explicitly designed for data analytics applications. To propel this vision forward, we unveil a new data recipe for post-training LLMs, enhancing their comprehension of data management and empowering them to tackle complex real-world analytics tasks. Specifically, our innovative approach includes a scalable synthetic data generation method that enables the creation of a broad spectrum of topics centered on data representation and manipulation. Furthermore, we introduce two new tasks that seamlessly bridge tables and text. We show that such tasks can enhance models' understanding of schema creation and the nuanced translation between natural language and tabular data. Leveraging this data recipe, we post-train a new foundation model, named CoddLLM, based on Mistral-NeMo-12B. To assess the language understanding and reasoning capabilities of LLMs in the realm of data analytics, we contribute AnalyticsMMLU, a benchmark containing thousands of multiple-choice questions on databases, data analysis, and machine learning. Our focus on data discovery, has resulted in the contribution of three comprehensive benchmarks that address both database and data lake scenarios. CoddLLM not only excels in performance but also sets a new standard, achieving the highest average accuracy across eight datasets. It outperforms GPT-3.5-Turbo on AnalyticsMMLU, exceeding GPT-4o by 12.1% in table selection and showing an average improvement of 24.9% in Text-to-SQL compared to the base model.

Figures

Figures reproduced from arXiv: 2502.00329 by the authors.

Figure 1
Figure 1. Building Chapter 1 data. Step 1 (the top and middle [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. A question and wikipage data sample from [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

87 extracted references · 28 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. Gqa: Training generalized multi-query trans- former models from multi-head checkpoints. arXiv preprint arXiv:2305.13245 (2023)

    arXiv 2023

  3. [3]

    Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Lo...

    2019

  4. [4]

    Eric Anderson, Jonathan Fritz, Austin Lee, Bohou Li, Mark Lindblad, Henry Lindeman, Alex Meyer, Parth Parmar, Tanvi Ranade, Mehul A Shah, et al. 2024. The Design of an LLM-powered Unstructured Analytics System. arXiv preprint arXiv:2409.00847 (2024)

    arXiv 2024

  5. [5]

    Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long- document transformer. arXiv preprint arXiv:2004.05150 (2020)

    arXiv 2020

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Ka- plan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott ...

    2020

  7. [7]

    Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Yuchen Mao, Wenjing Hu, et al. 2024. Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? arXiv preprint arXiv:2407.10956 (2024)

    arXiv 2024

  8. [8]

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

Show all 87 references
  1. [9]

    Daixuan Cheng, Yuxian Gu, Shaohan Huang, Junyu Bi, Minlie Huang, and Furu Wei. 2024. Instruction Pre-Training: Language Models are Supervised Multitask Learners. In Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing. Association for Comput...

    2024 doi

  2. [10]

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research 25, 70 (2024), 1–53

    2024

  3. [11]

    AnHai Doan, Alon Halevy, and Zachary Ives. 2012. Principles of data integration . Elsevier

    2012

  4. [12]

    Till Döhmen, Radu Geacu, Madelon Hulsebos, and Sebastian Schelter. 2024. SchemaPile: A Large Collection of Relational Database Schemas. Proceedings of the ACM on Management of Data 2, 3 (2024). https://doi.org/10.1145/3654975

    2024 doi

  5. [13]

    Raul Castro Fernandez, Aaron J Elmore, Michael J Franklin, Sanjay Krishnan, and Chenhao Tan. 2023. How large language models will disrupt data management. Proceedings of the VLDB Endowment 16, 11 (2023), 3302–3309

    2023

  6. [14]

    Benjamin Feuer, Yurong Liu, Chinmay Hegde, and Juliana Freire. 2023. ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models. arXiv preprint arXiv:2310.18208 (2023)

    2023 arXiv

  7. [15]

    Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, et al. 2024. XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL. arXiv preprint arXiv:2411.08599 (2024)

    2024 arXiv

  8. [16]

    Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al . 2023. Textbooks are all you need. arXiv preprint arXiv:2306.11644 (2023)

    2023 arXiv

  9. [17]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language under- standing. arXiv preprint arXiv:2009.03300 (2020)

    2020 arXiv

  10. [18]

    Danny Hernandez, Tom Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Tom Henighan, Tristan Hume, et al. 2022. Scaling laws and interpretability of learning from repeated data. arXiv preprint arXiv:2205.10487 (2022)

    2022 arXiv

  11. [19]

    Sirui Hong, Yizhang Lin, Bangbang Liu, Binhao Wu, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Lingyao Zhang, Mingchen Zhuge, et al. 2024. Data interpreter: An LLM agent for data science. arXiv preprint arXiv:2402.18679 (2024)

    2024 arXiv

  12. [20]

    Xueyu Hu, Ziyu Zhao, Shuang Wei, Ziwei Chai, Qianli Ma, Guoyin Wang, Xuwu Wang, Jing Su, Jingjing Xu, Ming Zhu, Yao Cheng, Jianbo Yuan, Ji- wei Li, Kun Kuang, Yang Yang, Hongxia Yang, and Fei Wu. 2024. InfiAgent- DABench: Evaluating Agents on Data Analysis Tasks. In Proceeding...

    2024

  13. [21]

    Madelon Hulsebos, Wenjing Lin, Shreya Shankar, and Aditya Parameswaran

  14. [22]

    Siegel, Nitya Nadgir, and Arvind Narayanan

    Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, and Arvind Narayanan. 2024. AI Agents That Matter. arXiv:2407.01502 [cs.LG] https: //arxiv.org/abs/2407.01502

    2024 arXiv

  15. [23]

    In Proceedings of the 2024 Workshop on Human-In-the-Loop Data Analytics

    It Took Longer than I was Expecting: Why is Dataset Search Still so Hard?. In Proceedings of the 2024 Workshop on Human-In-the-Loop Data Analytics . 1–4

    2024

  16. [24]

    Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu

  17. [25]

    Moe Kayali, Anton Lykov, Ilias Fountalis, Nikolaos Vasiloglou, Dan Olteanu, and Dan Suciu. 2024. Chorus: Foundation Models for Unified Data Discovery and Exploration. Proceedings of the VLDB Endowment 17, 8 (may 2024), 2104–2114. https://doi.org/10.14778/3659437.3659461

    2024

  18. [26]

    Keti Korini and Christian Bizer. 2024. Column Property Annotation using Large Language Models. ESWC: European Semantic Web Conference (2024)

    2024

  19. [27]

    Sunjun Kweon, Yeonsu Kwon, Seonhee Cho, Yohan Jo, and Edward Choi. 2023. Open-WikiTable : Dataset for Open Domain Question Answering with Complex Reasoning over Table. InFindings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguist...

    2023

  20. [28]

    Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo

  21. [29]

    In The Twelfth International Conference on Learning Representations

    Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models. In The Twelfth International Conference on Learning Representations . https://openreview.net/forum?id=8euJaTveKw

  22. [30]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing...

    2020

  23. [31]

    Fei Li and Hosagrahar V Jagadish. 2014. Constructing an interactive natural language interface for relational databases. Proceedings of the VLDB Endowment 8, 1 (2014), 73–84

    2014

  24. [32]

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al . 2023. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. 37th Conference on Neural Information Proces...

    2023

  25. [33]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAtten- tion. In Proceedings of the ACM SIGOPS 29th Symposium on Operating...

    2023

  26. [34]

    Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettle- moyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. DS-1000: A natural and reliable benchmark for data science code generation. InInternational Conference on Machine Learning . PMLR, 18319–18345

    2023

  27. [35]

    Ilya Loshchilov, Frank Hutter, et al. 2017. Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101 5 (2017)

    2017 arXiv

  28. [36]

    Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. 2024. FineWeb-Edu: the Finest Collection of Educational Content. https://doi.org/10. 57967/hf/2497

    2024

  29. [37]

    Madhavan, P.A

    J. Madhavan, P.A. Bernstein, A. Doan, and A. Halevy. 2005. Corpus-based schema matching. In 21st International Conference on Data Engineering (ICDE’05) . 57–68. https://doi.org/10.1109/ICDE.2005.39

    2005 doi

  30. [38]

    Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. 2024. Table-GPT: Table Fine-tuned GPT for Diverse Table Tasks. Proceedings of the ACM on Management of Data 2, 3, Article 176 (may 2024), 28 page...

    2024

  31. [39]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?...

    2024

  32. [40]

    Avanika Narayan, Ines Chami, Laurel Orr, and Christopher Ré. 2022. Can Foun- dation Models Wrangle Your Data? Proceedings of the VLDB Endowment 16, 4 (dec 2022), 738–746. https://doi.org/10.14778/3574245.3574258

    2022

  33. [41]

    Vaishali Pal, Andrew Yates, Evangelos Kanoulas, and Maarten de Rijke. 2023. MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Associati...

    2023 doi

  34. [42]

    Liana Patel, Siddharth Jha, Carlos Guestrin, and Matei Zaharia. 2024. LOTUS: En- abling Semantic Queries with LLMs Over Tables of Unstructured and Structured Data. arXiv preprint arXiv:2407.11418 (2024)

    2024 arXiv

  35. [43]

    Matías Mendieta, Boran Han, Xingjian Shi, Yi Zhu, and Chen Chen. 2023. Towards geospatial foundation models via continual pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 16806–16816

    2023

  36. [44]

    Yev Meyer, Marjan Emadi, Dhruv Nathawani, Lipika Ramaswamy, Kendrick Boyd, Maarten Van Segbroeck, Matthew Grossman, Piotr Mlocek, and Drew Newberry

  37. [45]

    https://huggingface.co/ Zhang et al

    Synthetic-Text-To-SQL: A synthetic dataset for training language models to generate SQL queries from natural language prompts . https://huggingface.co/ Zhang et al. datasets/gretelai/synthetic-text-to-sql

  38. [46]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

    2019

  39. [47]

    Stephen Robertson, Hugo Zaragoza, et al . 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389

    2009

  40. [48]

    Noam Shazeer. 2020. Glu variants improve transformer. arXiv preprint arXiv:2002.05202 (2020)

    2020 arXiv

  41. [49]

    Ralph Peeters and Christian Bizer. 2023. Entity matching using large language models. arXiv preprint arXiv:2310.11244 (2023)

    2023 arXiv

  42. [50]

    Guilherme Penedo, Hynek Kydlíček, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf, et al. 2024. The FineWeb Datasets: De- canting the Web for the Finest Text Data at Scale.arXiv preprint arXiv:2406.17557 (2024)

    2024 arXiv

  43. [51]

    Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O Arik

  44. [52]

    arXiv preprint arXiv:2410.01943 (2024)

    Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql. arXiv preprint arXiv:2410.01943 (2024)

    2024 arXiv

  45. [53]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  46. [54]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Atten- tion is All you Need. In Advances in Neural Information Processing Sys- tems, Vol. 30. https://proceedings.neurips.cc/paper_files/paper/20...

    2017

  47. [55]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. MMLU-Pro: A More Robust and Challenging Multi-Task Language ...

    2024 arXiv

  48. [56]

    Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, Haoze Li, et al. 2024. Tablegpt2: A large multimodal model with tabular data integration. arXiv preprint arXiv:2411.02059 (2024)

    2024 arXiv

  49. [57]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing 568 (2024), 127063

    2024

  50. [58]

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755 (2024)

    2024 arXiv

  51. [59]

    Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic

  52. [60]

    arXiv 2022

    Galactica: A large language model for science. arXiv 2022. arXiv preprint arXiv:2211.09085 10 (2023)

    2023 arXiv

  53. [61]

    Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xinrun Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, et al . 2024. TableBench: A Comprehensive and Complex Benchmark for Table Question Answering. arXiv preprint arXiv:2408.09174 (2024)

    2024 arXiv

  54. [62]

    Yong Xie, Karan Aggarwal, and Aitzaz Ahmad. 2023. Efficient continual pre- training for building domain specific large language models. arXiv preprint arXiv:2311.08545 (2023)

    2023

  55. [63]

    Longhui Yu, Weisen Jiang, Han Shi, YU Jincheng, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models. In The Twelfth International Conference on Learning Representations

    2024

  56. [64]

    Johannes Wehrstein, Carsten Binnig, Fatma Özcan, Shobha Vasudevan, Yu Gan, and Yawen Wang. 2025. Towards Foundation Database Models. (2025)

    2025

  57. [65]

    Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. [n. d.]. Magicoder: Empowering code generation with oss-instruct. In Proceedings of the 41st International Conference on Machine Learning , Vol. 2024. Article 2158, 26 pages

    2024

  58. [66]

    Niklas Wretblad, Fredrik Gordh Riseby, Rahul Biswas, Amin Ahmadi, and Os- kar Holmström. 2024. Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmark. arXiv:2402.12243 [cs.CL] https://arxiv.org/abs/2402.12243

    2024 arXiv

  59. [67]

    Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yan- feng Wang. 2024. PMC-LLaMA: toward building open-source language models for medicine. Journal of the American Medical Informatics Association (2024), ocae045

    2024

  60. [68]

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebas- tian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann

  61. [69]

    arXiv preprint arXiv:2303.17564 (2023)

    Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564 (2023)

    2023 arXiv

  62. [70]

    Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Shen Wang, Huzefa Rangwala, and George Karypis. 2023. NameGuess: Column Name Expansion for Tabular Data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computat...

    2023 doi

  63. [71]

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792 (2023)

    2023

  64. [72]

    Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. 2024. TableLlama: Towards Open Large Generalist Models for Tables. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Lon...

    2024 doi

  65. [73]

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. InPr...

    2018 doi

  66. [74]

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=yLClGs770I

    2024

  67. [75]

    Xiang Yue, Tuney Zheng, Ge Zhang, and Wenhu Chen. 2024. Mammoth2: Scaling instructions from the web. arXiv preprint arXiv:2405.03548 (2024)

    2024 arXiv

  68. [76]

    Bin Zhang, Yuxiao Ye, Guoqing Du, Xiaoru Hu, Zhishuai Li, Sun Yang, Chi Harold Liu, Rui Zhao, Ziyue Li, and Hangyu Mao. 2024. Benchmarking the text-to-sql capability of large language models: A comprehensive evaluation. arXiv preprint arXiv:2403.02951 (2024)

    2024

  69. [77]

    Dan Zhang, Ziniu Hu, Sining Zhoubian, Zhengxiao Du, Kaiyu Yang, Zihan Wang, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models. In The Thirty-eight Conference on Neural Information P...

    2024

  70. [78]

    Haochen Zhang, Yuyang Dong, Chuan Xiao, and Masafumi Oyamada. 2023. Jellyfish: A large language model for data preprocessing. arXiv preprint arXiv:2312.01678 (2023)

    2023 arXiv

  71. [82]

    Yuge Zhang, Qiyang Jiang, XingyuHan XingyuHan, Nan Chen, Yuqing Yang, and Kan Ren. 2024. Benchmarking Data Science Agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 5677–5700. https://...

    2024

  72. [83]

    Zeyu Zhang, Paul Groth, Iacer Calixto, and Sebastian Schelter. 2024. AnyMatch– Efficient Zero-Shot Entity Matching with a Small Language Model.arXiv preprint arXiv:2409.04073 (2024)

    2024 arXiv

  73. [84]

    Zeyu Zhang, Paul Groth, Iacer Calixto, and Sebastian Schelter. 2024. Directions Towards Efficient and Automated Data Wrangling with Large Language Mod- els. In 2024 IEEE 40th International Conference on Data Engineering Workshops (ICDEW). IEEE, 301–304

    2024

  74. [85]

    Xinyang Zhao, Xuanhe Zhou, and Guoliang Li. 2024. Chat2Data: An Interactive Data Analysis System with RAG, Vector Databases and LLMs. Proceedings of the VLDB Endowment: Demonstration Track (2024)

    2024

  75. [86]

    Xuanhe Zhou, Xinyang Zhao, and Guoliang Li. 2024. LLM-Enhanced Data Management. arXiv preprint arXiv:2402.02643 (2024)

    2024 arXiv

  76. [87]

    Jun-Peng Zhu, Peng Cai, Boyan Niu, Zheming Ni, Kai Xu, Jiajun Huang, Jianwei Wan, Shengbo Ma, Bing Wang, Donghui Zhang, et al . 2024. Chat2Query: A Zero-Shot Automatic Exploratory Data Analysis System with Large Language Models. In 2024 IEEE 40th International Conference on Da...

    2024

  77. [2023]

    In The Eleventh International Conference on Learning Representations

    Continual pre-training of language models. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=m_ GDIItaI3o

  78. [2024]

    In Findings of the Asso- ciation for Computational Linguistics ACL 2024

    M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. In Findings of the Asso- ciation for Computational Linguistics ACL 2024 . Association for Computational Linguistics, 2318–2335. https://doi.org/10.18653/...

    2024 doi

  79. [8297]

    https://doi.org/10.18653/v1/2023.findings-acl.526

    2023 doi

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.