Pith. sign in

REVIEW 4 major objections 5 minor 69 references

ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ExeSQL claims that a 7B model fine-tuned with execution-validated, self-generated SQL beats GPT-4o on PostgreSQL, MySQL, and Oracle benchmarks.

desk verdict A genuinely useful pipeline for dialect text-to-SQL, but the paper's core claim that execution feedback yields reliable training signals is undermined by its own Appendix A.15, which shows 7 of 8 executable candidates returning wrong answers. read the letter →

arxiv 2505.17231 v1 pith:XMLPONCX submitted 2025-05-22 cs.CL cs.AIcs.DB

Jipeng Zhang , Haolin Yang , Kehao Miao , Ruiyuan Zhang , Renjie Pi , Jiahui Gao , Xiaofang Zhou This is my paper · ORCID
classification cs.CLcs.AIcs.DB
keywords text-to-SQLSQLdialectsexecutionfeedbackrejectionsamplingdirectpreferenceoptimizationself-taughtbootstrappingLLMfine-tuningdatasynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a 7B-parameter model fine-tuned on its own execution-validated SQL can beat GPT-4o at writing queries for PostgreSQL, MySQL, and Oracle. The ExeSQL pipeline bootstraps dialect training data by translating SQLite question–query pairs with an LLM, then iteratively generates candidate queries, runs them in a real database, and keeps only those that execute without error. Those kept queries are used for supervised fine-tuning and as positive examples in direct preference optimization (DPO). The abstract reports average improvements of 15.2%, 10.38%, and 4.49% over GPT-4o on the three dialects across multiple datasets. If the claim holds, execution feedback can substitute for expensive manual annotation when adapting text-to-SQL models to new SQL dialects.

What carries the argument

The central mechanism is the execution-based reward R(S), which returns 1 when a candidate SQL query runs without error and 0 when it fails, driving rejection sampling, data curation, and DPO preference pairs. The pipeline is dialect-agnostic: it needs only a live engine of the target dialect to score candidates. The translation bootstrapping stage provides the cold-start corpus that lets this loop start, and the loop itself iteratively expands the dataset with new questions and validated queries.

What would settle it

Re-evaluate the validated training set by comparing each kept query's execution result against the gold answer's result rows; if a random sample of 100 such queries shows the executable-but-wrong majority found in Appendix A.15, the execution-success reward is not a reliable proxy and the reported accuracy gains would not reflect true question-answering quality.

Watch

Extended reading notes

Core claim

ExeSQL's central claim is that a small open model, trained through a self-evolving loop of generation, execution, and preference optimization, can close the dialect gap in text-to-SQL. Stage one translates existing SQLite question–query pairs into target-dialect SQL with GPT-4o, feeding execution error messages back into the translator until a valid query is produced. Stage two samples multiple alternatives per question, executes them against a real database engine, retains executables in the validated set and failures in the negative set, and fine-tunes a 7B DeepSeek-Coder model on the validated set. Stage three applies DPO to raise the probability of executable queries over failed ones. The paper reports 66.70% average accuracy across the PostgreSQL, MySQL, and Oracle benchmarks, against 55.69% for GPT-4o, and shows that removing the iterative execution loop causes the largest drop in performance.

Load-bearing premise

The load-bearing premise is that a SQL query that executes without error is a correct answer; the paper's own Appendix A.15 shows a sampled case where only 1 of 8 executable queries answered the question correctly.

Editorial extensions

If this is right

  • A 7B open model beats GPT-4o on dialect-specific text-to-SQL, with average gains of 15.2%, 10.38%, and 4.49% on PostgreSQL, MySQL, and Oracle respectively.
  • The pipeline transfers to any dialect with a usable engine, requiring only SQLite question–query pairs plus a database server for the target dialect.
  • Iterative execution feedback is the main driver: removing it drops PostgreSQL accuracy from 71.98% to 63.49% and MySQL from 72.87% to 60.09%.
  • The trained model generalizes out-of-distribution, scoring 59.16% on PostgreSQL and 56.02% on MySQL for Dr.Spider, well above the best baseline's 40.00%.
  • Preference training also improves robustness under database and SQL perturbations, with the SFT+DPO model averaging 60.60% versus 58.85% for SFT alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reward's assumption that execution success equals correctness is the main risk: the paper's own Appendix A.15 shows a sampled case where only 1 of 8 executable queries answered correctly, so if that ratio is typical, the SFT and DPO positives are contaminated and reported accuracy overstates answer quality.
  • Because GPT-4o performs the bootstrapping translation, the final model inherits GPT-4o's translation errors and coverage gaps; a variation using a smaller translator or a fully engine-driven oracle would isolate how much of the gain comes from the execution loop itself.
  • The execution-driven loop is directly portable to other code-generation settings with deterministic verifiers, such as compilers or test runners, though SQL's cheap, deterministic execution makes it an especially favorable testbed.
  • The paper's Appendix A.6 shows mixed-dialect training sacrifices some per-dialect peak performance, suggesting production systems may prefer per-dialect experts or model routing over a single multi-dialect model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces ExeSQL, a pipeline that adapts a 7B text-to-SQL model to PostgreSQL, MySQL, and Oracle through three stages: (1) translation bootstrapping, in which GPT-4o translates SQLite queries into dialect SQL with execution-based correction; (2) iterative data generation, in which the student model proposes multiple SQL candidates that are filtered by whether they execute successfully; and (3) preference optimization, in which executable and non-executable candidates are used as DPO positive/negative pairs. The authors report sizable accuracy gains over GPT-4o and other baselines on Spider, WikiSQL, BIRD, Dr.Spider, and MimicSQL across the three dialects.

Significance. If the claims hold, ExeSQL provides a practical, data-centric recipe for adapting small text-to-SQL models to multiple SQL dialects using execution feedback, which would be a useful contribution to the field. The paper includes several strengths: it releases code, provides detailed appendices on the generation and evaluation process, reports execution overhead, and includes ablations that isolate the contributions of iterative refinement and preference learning. However, the central claim is currently weakened by a load-bearing issue: the training reward signals only whether a query executes, not whether it returns the correct answer, and the paper's own Appendix A.15 demonstrates that this reward admits many semantically wrong queries. In addition, the evaluation gold for dialect SQL is largely GPT-4o-generated, and GPT-4o is also the bootstrapping teacher and a zero-shot baseline, which creates a distributional advantage for the student. These issues need to be addressed before the reported improvements can be attributed to execution-verified quality.

major comments (4)
  1. [Section 3.3.2 and Appendix A.15] The reward R(S) defined in Section 3.3.2 returns 1 for any query that executes without error, and every such query is added to DValid and later used as an SFT target and as a DPO positive. Appendix A.15's own example shows the failure mode clearly: among 8 generated candidates for one WikiSQL question, only 1 returns the correct answer, while the other 7 execute successfully but are semantically wrong (e.g., using "Jan 16" instead of "January 16", or "> 24" instead of "= 24"). The paper never measures the false-positive rate of DValid, nor does it compare execution results to gold answers during training. Consequently, the large improvement attributed to iterative generation (Table 2: PostgreSQL 63.49 to 71.98) cannot be attributed to execution-verified correctness rather than to dataset size or question augmentation. The authors should either filter DValid by comparing execution results to gold answers or measure and report the false-positive rate and show that contaminated positives are not driving the gains.
  2. [Section 4.1] The evaluation description says only that the authors "verify execution accuracy" but never defines the metric. It should specify whether accuracy is the fraction of questions where the result set of the generated SQL exactly matches the result set of the gold SQL, and how ordering, duplicate rows, NULLs, empty result sets, and non-deterministic functions are handled. Without this definition, the headline numbers in Table 1 cannot be reproduced or independently verified.
  3. [Abstract and Table 1] The abstract claims "average improvements of 15.2%, 10.38%, and 4.49% over GPT-4o on PostgreSQL, MySQL, and Oracle, respectively." From Table 1, the PostgreSQL improvement is indeed 15.2 percentage points (average of Spider and WikiSQL), but the MySQL average improvement is 13.2 percentage points, not 10.38, and the Oracle average improvement is 4.62 percentage points, with 4.49 being the improvement on Spider only. Please reconcile the abstract numbers with the reported table or correct the abstract.
  4. [Sections 3.2, 4.2, and Appendix A.13] GPT-4o serves simultaneously as the bootstrapping teacher for training data, as a zero-shot baseline, and as the generator of the dialect-SQL gold standard (with partial manual correction of only the last 30/20 errors, per Appendix A.13). Since ExeSQL is trained to imitate GPT-4o's translations, part of the reported gain may reflect adaptation to the teacher's translation style rather than a general dialect capability. The authors should add a small human-verified gold subset or an independent teacher model for evaluation, and discuss the extent to which the comparison is affected by this circularity.
minor comments (5)
  1. [Section 4.1] The text says "we chose these two dialects" immediately after listing three dialects (PostgreSQL, MySQL, Oracle), and the Limitations section also says the work focuses on "two mainstream dialects" despite evaluating Oracle. Please correct these internal inconsistencies.
  2. [Section 4.2] The paragraph states "We evaluate four large language models (LLMs)" but then names only GPT-4o, Gemini-1.5-pro, and Llama3.1-Instruct; either list the fourth model or correct the count.
  3. [Section 3.1] Equation (2) uses R(S*, Ŝ) without defining a pairwise reward; the earlier reward R(S) was defined only for a single SQL. Please define the pairwise reward or replace it with a clearly specified preference objective.
  4. [References] The reference list contains duplicated entries (for example, several papers appear twice in the references section). Please deduplicate the bibliography.
  5. [Appendix A.7] The similarity comparison in Table 13 is described confusingly: "Our data (10K samples paired with 5K Spider samples) shows a lower average similarity score compared to the similarity within the Spider dataset (10K Spider samples paired with highest similarity HumanEval samples)" mixes different pairings and makes the claimed higher diversity hard to interpret. Please clarify exactly which sets are being compared.

Circularity Check

2 steps flagged · score 5.0 of 10

Partial circularity: training 'correctness' is defined as executability, and the dialect benchmark gold is generated by the same GPT-4o teacher that produced the training data.

  1. self definitional [Section 3.3.2 and Section 3.4; Appendix A.15, Table 23]
    "If Scand exectues successfully, i.e., R(Scand,i) = 1. The query is added to the validated dataset: DValid = DValid ∪ {(Qi, Scand,i)} ... Spos ∈ DValid, S neg ∈ DNeg These pairs allow the model to differentiate between correct and incorrect SQL."

    By construction, 'validated' and 'correct' in the training loop are defined as R(S)=1, i.e., successful execution. DPO positives are exactly the executable candidates, so preference learning is defined to prefer executable SQL, not SQL that returns the gold answer. Appendix A.15 (Table 23) shows 7 of 8 executable candidates return wrong rows (e.g., Date = 'Jan 16' instead of 'January 16'; Hurricanes_points > 24 instead of = 24), so the paper's own evidence shows the correctness notion in the method reduces to executability and is not answer correctness. Any claim that execution feedback yields 'reliable training signals' is therefore true by definition of R, not by verification against ground truth.

  2. other [Section 3.2, Section 4, Section 4.2, Appendix A.13]
    "1) An LLM (GPT-4o here) generates candidate SQL queries ... The bootstrap dataset and new questions for ExeSQL are generated using GPT-4o ... For baseline comparisons, we evaluate GPT-4o-2024-11-20 ... We used GPT-4o API to generate MySQL and PostgreSQL SQL queries based on the given SQLite SQL."

    The same GPT-4o model is both the teacher that generates the dialect training data (DTrans) and the zero-shot baseline, and the dialect 'ground truth' for the benchmarks is also a GPT-4o translation of the original SQLite queries (with manual correction of residual errors). The headline 'improvements over GPT-4o' therefore measure how well a 7B student imitates its teacher's dialect translations on teacher-generated labels. The comparison is not statistically forced---the student could still underperform---but the benchmark does not independently establish dialect competence; it is partly a distillation-fidelity test against the teacher that authored both training labels and evaluation gold.

full rationale

The core training objective is not entirely circular: the final evaluation executes generated SQL and compares against ground-truth results, so the 7B model's reported scores are not guaranteed by the equations. However, two load-bearing steps reduce the independence of the claimed result. First, the only correctness oracle in the bootstrapping loop is R(S)=1 (executability), and the paper explicitly labels executable candidates as 'validated' and DPO-positive, while its own Appendix A.15 demonstrates that 7/8 executable candidates are semantically wrong; thus the claimed 'execution-driven' quality signal is definitionally executability, not answer correctness. Second, the dialect evaluation gold is itself produced by GPT-4o, the same model used to generate the training data and used as the headline baseline, so the comparison is a distilled-student-vs-teacher evaluation on teacher-generated labels rather than an independent benchmark. These are partial circularities rather than a full self-derivation: nothing in the equations forces the reported accuracy numbers, and the manual-correction step in Appendix A.13 provides some external anchoring. Score 5 reflects substantial but not complete circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The reported gains rest on the assumption that execution success is a sufficient training signal, that the GPT-4o-built evaluation gold is reliable, and that the base model is not contaminated by the test benchmarks. Hand-picked hyperparameters (worst-of-N=8, temperature 0.7, epochs) are not swept, so the result is tied to that configuration.

free parameters (4)
  • Worst-of-N for DPO negative sampling = N=8
    Set in Appendix A.3; no sweep is reported, and the size of N controls the difficulty of the negative examples used in preference learning.
  • DPO training epochs = 3
    Set in Appendix A.3 together with SFT loss weight 1; the paper does not vary this, so the DPO contribution may be configuration-specific.
  • SFT dataset size = 20.6k samples (6.9k Spider, 10k WikiSQL, 3.7k new)
    The pipeline's performance could scale with data size; the paper does not separate the effect of data quantity from the effect of execution filtering.
  • Rejection sampling decoding parameters = temperature=0.7, top-p=0.9, top-k=50
    Appendix A.3; these control the diversity of generated candidates. The paper does not analyze sensitivity to these values.
assumptions (4)
  • domain assumption Execution success is treated as a sufficient criterion for accepting generated SQL as a valid training example.
    Section 3.3.2 defines R(S)=1 if the query executes and adds the query to DValid. The paper's own Appendix A.15 shows executable queries can return wrong answers.
  • domain assumption The dialect gold standard used for evaluation is a correct and unbiased measure of SQL quality.
    The gold dialect SQL is produced by GPT-4o translation with manual correction (Appendix A.13). The paper does not provide human verification of the evaluation gold or a native dialect benchmark.
  • domain assumption DeepSeek-Coder-7B has not already memorized the test examples or the translated dialect queries.
    The base model is a public code LLM; the paper asserts fairness because baseline models were released before the datasets, but it does not test for contamination.
  • standard math Standard SFT and DPO objectives can transfer the execution-filtered data into improved generation accuracy.
    The paper relies on the standard DPO/SFT training framework (Sections 3.3 and 3.4) without modification; this is a standard background assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects." pith.science (2026). https://pith.science/paper/XMLPONCX

@misc{pith2026250517231,
  author       = {Pith},
  title        = {Pith review of: ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XMLPONCX}},
  note         = {Machine review of arXiv:2505.17231}
}
read the original abstract

Recent text-to-SQL models have achieved strong performance, but their effectiveness remains largely confined to SQLite due to dataset limitations. However, real-world applications require SQL generation across multiple dialects with varying syntax and specialized features, which remains a challenge for current models. The main obstacle in building a dialect-aware model lies in acquiring high-quality dialect-specific data. Data generated purely through static prompting - without validating SQLs via execution - tends to be noisy and unreliable. Moreover, the lack of real execution environments in the training loop prevents models from grounding their predictions in executable semantics, limiting generalization despite surface-level improvements from data filtering. This work introduces ExeSQL, a text-to-SQL framework with execution-driven, agentic bootstrapping. The method consists of iterative query generation, execution-based filtering (e.g., rejection sampling), and preference-based training, enabling the model to adapt to new SQL dialects through verifiable, feedback-guided learning. Experiments show that ExeSQL bridges the dialect gap in text-to-SQL, achieving average improvements of 15.2%, 10.38%, and 4.49% over GPT-4o on PostgreSQL, MySQL, and Oracle, respectively, across multiple datasets of varying difficulty.

Figures

Figures reproduced from arXiv: 2505.17231 by the authors.

Figure 1
Figure 1. Given a natural language question, different [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Execution-based error feedback loop for dialect [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Pipeline for Dialect Text-to-SQL Data Generation and Model Training. The framework consists of three [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Retention rate of correct dialect SQL under [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: SQLite to PostgreSQL process [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 19 canonical work pages

  1. [1]

    Github copilot – your ai pair programmer

    Microsoft. Github copilot – your ai pair programmer. GitHub repository, 2023. URL https://github.com/features/copilot

    work page 2023

  2. [2]

    Services. A. w. ai code generator - amazon codewhisperer - aws. Amazon Page, 2023. URL https://aws.amazon.com/codewhisperer/

    work page 2023

  3. [4]

    Seq2sql: Generating structured queries from natural language using reinforcement learning

    Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103, 2017

    arXiv 2017

  4. [5]

    Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887, 2018

    arXiv 2018

  5. [6]

    Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems, 36, 2024 a

    work page 2024

  6. [7]

    Wang, and Tao Yu

    Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida I. Wang, and Tao Yu. Spider 2.0: Evaluating language models on real-world enterprise text-to-sql workflows. CoRR, abs/2411.07763, 2024. doi:10.48550/ARXIV.2411.07763. UR...

    arXiv doi:10.48550/arxiv.2411.07763 2024

  7. [8]

    Codes: Towards building open-source language models for text-to-sql

    Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. Codes: Towards building open-source language models for text-to-sql. Proceedings of the ACM on Management of Data, 2 0 (3): 0 1--28, 2024 b

    work page 2024

  8. [9]

    Structlm: Towards building generalist models for structured knowledge grounding

    Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W Huang, Jie Fu, Xiang Yue, and Wenhu Chen. Structlm: Towards building generalist models for structured knowledge grounding. arXiv preprint arXiv:2402.16671, 2024

    arXiv 2024

Show all 69 references
  1. [10]

    C3: Zero-shot text-to-sql with chatgpt

    Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, Jinshu Lin, Dongfang Lou, et al. C3: Zero-shot text-to-sql with chatgpt. arXiv preprint arXiv:2307.07306, 2023 a

    2023 arXiv

  2. [11]

    Din-sql: Decomposed in-context learning of text-to-sql with self-correction

    Mohammadreza Pourreza and Davood Rafiei. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. Advances in Neural Information Processing Systems, 36, 2024

    2024

  3. [12]

    Mac-sql: Multi-agent collaboration for text-to-sql

    Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Qian-Wen Zhang, Zhao Yan, and Zhoujun Li. Mac-sql: Multi-agent collaboration for text-to-sql. arXiv preprint arXiv:2312.11242, 2023 a

    2023 arXiv

  4. [13]

    Natural sql: Making sql easier to infer from natural language specifications

    Yujian Gan, Xinyun Chen, Jinxia Xie, Matthew Purver, John R Woodward, John Drake, and Qiaofu Zhang. Natural sql: Making sql easier to infer from natural language specifications. arXiv preprint arXiv:2109.05153, 2021

    2021 arXiv

  5. [14]

    Structure-grounded pretraining for text-to-sql

    Xiang Deng, Ahmed Hassan, Christopher Meek, Oleksandr Polozov, Huan Sun, and Matthew Richardson. Structure-grounded pretraining for text-to-sql. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language...

    2021

  6. [15]

    Mohammadreza Pourreza, Ruoxi Sun, Hailong Li, Lesly Miculicich, Tomas Pfister, and Sercan \" O . Arik. SQL-GEN: bridging the dialect gap for text-to-sql via synthetic data and model merging. CoRR, abs/2408.12733, 2024. doi:10.48550/ARXIV.2408.12733. URL https://doi.org/10.4855...

    2024 doi arXiv

  7. [16]

    Toby Mao. Sqlglot. https://github.com/tobymao/sqlglot, 2023. Accessed: 2024-06-09

    2023

  8. [17]

    Translating between sql dialects for cloud migration

    Ran Zmigrod, Salwa Alamir, and Xiaomo Liu. Translating between sql dialects for cloud migration. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, pages 189--191, 2024

    2024

  9. [18]

    Magicoder: Source code is all you need

    Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Source code is all you need. arXiv preprint arXiv:2312.02120, 2023

    2023 arXiv

  10. [19]

    Self-instruct: Aligning language model with self generated instructions

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022

    2022 arXiv

  11. [20]

    Recent advances in text-to-sql: A survey of what we have and what we expect

    Naihao Deng, Yulong Chen, and Yue Zhang. Recent advances in text-to-sql: A survey of what we have and what we expect. In Nicoletta Calzolari, Chu - Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key - Sun Choi, Pum - Mo Ryu, Hsin - Hsi Chen, Lucia Donatelli, Heng Ji, S...

    2022

  12. [21]

    TAPEX: table pre-training via learning a neural SQL executor

    Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian - Guang Lou. TAPEX: table pre-training via learning a neural SQL executor. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview...

    2022

  13. [22]

    Learning from executions for semantic parsing

    Bailin Wang, Mirella Lapata, and Ivan Titov. Learning from executions for semantic parsing. arXiv preprint arXiv:2104.05819, 2021

    2021 arXiv

  14. [23]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jef...

    2020

  15. [24]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

    2022

  16. [25]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023. doi:10.48550/arXiv.2303.08774. URL https://doi.org/10.48550/arXiv.2303.08774

    2023 doi arXiv

  17. [26]

    Chess: Contextual harnessing for efficient sql synthesis

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755, 2024

    2024 arXiv

  18. [27]

    Benchmarking meaning representations in neural semantic parsing

    Jiaqi Guo, Qian Liu, Jian - Guang Lou, Zhenwen Li, Xueqing Liu, Tao Xie, and Ting Liu. Benchmarking meaning representations in neural semantic parsing. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Na...

    2020 doi

  19. [28]

    Momq: Mixture-of-experts enhances multi-dialect query generation across relational and non-relational databases

    Zhisheng Lin, Yifu Liu, Zhiling Luo, Jinyang Gao, and Yu Li. Momq: Mixture-of-experts enhances multi-dialect query generation across relational and non-relational databases. arXiv preprint arXiv:2410.18406, 2024 a

    2024 arXiv

  20. [29]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, ...

    2021 arXiv

  21. [30]

    Deepseek-coder: When the large language model meets programming--the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024

    2024 arXiv

  22. [31]

    Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161, 2023 a

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161, 2023 a

    2023 arXiv

  23. [32]

    Code llama: Open foundation models for code

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  24. [33]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024

    2024 arXiv

  25. [34]

    Wizardcoder: Empowering code large language models with evol-instruct

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. International Conference on Learning Representations (ICLR), 2024

    2024

  26. [35]

    Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation

    Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, and Qiufeng Yin. Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation. arXiv preprint arXiv:2312.14187, 2023

    2023 arXiv

  27. [36]

    Wizardlm: Empowering large language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. International Conference on Learning Representations (ICLR), 2024

    2024

  28. [37]

    Octopack: Instruction tuning code large language models

    Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. arXiv preprint arXiv:2308.07124, 2023

    2023 arXiv

  29. [38]

    Ircoder: Intermediate representations make language models robust multilingual code generators

    Indraneil Paul, Jun Luo, Goran Glava s , and Iryna Gurevych. Ircoder: Intermediate representations make language models robust multilingual code generators. arXiv preprint arXiv:2403.03894, 2024

    2024 arXiv

  30. [39]

    Unicoder: Scaling code large language model via universal code

    Tao Sun, Linzheng Chai, Jian Yang, Yuwei Yin, Hongcheng Guo, Jiaheng Liu, Bing Wang, Liqun Yang, and Zhoujun Li. Unicoder: Scaling code large language model via universal code. arXiv preprint arXiv:2406.16441, 2024

    2024 arXiv

  31. [41]

    Self-guided noise-free data generation for efficient zero-shot learning, 2023 a

    Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Weizhong Zhang, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. Self-guided noise-free data generation for efficient zero-shot learning, 2023 a . URL https://arxiv.org/abs/2205.12679

    2023 arXiv

  32. [42]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan L. Boyd - Graber, and Naoaki Okazaki, editors, Proceedings of the 6...

    2023

  33. [43]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825, 2023

    2023 arXiv

  34. [44]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  35. [45]

    Star: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35: 0 15476--15488, 2022

    2022

  36. [46]

    G-llava: Solving geometric problem with multi-modal large language model, 2023 b

    Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong. G-llava: Solving geometric problem with multi-modal large language model, 2023 b . URL https://arxiv.org/abs/2312.11370

    2023 arXiv

  37. [47]

    Personalized visual instruction tuning, 2024 a

    Renjie Pi, Jianshu Zhang, Tianyang Han, Jipeng Zhang, Rui Pan, and Tong Zhang. Personalized visual instruction tuning, 2024 a . URL https://arxiv.org/abs/2410.07113

    2024 arXiv

  38. [48]

    Mitigating hallucination in large multi-modal models via robust instruction tuning, 2024 a

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning, 2024 a . URL https://arxiv.org/abs/2306.14565

    2024 arXiv

  39. [49]

    Videodpo: Omni-preference alignment for video diffusion generation, 2024 b

    Runtao Liu, Haoyu Wu, Zheng Ziqiang, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. Videodpo: Omni-preference alignment for video diffusion generation, 2024 b . URL https://arxiv.org/abs/2412.14167

    2024 arXiv

  40. [50]

    Image textualization: An automatic framework for creating accurate and detailed image descriptions, 2024 b

    Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, and Tong Zhang. Image textualization: An automatic framework for creating accurate and detailed image descriptions, 2024 b . URL https://arxiv.org/abs/2406.07502

    2024 arXiv

  41. [51]

    Allava: Harnessing gpt4v-synthesized data for lite vision-language models, 2024

    Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing gpt4v-synthesized data for lite vision-language models, 2024. URL https://arxiv.org/abs/2402.11684

    2024 arXiv

  42. [52]

    RAFT: reward ranked finetuning for generative foundation model alignment

    Hanze Dong, Wei Xiong, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. RAFT: reward ranked finetuning for generative foundation model alignment. CoRR, abs/2304.06767, 2023 b . doi:10.48550/ARXIV.2304.06767. URL https://doi.org/10.48550/arXiv.2...

    2023 doi arXiv

  43. [53]

    Reinforced self-training (rest) for language modeling

    C aglar G \" u l c ehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. Reinforced self-training (rest) for langu...

    2023 doi arXiv

  44. [54]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  45. [55]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36: 0 53728--53741, 2023

    2023

  46. [56]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv pre...

    2024 arXiv

  47. [57]

    Shuaichen Chang, Jun Wang, Mingwen Dong, Lin Pan, Henghui Zhu, Alexander Hanbo Li, Wuwei Lan, Sheng Zhang, Jiarong Jiang, Joseph Lilien, et al. Dr. spider: A diagnostic evaluation benchmark towards text-to-sql robustness. arXiv preprint arXiv:2301.08881, 2023

    2023 arXiv

  48. [58]

    Ping Wang, Tian Shi, and Chandan K. Reddy. Text-to-sql generation for question answering on electronic medical records, 2020. URL https://arxiv.org/abs/1908.01839

    2020 arXiv

  49. [59]

    Recent advances in text-to- SQL : A survey of what we have and what we expect

    Naihao Deng, Yulong Chen, and Yue Zhang. Recent advances in text-to- SQL : A survey of what we have and what we expect. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao K...

    2022

  50. [60]

    https://ai.meta.com/blog/meta-llama-3/

    Meta llama 3. https://ai.meta.com/blog/meta-llama-3/. Accessed: 2024-06-10

    2024

  51. [61]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  52. [62]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024 arXiv

  53. [63]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems...

    2023

  54. [64]

    Lima: Less is more for alignment, 2023 b

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. Lima: Less is more for alignment, 2023 b . URL https://arxiv.org/abs/2305.11206

    2023 arXiv

  55. [65]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting i...

    2017 doi

  56. [66]

    Mitigating catastrophic forgetting in language transfer via model merging, 2024

    Anton Alexandrov, Veselin Raychev, Mark Niklas Müller, Ce Zhang, Martin Vechev, and Kristina Toutanova. Mitigating catastrophic forgetting in language transfer via model merging, 2024. URL https://arxiv.org/abs/2407.08699

    2024 arXiv

  57. [67]

    Scalebio: Scalable bilevel optimization for llm data reweighting, 2024

    Rui Pan, Jipeng Zhang, Xingyuan Pan, Renjie Pi, Xiaoyu Wang, and Tong Zhang. Scalebio: Scalable bilevel optimization for llm data reweighting, 2024. URL https://arxiv.org/abs/2406.19976

    2024 arXiv

  58. [68]

    Le, Tengyu Ma, and Adams Wei Yu

    Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V. Le, Tengyu Ma, and Adams Wei Yu. Doremi: Optimizing data mixtures speeds up language model pretraining, 2023. URL https://arxiv.org/abs/2305.10429

    2023 arXiv

  59. [69]

    Mitigating the alignment tax of rlhf, 2024 b

    Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, and Tong Zhang. Mitigating the alignment tax of rlhf, 2024 b . URL https://arxiv.org/abs/2...

    2024 arXiv

  60. [70]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st A...

    2023

  61. [71]

    Can LLM already serve as a database interface? a BI g bench for large-scale database grounded text-to- SQL s

    Jinyang Li, Binyuan Hui, GE QU, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chang, Fei Huang, Reynold Cheng, and Yongbin Li. Can LLM already serve as a database interface? a BI g bench for large-sc...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.