Pith. sign in

REVIEW 4 major objections 6 minor 50 references

ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Multitask tuning and prompting push open LLMs to lead Text2SQL

desk verdict A well-ablated multitask SFT + collaborative prompting recipe for open-source Text2SQL; the core result holds up, but the 'leading performance' claim outruns the significance testing and the baseline comparison is not fully controlled. read the letter →

arxiv 2412.10138 v3 pith:XTXHSKF2 submitted 2024-12-13 cs.CL cs.AI

Yang Qin , Chao Chen , Zhihang Fu , Ze Chen , Dezhong Peng , Peng Hu , Jieping Ye This is my paper · ORCID
classification cs.CLcs.AI
keywords multitasksupervisedfine-tuningschemalinkingtext-to-SQLLLMpromptingnoisecorrectioncontinuationwritingopen-sourceLLMsSPIDERandBIRDbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that open-source LLMs can reach leading Text2SQL performance if they are fine-tuned on several SQL-related tasks at once and then asked to perform those tasks in sequence at inference. The proposed ROUTE method combines multitask supervised fine-tuning (MSFT) over four tasks — Text2SQL, schema linking, noise correction, and continuation writing — with a multitask collaboration prompting (MCP) pipeline. On SPIDER and BIRD, ROUTE with Qwen2.5-14B reports 87.3 and 60.9 execution accuracy, the best among the open-source fine-tuning baselines compared and close to prompting pipelines built on GPT-4. If the claim holds, organizations can get near-proprietary-model SQL generation while keeping data on their own hardware, because the whole pipeline runs on a single open model.

What carries the argument

The load-bearing mechanism is the four-task MSFT objective followed by the three-step MCP loop. The central object is a fused simplified database $\tilde{d}_i$ from Equation (2): the model's schema linking output is merged with tables and columns fuzzy-matched from a pseudo-SQL query generated on the full schema. That merged schema is fed to a Text2SQL step, whose output goes through noise correction that consults execution exceptions and, for hard or failing queries, continuation writing from a truncated prefix. The merge operation $\uplus$ is what prevents the pseudo-SQL from silently dropping relevant entities; the paper's ablations show this schema-linking step contributes the largest share of the MCP improvement, with noise correction and continuation writing adding the rest.

What would settle it

Take a held-out set of questions in a new domain, run ROUTE, and compare the tables and columns in the fused simplified database $\tilde{d}_i$ with the tables and columns appearing in the ground-truth SQL. If a substantial fraction of wrong answers have ground-truth columns missing from $\tilde{d}_i$, the load-bearing schema-linking assumption fails; if the fused schema has full recall and EX is still low, the bottleneck lies elsewhere.

Watch

Extended reading notes

Core claim

ROUTE's central claim is that the reason small open-source models lag behind closed ones on Text2SQL is not lack of SQL knowledge alone, but a mismatch between the single task they are trained for and the several subtasks that accurate SQL generation requires. By fine-tuning one model on Text2SQL, schema linking, noise correction, and continuation writing, and then at inference linking the schema, generating SQL, checking the generated query against execution errors, and rewriting difficult queries from a prefix, the same model can outperform single-task SFT and narrow the gap with GPT-4-based prompting. The strongest reported evidence is Table 1: ROUTE with Qwen2.5-14B scores 87.3 EX on SPIDER dev and 60.9 EX on BIRD dev, versus 84.9 and 58.5 for CODES-15B and 83.2 and 51.8 for SENSE-7B in the same table.

Load-bearing premise

The approach relies on the first rough SQL guess being good enough that the simplified database built from it does not drop a table or column the true answer needs.

Editorial extensions

If this is right

  • A single open model can carry the whole Text2SQL pipeline, so deployment does not need a separate schema-linking model the way DTS-SQL does.
  • Training on one task alone erodes the other SQL-related skills; the paper's MSFT is what keeps schema linking, noise correction, and continuation writing usable, and without them MCP gains shrink.
  • The MCP prompting stage transfers across models and sizes, improving 7B code models such as CodeLlama and Deepseek-Coder and also 70B-class Llama3 and Qwen2.5, with the largest average gains on BIRD.
  • Schema linking is the dominant contributor to MCP's gains, and an ideal schema-linking oracle would lift ROUTE with Llama3-8B to 87.4 EX on SPIDER and 69.6 EX on BIRD, indicating where future work should focus.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the four-task recipe is not SQL-specific in its mechanism, so the same MSFT-plus-collaboration design is a plausible template for other schema-grounded generation tasks, such as text-to-Pandas or text-to-GraphQL, where hallucination follows from schema misalignment.
  • Editorial extension: the paper's Table 5 implies a cheap diagnostic: measure recall of the fused simplified database's columns against the ground-truth SQL's columns; failures that coincide with omitted columns would identify exactly which questions are at risk before execution.
  • Editorial extension: because the ablations give each MCP component's marginal contribution, one could build a budget-aware variant that runs schema linking and Text2SQL always, noise correction only on execution failure, and continuation writing only on hard queries, sacrificing a little accuracy for fewer LLM calls.
  • Editorial extension: noisy correspondence filtering helps SPIDER but slightly hurts BIRD in the ablations, suggesting the filtering threshold could be tuned per dataset or replaced by a curriculum that keeps hard examples late in training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ROUTE, a two-stage method for improving Text2SQL performance of open-source LLMs. In the first stage, Multitask Supervised Fine-Tuning (MSFT) trains a single model on four SQL-related tasks: Text2SQL, schema linking, noise correction, and continuation writing, with synthetic data derived from the SPIDER and BIRD training folds. In the second stage, Multitask Collaboration Prompting (MCP) combines these capabilities at inference: schema linking plus pseudo-SQL-based database simplification, SQL-executor feedback with noise correction, and continuation writing for hard queries. The paper reports execution accuracy and test-suite accuracy on five benchmarks, with the strongest result being ROUTE with Qwen2.5-14B reaching 87.3 EX on SPIDER dev and 60.9 EX on BIRD dev, exceeding several open-source fine-tuning baselines and closing part of the gap with GPT-4-based prompting methods. The central claim is that multitask tuning plus collaborative prompting makes open-source models practical for Text2SQL without relying on closed-source APIs.

Significance. If the reported results hold, ROUTE is a practically valuable recipe for on-premises Text2SQL: it uses only open-source models, derives training data programmatically from public training sets, and shows strong transfer across multiple base LLMs and robustness benchmarks. The paper has several genuine strengths: evaluation on five benchmarks and eight open-source LLMs, including Dr.Spider variants; a detailed ablation study isolating the contributions of MSFT, MCP, and each prompting component; explicit prompt templates and training settings in the appendix; and released code and synthetic data. There is no circularity in the evaluation setup: training uses the public training folds and held-out dev sets are used for testing. However, the head-to-head claim of outperforming prior open-source fine-tuning methods is not yet conclusive, because the reported margins are small, no uncertainty estimates are given, and the comparison mixes different inference budgets and potentially different training settings.

major comments (4)
  1. [Section 4.1, Table 1] The central empirical claim that ROUTE outperforms existing open-source fine-tuning methods is not supported with uncertainty quantification. The reported margins over CODES are small (1.9 EX on SPIDER dev and 2.4 EX on BIRD dev for Qwen2.5-14B), and the paper reports a single run per configuration at temperature 0.01. Fine-tuning is sensitive to seed, data ordering, and evaluation-harness differences, and temperature 0.01 does not control those sources of variance. I ask for at least three independent fine-tuning runs per main configuration, or a justified alternative such as confidence intervals or bootstrap significance testing, so that the reader can assess whether the reported margins are meaningful.
  2. [Section 3.3 and Table 1] The comparison against CODES and SENSE is not fully controlled: ROUTE's inference pipeline includes SQL-executor exception feedback in the noise-correction step and continuation writing, whereas the fine-tuning baselines are reported as single-pass SQL generators. Part of the observed gain may therefore come from additional inference-time computation rather than from the proposed multitask training. Please provide a controlled experiment in which the same multi-step inference procedure, including executor feedback and continuation writing, is applied to the fine-tuned baselines, or report ROUTE without NC and CW to match the baseline inference budget. This attribution is load-bearing for the claim that multitask training, rather than extra compute, drives the improvement.
  3. [Section 3.3, Eq. (2), Table 5] The enhanced schema-linking strategy depends on the accuracy of the pseudo-SQL generated by the model on the full schema. Table 5 shows that relying on SL_sigma_t alone can degrade SPIDER EX from 69.3 to 64.5 on base Llama3-8B, and the paper's fusion mitigates this on the tested benchmarks. The generalization risk remains for new domains where the pseudo-SQL is inaccurate and the simplified database may omit relevant entities. I request a direct analysis of pseudo-SQL quality, for example the execution-match rate against ground-truth SQL or a qualitative study of cases where the fusion drops a needed table or column, together with a statement of the conditions under which the simplification is safe. This is a correctness-risk concern rather than a claim that the method fails on the evaluated benchmarks.
  4. [Section 4.1 and Table 1] The baseline comparison is not fully controlled in terms of training data and evaluation protocol. DTS-SQL is marked as re-evaluated with an open-source repository, but CODES and SENSE results appear to be taken from their original papers, which may use different training sets, different base-model checkpoints, and different evaluation settings. Please state explicitly for every fine-tuning baseline which training folds were used, which checkpoints were used, and whether the numbers are re-evaluated or quoted; where feasible, retrain and evaluate the baselines under identical conditions so that the margins in Table 1 are directly comparable.
minor comments (6)
  1. [Section 4.3] The text says "Table 5 shows the performance on the benchmarks derived from SPIDER," but the correct reference is Table 2.
  2. [Section 4.2 and Appendix A.3] The BIRD dev EX score for ROUTE with Qwen2.5-14B is reported as 60.8 in Section 4.2 and in Table 11, but Table 1 reports 60.9; please make these consistent.
  3. [Section 3.2] The sentence containing "widely exist in various field ( ???? )" has a malformed citation placeholder and should be corrected or removed.
  4. [Appendix A.4 and A.6] There are typos such as "demonstartes" and "Precession" for "Precision." Also, Appendix A.3 misspells "Gemini" as "Gimini" and "CHASE-SQL" as "CHASS-SQL."
  5. [Abstract and Appendix A.3] The abstract claims "leading performance," but Appendix A.3 shows CHASE-SQL + Gemini 1.5 outperforming ROUTE on both SPIDER and BIRD; please soften the claim to "competitive with closed-source prompting methods" or clearly contextualize the comparison.
  6. [Table 2] The header row for Table 2 is difficult to parse because the column labels for SYN, Realistic, and DK are interleaved with the metric names; consider restructuring the table so each metric is clearly aligned with its column.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the evaluation is self-contained against external benchmarks and the derivation chain does not reduce to its own inputs.

full rationale

The paper's central claim is that multitask supervised fine-tuning (MSFT) plus multitask collaboration prompting (MCP) improves open-source LLM Text2SQL performance. I walked the derivation chain and found no step where an output is equivalent to an input by construction. The MSFT data are synthesized programmatically from the public SPIDER/BIRD training folds: Text2SQL responses are ground-truth SQL, schema-linking labels are parsed from ground-truth SQL via f(s_i,d_i), noise-correction labels are built from execution-success/failure comparisons, and continuation-writing responses are truncated ground-truth SQL. None of these use the development-set answers that are later reported. The MCP schema-linking fusion in Eq. (2) combines the model's schema linking with entities parsed from a pseudo-SQL generated on the full schema; the final SQL is then generated on the simplified schema. This is a two-stage pipeline, not a tautology: the pseudo-SQL is produced before the schema is simplified, and the final query is generated anew on the simplified schema. The ablation tables (Tables 4 and 5) decompose the contribution of each MCP component and of SL_sigma_s versus SL_sigma_t, showing that neither component alone equals the full method, so the reported gain is not just a relabeled fit. The upper-bound experiments in Table 7 use oracle inputs (ground-truth simplified schema or half of the ground-truth SQL) and are explicitly presented as upper bounds, not as predictions. The paper's citations to prior work are external, not self-citations forming a load-bearing chain, and no uniqueness theorem or forced-choice argument is imported. The skeptical concerns about small margins over CODES/SENSE and lack of error bars are legitimate threats to the strength of the empirical claim, but they are statistical-evidence issues, not circularity: the dev-set evaluations are held out, the training data come from public training folds, and no fitted parameter is renamed as a prediction. I therefore find no specific reduction of the claimed results to the paper's own inputs or to self-citation, and the appropriate circularity score is 0.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on six free design choices (most of them unspecified) and six domain assumptions. No new physical or non-physical entities are introduced. The most consequential assumption is that the pseudo-SQL-based schema linking in Eq. (2) does not omit relevant database entities, since schema linking is the dominant source of the reported gains.

free parameters (6)
  • Number of synthetic auxiliary samples per task = 10,000 each
    Appendix A.10 states 10,000 data pairs are constructed for each auxiliary task; chosen by hand without sensitivity analysis.
  • Hardness threshold for continuation writing = >2 tables
    Section 3.3 defines h(si,di) in {1,2,3} and applies CW only to Hard queries involving more than two tables.
  • Noise filtering discriminator criterion = unspecified
    Section 3.2 fine-tunes a Llama3-8B discriminator and filters the dataset, but no threshold or probability for declaring a pair 'noisy' is provided.
  • Artificial error type probabilities = certain probability, unspecified
    Appendix A.7 says errors are introduced according to 'a certain probability' without quantifying the distribution over the five error types.
  • Inference-time truncation position for CW = not specified
    Algorithm 1 line 9 says 'construct the truncated SQL query based on s*' but does not define the truncation rule, unlike training which uses random positions on ground-truth SQL.
  • Schema merging operation in Eq. 2 = not specified
    The operation ⊎ is described only as merging tables and columns through fuzzy matching; no concrete algorithm or threshold is given.
assumptions (6)
  • domain assumption Supervised fine-tuning on the synthesized multitask data transfers to held-out SPIDER and BIRD dev sets
    The whole MSFT stage relies on this generalization, tested empirically but not guaranteed for new databases.
  • domain assumption Execution-result matching is a valid correctness metric for Text2SQL
    EX is used both for evaluation and for constructing noise-correction training labels; it can miss semantically equivalent SQL with different execution results.
  • domain assumption The remaining training pairs after noisy correspondence filtering are clean and correct
    The filtering pipeline assumes that pairs not flagged by the discriminator are trustworthy, even though the discriminator was trained on synthetic labels.
  • domain assumption The pseudo-SQL generated on the full schema is accurate enough for entity extraction
    Equation (2) builds the simplified database from the parsed pseudo-SQL; if the pseudo-SQL omits entities, the simplified schema is incomplete.
  • domain assumption Continuation writing from truncated SQL is easier for LLMs and training on ground-truth truncations transfers to model-generated SQL
    Section 3.3 justifies CW by this assumed ease; Table 4 shows only a small realized gain, so the transfer is weakly supported.
  • domain assumption Base LLMs have not memorized SPIDER and BIRD dev answers during pretraining
    If dev answers leaked into pretraining, reported gains would be inflated; this applies equally to all baselines but is not verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL." pith.science (2026). https://pith.science/paper/XTXHSKF2

@misc{pith2026241210138,
  author       = {Pith},
  title        = {Pith review of: ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XTXHSKF2}},
  note         = {Machine review of arXiv:2412.10138}
}
read the original abstract

Despite the significant advancements in Text-to-SQL (Text2SQL) facilitated by large language models (LLMs), the latest state-of-the-art techniques are still trapped in the in-context learning of closed-source LLMs (e.g., GPT-4), which limits their applicability in open scenarios. To address this challenge, we propose a novel RObust mUltitask Tuning and collaboration mEthod (ROUTE) to improve the comprehensive capabilities of open-source LLMs for Text2SQL, thereby providing a more practical solution. Our approach begins with multi-task supervised fine-tuning (SFT) using various synthetic training data related to SQL generation. Unlike existing SFT-based Text2SQL methods, we introduced several additional SFT tasks, including schema linking, noise correction, and continuation writing. Engaging in a variety of SQL generation tasks enhances the model's understanding of SQL syntax and improves its ability to generate high-quality SQL queries. Additionally, inspired by the collaborative modes of LLM agents, we introduce a Multitask Collaboration Prompting (MCP) strategy. This strategy leverages collaboration across several SQL-related tasks to reduce hallucinations during SQL generation, thereby maximizing the potential of enhancing Text2SQL performance through explicit multitask capabilities. Extensive experiments and in-depth analyses have been performed on eight open-source LLMs and five widely-used benchmarks. The results demonstrate that our proposal outperforms the latest Text2SQL methods and yields leading performance.

Figures

Figures reproduced from arXiv: 2412.10138 by the authors.

Figure 1
Figure 1. Overall framework of our ROUTE. Our approach consists of two core stages, i.e., Multi￾task Supervised Fine-tuning (MSFT) and Multitask Collaboration Prompting (MCP). Different from existing methods that only focus on unitask learning in a single LLM, our MSFT aims to empower LLMs to handle multiple SQL-specific tasks by utilizing synthetic data for supervised fine-tuning. MCP mainly leverages the capabilities of ind… view at source ↗
Figure 2
Figure 2. Illustration of all SQL-related tasks involved in our R [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The examples of noisy pairs in the BIRD training set. R1 and R2 are the corrected SQL queries. More noisy examples can be found in Ap￾pendix A.12. To obtain negative SQL examples for the construction of Aneg, we employ open-source LLMs Qwen2-7B and Llama3-8B to generate all SQL responses for the questions in D in a zero-shot manner and then compare the execu￾tion results of each predicted SQL query with those of the… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The transferability results on different open-source LLMs on SPIDER (the first row) and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The prompt of Text-to-SQL. Your task is to determine whether the execution results of a SQL query can answer the given question according to the following database schema. If the execution results cannot correctly answer the question, please give me the correct SQL que…
Figure 6
Figure 6. Figure 6: The prompt template of noise correction. [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: The prompt template of schema linking. Given the following database schema and question, your task is to write an incomplete SQL query into a complete SQL query whose execution results can correctly answer the question. /* Examples */ ...few-shot examples omitted... /*…
Figure 8
Figure 8. Figure 8: The prompt template of continuation writing. [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: The prompt of STF data for noisy correspondence filtering. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 12 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing

    Ben Bogin, Matt Gardner, and Jonathan Berant. Representing schema structure with graph neural networks for text-to-sql parsing. arXiv preprint arXiv:1905.06241, 2019

    work page Pith review arXiv 1905

  3. [3]

    Shuaichen Chang, Jun Wang, Mingwen Dong, Lin Pan, Henghui Zhu, Alexander Hanbo Li, Wuwei Lan, Sheng Zhang, Jiarong Jiang, Joseph Lilien, et al. Dr. spider: A diagnostic evaluation benchmark towards text-to-sql robustness. arXiv preprint arXiv:2301.08881, 2023

    arXiv 2023

  4. [4]

    Teaching large language models to self-debug

    Xinyun Chen, Maxwell Lin, Nathanael Sch \"a rli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128, 2023

    arXiv 2023

  5. [5]

    Structure-grounded pretraining for text-to-sql

    Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, and Matthew Richardson. Structure-grounded pretraining for text-to-sql. arXiv preprint arXiv:2010.12773, 2020

    arXiv 2010

  6. [6]

    C3: Zero-shot text-to-sql with chatgpt

    Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, Jinshu Lin, Dongfang Lou, et al. C3: Zero-shot text-to-sql with chatgpt. arXiv preprint arXiv:2307.07306, 2023

    arXiv 2023

  7. [7]

    Miga: a unified multi-task generation framework for conversational text-to-sql

    Yingwen Fu, Wenjie Ou, Zhou Yu, and Yue Lin. Miga: a unified multi-task generation framework for conversational text-to-sql. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 12790--12798, 2023

    work page 2023

  8. [8]

    Towards robustness of text-to-sql models against synonym substitution

    Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R Woodward, Jinxia Xie, and Pengsheng Huang. Towards robustness of text-to-sql models against synonym substitution. arXiv preprint arXiv:2106.01065, 2021 a

    arXiv 2021

Show all 50 references
  1. [9]

    Exploring underexplored limitations of cross-domain text-to-sql generalization

    Yujian Gan, Xinyun Chen, and Matthew Purver. Exploring underexplored limitations of cross-domain text-to-sql generalization. arXiv preprint arXiv:2109.05157, 2021 b

    2021 arXiv

  2. [10]

    Text-to-sql empowered by large language models: A benchmark evaluation

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. Text-to-sql empowered by large language models: A benchmark evaluation. Proceedings of the VLDB Endowment, 17 0 (5): 0 1132--1145, 2024 a

    2024

  3. [11]

    Xiyan-sql: A multi-generator ensemble framework for text-to-sql

    Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, Jinyang Gao, Liyu Mou, and Yu Li. Xiyan-sql: A multi-generator ensemble framework for text-to-sql. arXiv preprint arXiv:2411.08599, 2024 b . URL https://arxiv.org/...

    2024 arXiv

  4. [12]

    Interleaving pre-trained language models and large language models for zero-shot nl2sql generation

    Zihui Gu, Ju Fan, Nan Tang, Songyue Zhang, Yuxin Zhang, Zui Chen, Lei Cao, Guoliang Li, Sam Madden, and Xiaoyong Du. Interleaving pre-trained language models and large language models for zero-shot nl2sql generation. arXiv preprint arXiv:2306.08891, 2023

    2023 arXiv

  5. [13]

    Deepseek-coder: When the large language model meets programming--the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024

    2024 arXiv

  6. [14]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  7. [15]

    A deep dive into deep learning approaches for text-to-sql systems

    George Katsogiannis-Meimarakis and Georgia Koutrika. A deep dive into deep learning approaches for text-to-sql systems. In Proceedings of the 2021 International Conference on Management of Data, pp.\ 2846--2851, 2021

    2021

  8. [16]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems...

    2023

  9. [17]

    Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation

    Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation. arXiv preprint arXiv:2405.07467, 2024

    2024 arXiv

  10. [18]

    The dawn of natural language to sql: Are we fully ready? arXiv preprint arXiv:2406.01265, 2024 a

    Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. The dawn of natural language to sql: Are we fully ready? arXiv preprint arXiv:2406.01265, 2024 a

    2024 arXiv

  11. [19]

    Resdsql: Decoupling schema linking and skeleton parsing for text-to-sql

    Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. Resdsql: Decoupling schema linking and skeleton parsing for text-to-sql. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 13067--13075, 2023 a

    2023

  12. [20]

    Codes: Towards building open-source language models for text-to-sql

    Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. Codes: Towards building open-source language models for text-to-sql. Proceedings of the ACM on Management of Data, 2 0 (3): 0 1--28, 2024 b

    2024

  13. [21]

    Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing

    Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei Huang, Wenyu Du, Luo Si, and Yongbin Li. Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing. In Proceedings of the AAAI Conference on Artificial Intelligence, v...

    2023

  14. [22]

    Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems, ...

    2024

  15. [23]

    Starcoder: may the source be with you! Transactions on Machine Learning Research

    Raymond Li, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, LI Jia, Jenny Chim, Qian Liu, et al. Starcoder: may the source be with you! Transactions on Machine Learning Research

  16. [24]

    Pet-sql: A prompt-enhanced two-stage text-to-sql framework with cross-consistency

    Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye, Ziyue Li, Rui Zhao, et al. Pet-sql: A prompt-enhanced two-stage text-to-sql framework with cross-consistency. arXiv preprint arXiv:2403.09732, 2024 d

    2024 arXiv

  17. [25]

    A comprehensive evaluation of chatgpt's zero-shot text-to-sql capability

    Aiwei Liu, Xuming Hu, Lijie Wen, and Philip S Yu. A comprehensive evaluation of chatgpt's zero-shot text-to-sql capability. arXiv preprint arXiv:2303.13547, 2023

    2023 arXiv

  18. [26]

    The death of schema linking? text-to-sql in the age of well-reasoned language models

    Karime Maamari, Fadhil Abubaker, Daniel Jaroslawicz, and Amine Mhedhbi. The death of schema linking? text-to-sql in the age of well-reasoned language models. arXiv preprint arXiv:2408.07702, 2024

    2024 arXiv

  19. [27]

    Din-sql: Decomposed in-context learning of text-to-sql with self-correction

    Mohammadreza Pourreza and Davood Rafiei. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  20. [28]

    Dts-sql: Decomposed text-to-sql with small large language models

    Mohammadreza Pourreza and Davood Rafiei. Dts-sql: Decomposed text-to-sql with small large language models. arXiv preprint arXiv:2402.01117, 2024 b

    2024 arXiv

  21. [29]

    Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql

    Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O Arik. Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql. arXiv preprint arXiv:2410.01943, 2024

    arXiv

  22. [30]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  23. [31]

    Evaluating the text-to-sql capabilities of large language models

    Nitarshan Rajkumar, Raymond Li, and Dzmitry Bahdanau. Evaluating the text-to-sql capabilities of large language models. arXiv preprint arXiv:2204.00498, 2022

    2022 arXiv

  24. [32]

    Code llama: Open foundation models for code

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  25. [33]

    Exploring chain-of-thought style prompting for text-to-sql

    Chang-You Tai, Ziru Chen, Tianshu Zhang, Xiang Deng, and Huan Sun. Exploring chain-of-thought style prompting for text-to-sql. arXiv preprint arXiv:2305.14215, 2023

    2023 arXiv

  26. [34]

    Chess: Contextual harnessing for efficient sql synthesis

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755, 2024

    2024 arXiv

  27. [35]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  28. [36]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  29. [37]

    Mac-sql: Multi-agent collaboration for text-to-sql

    Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Qian-Wen Zhang, Zhao Yan, and Zhoujun Li. Mac-sql: Multi-agent collaboration for text-to-sql. arXiv preprint arXiv:2312.11242, 2023

    2023 arXiv

  30. [38]

    Bridging language & data: Optimizing text-to-sql generation in large language models, 2024

    Niklas Wretblad and Fredrik Gordh Riseby. Bridging language & data: Optimizing text-to-sql generation in large language models, 2024

    2024

  31. [39]

    Understanding the effects of noise in text-to-sql: An examination of the bird-bench benchmark

    Niklas Wretblad, Fredrik Gordh Riseby, Rahul Biswas, Amin Ahmadi, and Oskar Holmstr \"o m. Understanding the effects of noise in text-to-sql: An examination of the bird-bench benchmark. arXiv preprint arXiv:2402.12243, 2024

    2024 arXiv

  32. [40]

    Sequence-based structured prediction for semantic parsing

    Chunyang Xiao, Marc Dymetman, and Claire Gardent. Sequence-based structured prediction for semantic parsing. In Annual meeting of the Association for Computational Linguistics (ACL), pp.\ 1341--1350, 2016

    2016

  33. [41]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024 a

    2024 arXiv

  34. [42]

    Synthesizing text-to-sql data from weak and strong llms

    Jiaxi Yang, Binyuan Hui, Min Yang, Jian Yang, Junyang Lin, and Chang Zhou. Synthesizing text-to-sql data from weak and strong llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 7864--7875, 2024 b

    2024

  35. [43]

    Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887, 2018

    2018 arXiv

  36. [44]

    Act-sql: In-context learning for text-to-sql with automatically-generated chain-of-thought

    Hanchong Zhang, Ruisheng Cao, Lu Chen, Hongshen Xu, and Kai Yu. Act-sql: In-context learning for text-to-sql with automatically-generated chain-of-thought. In The 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  37. [45]

    A survey of table reasoning with large language models

    Xuanliang Zhang, Dingzirui Wang, Longxu Dou, Qingfu Zhu, and Wanxiang Che. A survey of table reasoning with large language models. arXiv preprint arXiv:2402.08259, 2024

    2024 arXiv

  38. [46]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024 arXiv

  39. [47]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  40. [48]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  41. [49]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  42. [50]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.