Pith. sign in

REVIEW 4 major objections 6 minor 40 references

SEED: Enhancing Text-to-SQL Performance and Practical Usability Through Automatic Evidence Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SEED automatically generates evidence from database schema, description files, and sampled values, improving text-to-SQL accuracy when no human evidence is available and sometimes beating the human-evidence baseline.

desk verdict SEED is a solid, honest incremental advance in automatic evidence generation for text-to-SQL, but the 'automatic' claim is softer than the title suggests because the few-shot examples embed human BIRD evidence. read the letter →

arxiv 2506.07423 v1 pith:3C3ZQR3B submitted 2025-06-09 cs.CL

Janghyeon Yun , Sang-goo Lee This is my paper
classification cs.CL
keywords text-to-SQLevidencegenerationexecutionaccuracyBIRDbenchmarkSpiderdomainknowledgeschemasummarizationLLMprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that text-to-SQL systems have come to depend on human-written evidence supplied with the BIRD benchmark, and that this dependency makes them unusable where no such evidence exists. It reports that removing evidence drops execution accuracy by 8.35 to 20.86 points across recent models, and that roughly 16 percent of BIRD development-set evidence is missing or wrong. To address this, SEED automatically constructs evidence from the database schema, description files, and sampled values, and the authors show that models prompted with SEED evidence beat the no-evidence baseline in nearly all settings, sometimes even outperforming human evidence. A sympathetic reader would take the central claim to be that most evidence knowledge is actually recoverable from the database itself, and that automating this extraction is a practical path toward real-world text-to-SQL.

What carries the argument

SEED is a three-stage evidence-generation pipeline: optionally summarize the schema when context is tight; extract question keywords, pair them with candidate columns, and execute sample SQL queries (including LIKE and edit-distance variants for strings) to surface actual database values; then prompt a strong LLM to write evidence, with an instruction, a few training examples chosen by cosine similarity over question embeddings, the sample SQL results, the schema, and the question. The full schema is deliberately kept rather than pruned, following evidence that schema linking can hurt strong LLMs. What carries the argument is the constructed evidence text itself, since every downstream gain is obtained by swapping that text into the prompt in place of human evidence.

What would settle it

Remove the human-written BIRD evidence from SEED's few-shot examples and run it on a database that has never been seen; if execution accuracy reverts to the no-evidence baseline, then the claimed automatic evidence generation still depends on human annotation.

Watch

Extended reading notes

Core claim

The central discovery is that the information in BIRD's evidence is not external knowledge that only humans can supply: three of the four evidence categories (domain knowledge, synonym knowledge, and value illustration) can be reconstructed by inspecting the schema, reading description files, and executing sample SQL queries against stored values. SEED operationalizes this in two variants, one for long-context LLMs that reads the full schema and one for token-limited LLMs that first summarizes the schema. On the BIRD development set, SEED raises execution accuracy in the no-evidence setting for every model configuration tested except one, with gains up to 17.73 EX points, and it beats the human-evidence setting in several cases: for example, SFT CodeS-15B reaches 56.78 with SEED_gpt and 57.69 with SEED_deepseek, compared with 55.35 using BIRD evidence. The paper also shows that evidence format matters: when SEED evidence is edited to remove join hints, a model whose prompts are tuned for BIRD-format evidence improves, while a model that simply concatenates evidence gets worse.

Load-bearing premise

The load-bearing premise is that a training set with human-written question-evidence pairs is available to serve as few-shot examples; on a brand-new database with no such annotated evidence, SEED's automatic evidence generation has not been shown to work.

Editorial extensions

If this is right

  • If SEED's evidence is used, text-to-SQL deployments on databases with schema descriptions and sample data no longer need human-written evidence to stay accurate.
  • The measured gains mean that, on BIRD dev, models that would otherwise lose 8.35 to 20.86 execution-accuracy points without evidence recover most of the gap, and several configurations outperform the human-evidence condition.
  • SEED's gains on Spider show the benefit is not an artifact of one benchmark's evidence style.
  • The evidence-format experiment indicates that downstream systems should be trained or prompted to consume automatically generated evidence; the paper explicitly calls for research on matching evidence format to how models consume it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the automatic-generation claim is untested in a true cold start, because SEED's evidence-generation prompt uses human-written BIRD evidence from similar training questions as few-shot examples.
  • Editorial inference: the format-sensitivity result generalizes into a design rule: auxiliary context must match the prompt distribution a model was trained on, which predicts that simply changing evidence format can flip which model benefits.
  • Editorial inference: SEED could be extended into a closed loop, using execution feedback from generated SQL to filter or refine the evidence it produces, a testable extension beyond what the paper reports.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SEED, a three-stage pipeline (schema summarization, sample SQL execution, and evidence generation) that automatically produces textual evidence for text-to-SQL systems by analyzing database schemas, description files, and sampled values. The method is evaluated on BIRD and Spider with several models (CHESS, RSL-SQL, CodeS, DAIL-SQL, C3) under three conditions: with BIRD evidence, without evidence, and with SEED-generated evidence. The main empirical claim is that SEED improves execution accuracy and valid efficiency scores relative to the no-evidence baseline, and in some configurations it even surpasses the human-evidence setting. The paper additionally documents flaws in BIRD's human-generated evidence and shows that correcting these errors improves CodeS performance.

Significance. The central idea is practically relevant: removing the reliance on manually curated evidence would make text-to-SQL models more deployable in real-world settings. The paper provides a broad evaluation across multiple models and datasets, and it releases code. The analysis of BIRD evidence defects is useful and the proposed pipeline is a sensible combination of schema analysis, value sampling, and LLM-based generation. However, the significance is limited by the fact that the 'automatic' claim is not fully demonstrated for cold-start deployments, and several comparisons lack statistical grounding.

major comments (4)
  1. [Section III-C and Section IV-E2] The evidence generation prompt uses human-written BIRD evidence as few-shot demonstrations retrieved from the training set, as explicitly acknowledged in Section IV-E2 ('SEED utilized human-generated BIRD evidence as few-shot examples'). As a result, the BIRD dev evaluation measures a setup that still injects human annotations indirectly through the few-shot examples. On a new database for which no such annotated question-evidence pairs exist, the retriever cannot supply these demonstrations, so the claim of human-intervention-free evidence generation is not demonstrated. Please add a cold-start experiment (for example, using few-shot examples constructed from schema/value information only, or evaluating on databases outside the training set) or substantially soften the 'automatic' and 'without human intervention' claims in the abstract and conclusions.
  2. [Tables IV, V, and VII] All results are reported as single-point estimates with no variance, confidence intervals, or significance tests. Some of the headline differences are small (CHESS IR+CG+UT +1.57 EX with SEED gpt on BIRD dev; SFT CodeS-7B +0.4 EX on Spider dev), and the word 'significantly' in the abstract is therefore not supported for these cases. Reporting multiple runs with means and standard deviations, or at least a paired bootstrap across the dev set, would allow the reader to assess whether the improvements are reliable.
  3. [Section IV-E3] The Spider experiment uses description files generated by DeepSeek-V3 rather than real human-authored description files. This introduces an unvalidated assumption that LLM-generated descriptions are equivalent to real ones for the purpose of evidence generation. Since Spider is the primary cross-dataset evidence, the robustness claim would be stronger if the generated descriptions were checked for fidelity (for example, by comparing a sample to manually written descriptions) or if an ablation removed the description-file input entirely.
  4. [Table IV and Section IV-E2] The CHESS IR+CG+UT configuration degrades by -0.58 EX and -0.58 VES when SEED deepseek evidence is used instead of no evidence. The authors propose a revised evidence format that recovers only +0.79 EX over the no-evidence baseline, still well short of the +8.35 EX gain from human evidence. This is a clear boundary on the generality of SEED and should be acknowledged more prominently in the abstract and contribution claims, where the paper currently reads as though SEED is uniformly beneficial.
minor comments (6)
  1. [Table VII] The caption of Table VII says 'Performance improvement ... on the Spider dataset,' but the reported numbers (e.g., CHESSIR+CG+UT 54.69) match the BIRD dev results in Table IV; the caption should be corrected to say BIRD.
  2. [Table IV] The row labels 'CHESSIR+CG+UT' and 'CHESSIR+SS+CG' are missing spaces; they should be formatted as 'CHESS IR+CG+UT' and 'CHESS IR+SS+CG' for readability.
  3. [Section III-B] The keyword extraction step ('SEED extracts keywords that represent database columns and values') is underspecified; providing a concrete algorithm, an example, or a reference would improve reproducibility.
  4. [Section III-B] The edit distance threshold and the number of similar values selected by the LIKE/edit-distance procedure are not reported anywhere; please include these hyperparameters in the implementation details or in the released code documentation.
  5. [Section I (Table II)] The manual correction process for the 105 erroneous BIRD evidence pairs is not described; please state who performed the correction, what guidelines were used, and whether the corrected evidence will be released along with the code.
  6. [Section I] The statement 'among the top 30 entries on the BIRD leaderboard, all but one unpublished study use evidence' would benefit from a version and date of the leaderboard, so that it is verifiable in the future.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SEED's evidence is generated from schema/values, not fitted to target SQLs; the few-shot human-evidence caveat affects deployment claims, not derivation equivalence.

full rationale

The paper's central comparison is SEED-generated evidence versus no evidence on the BIRD dev set. The generated evidence is produced from schema, description files, sampled values, and question-similarity-retrieved training examples; the target dev SQLs are never used to construct evidence, so the reported accuracy gains (e.g., CodeS-15B from 44.39 to 56.78 EX) are not forced by construction, and no equation-level reduction exists. The only in-scope caveat is Section IV-E2, which states "SEED utilized human-generated BIRD evidence as few-shot examples" (and Section III-C retrieves similar training questions); this weakens the "automatic / without human intervention" deployment claim for new databases, but it is an external-validity limitation rather than a circular equivalence, because the human examples are not the target evidence and the measured SQL predictions are not used to fit or select evidence. No self-citation is load-bearing, no prior-work uniqueness theorem is imported, and no known result is merely renamed. Accordingly, there is no significant circularity in the derivation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or mathematical entities are introduced. The system is a pipeline over existing components. The main assumptions are about the sufficiency of database-derived information, the suitability of the BIRD evidence format, and the validity of synthetic descriptions for Spider.

free parameters (2)
  • Number of few-shot evidence examples = 5 (question plus 4 related)
    Hand-chosen in Section III-C; no ablation tests sensitivity to this number, so the claim is assumed robust to it.
  • Edit distance threshold or LIKE-based value selection = unspecified
    Section III-B says similar values are extracted using LIKE and edit distance, but no threshold is reported, leaving a hidden free choice that may affect evidence content.
assumptions (4)
  • domain assumption BIRD's human-written evidence format is the appropriate target for generated evidence.
    The method generates evidence and evaluates by feeding it to models trained or tuned on BIRD evidence; Section IV-E2 shows that format changes flip which model benefits.
  • domain assumption Database description files and sampled values contain enough information to reconstruct most of BIRD's evidence.
    Section II-A argues that domain, synonym, and value illustration evidence can be inferred from the database itself, which is the core premise of the system.
  • ad hoc to paper LLM-generated description files for Spider behave like real description files.
    Section IV-E3 uses DeepSeek-V3 to fabricate Spider descriptions; no validation shows these match real metadata, so the Spider result may overstate generalizability.
  • domain assumption Single-run execution accuracy on BIRD dev is a stable estimate of performance.
    Tables IV, V, and VII report one EX or VES number per condition with no variance or significance testing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SEED: Enhancing Text-to-SQL Performance and Practical Usability Through Automatic Evidence Generation." pith.science (2026). https://pith.science/paper/3C3ZQR3B

@misc{pith2026250607423,
  author       = {Pith},
  title        = {Pith review of: SEED: Enhancing Text-to-SQL Performance and Practical Usability Through Automatic Evidence Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3C3ZQR3B}},
  note         = {Machine review of arXiv:2506.07423}
}
read the original abstract

Text-to-SQL enables non-experts to retrieve data from databases by converting natural language queries into SQL. However, state-of-the-art text-to-SQL studies rely on the BIRD dataset, which assumes that evidence is provided along with questions. Although BIRD facilitates research advancements, it assumes that users have expertise and domain knowledge, contradicting the fundamental goal of text-to-SQL. In addition, human-generated evidence in BIRD contains defects, including missing or erroneous evidence, which affects model performance. To address this issue, we propose SEED (System for Evidence Extraction and Domain knowledge generation), an approach that automatically generates evidence to improve performance and practical usability in real-world scenarios. SEED systematically analyzes database schema, description files, and values to extract relevant information. We evaluated SEED on BIRD and Spider, demonstrating that it significantly improves SQL generation accuracy in the no-evidence scenario, and in some cases, even outperforms the setting where BIRD evidence is provided. Our results highlight that SEED-generated evidence not only bridges the gap between research and real-world deployment but also improves the adaptability and robustness of text-to-SQL models. Our code is available at https://github.com/felix01189/SEED

Figures

Figures reproduced from arXiv: 2506.07423 by the authors.

Figure 1
Figure 1. Diagram of assumptions for the text-to-SQL problems. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. BIRD development set evidence error rate(left) and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The structure of SEED [23], which employs BERT-based encoding combined with a Pointer-Generator Network [24] for decoding SQL queries. Recently, with the rise of closed-source LLMs such as GPT-4 [25] and Gemini [26], and open-source LLMs like LLaMA [27] and StarCoder [28], most state-of-the-art text-to￾SQL methods have adopted LLM-based approaches. C3 [29] is a zero-shot text-to-SQL method built on ChatGPT. DIN￾SQL … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 12 canonical work pages

  1. [1]

    A survey on text-to-sql parsing: Concepts, methods, and future directions,

    B. Qin, B. Hui, L. Wang, M. Yang, J. Li, B. Li, R. Geng, R. Cao, J. Sun, L. Si, F. Huang, and Y . Li, “A survey on text-to-sql parsing: Concepts, methods, and future directions,” 2022. [Online]. Available: https://arxiv.org/abs/2208.13629

    arXiv 2022

  2. [2]

    A survey on employing large language models for text-to-sql tasks,

    L. Shi, Z. Tang, N. Zhang, X. Zhang, and Z. Yang, “A survey on employing large language models for text-to-sql tasks,” 2024. [Online]. Available: https://arxiv.org/abs/2407.15186

    arXiv 2024

  3. [3]

    Next-generation database interfaces: A survey of llm-based text-to-sql,

    Z. Hong, Z. Yuan, Q. Zhang, H. Chen, J. Dong, F. Huang, and X. Huang, “Next-generation database interfaces: A survey of llm-based text-to-sql,” 2024. [Online]. Available: https://arxiv.org/abs/2406.08426

    arXiv 2024

  4. [4]

    Seq2sql: Generating structured queries from natural language using reinforcement learning,

    V . Zhong, C. Xiong, and R. Socher, “Seq2sql: Generating structured queries from natural language using reinforcement learning,” 2017. [Online]. Available: https://arxiv.org/abs/1709.00103

    arXiv 2017

  5. [5]

    Spider: A large- scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task,

    T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman, Z. Zhang, and D. R. Radev, “Spider: A large- scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task,” in2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018). Google; Facebook; Bloomberg; Salesforce; App...

    work page 2018

  6. [6]

    Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls,

    J. Li, B. Hui, G. Qu, J. Yang, B. Li, B. Li, B. Wang, B. Qin, R. Geng, N. Huo, X. Zhou, M. Chenhao, G. Li, K. Chang, F. Huang, R. Cheng, and Y . Li, “Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls,” inAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M....

    work page 2023

  7. [7]

    Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql,

    M. Pourreza, H. Li, R. Sun, Y . Chung, S. Talaei, G. T. Kakkar, Y . Gan, A. Saberi, F. Ozcan, and S. O. Arik, “Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql,”

    work page

  8. [8]

    Chess: Contextual harnessing for efficient sql synthesis,

    S. Talaei, M. Pourreza, Y .-C. Chang, A. Mirhoseini, and A. Saberi, “Chess: Contextual harnessing for efficient sql synthesis,” 2024. [Online]. Available: https://arxiv.org/abs/2405.16755

    arXiv 2024

Show all 40 references
  1. [9]

    Rsl-sql: Robust schema linking in text-to-sql generation,

    Z. Cao, Y . Zheng, Z. Fan, X. Zhang, W. Chen, and X. Bai, “Rsl-sql: Robust schema linking in text-to-sql generation,” 2024. [Online]. Available: https://arxiv.org/abs/2411.00073

    2024 arXiv

  2. [10]

    Xiyan-sql: A multi-generator ensemble framework for text-to-sql,

    Y . Gao, Y . Liu, X. Li, X. Shi, Y . Zhu, Y . Wang, S. Li, W. Li, Y . Hong, Z. Luo, J. Gao, L. Mou, and Y . Li, “Xiyan-sql: A multi-generator ensemble framework for text-to-sql,” 2024. [Online]. Available: https://arxiv.org/abs/2411.08599

    2024 arXiv

  3. [11]

    The death of schema linking? text-to-sql in the age of well-reasoned language models,

    K. Maamari, F. Abubaker, D. Jaroslawicz, and A. Mhedhbi, “The death of schema linking? text-to-sql in the age of well-reasoned language models,” 2024. [Online]. Available: https://arxiv.org/abs/2408.07702

    2024 arXiv

  4. [12]

    Purple: Making a large language model a better sql writer,

    T. Ren, Y . Fan, Z. He, R. Huang, J. Dai, C. Huang, Y . Jing, K. Zhang, Y . Yang, and X. S. Wang, “Purple: Making a large language model a better sql writer,” in2024 IEEE 40th International Conference on Data Engineering (ICDE), 2024, pp. 15–28

    2024

  5. [13]

    E-sql: Direct schema linking via question enrichment in text-to-sql,

    H. A. Cafero ˘glu and ¨Ozg¨ur Ulusoy, “E-sql: Direct schema linking via question enrichment in text-to-sql,” 2024. [Online]. Available: https://arxiv.org/abs/2409.16751

    2024 arXiv

  6. [14]

    Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation,

    D. Lee, C. Park, J. Kim, and H. Park, “Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation,” 2024. [Online]. Available: https://arxiv.org/abs/2405.07467

    2024 arXiv

  7. [15]

    Codes: Towards building open-source language models for text-to-sql,

    H. Li, J. Zhang, H. Liu, J. Fan, X. Zhang, J. Zhu, R. Wei, H. Pan, C. Li, and H. Chen, “Codes: Towards building open-source language models for text-to-sql,”Proc. ACM Manag. Data, vol. 2, no. 3, May

  8. [16]

    Synthesizing text-to-SQL data from weak and strong LLMs,

    J. Yang, B. Hui, M. Yang, J. Yang, J. Lin, and C. Zhou, “Synthesizing text-to-SQL data from weak and strong LLMs,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L.-W. Ku, A. Martins, and V . Srikumar, Eds. B...

    2024

  9. [17]

    Available: https://doi.org/10.1145/3654930

    [Online]. Available: https://doi.org/10.1145/3654930

    doi

  10. [18]

    Nalir: an interactive natural language interface for querying relational databases,

    F. Li and H. V . Jagadish, “Nalir: an interactive natural language interface for querying relational databases,” inProceedings of the 2014 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD ’14. New York, NY , USA: Association for Computing Machinery, 2014,...

    2014

  11. [19]

    The dawn of natural language to sql: Are we fully ready?

    B. Li, Y . Luo, C. Chai, G. Li, and N. Tang, “The dawn of natural language to sql: Are we fully ready?”Proc. VLDB Endow., vol. 17, no. 11, p. 3318–3331, Aug. 2024. [Online]. Available: https://doi.org/10.14778/3681954.3682003

    2024

  12. [20]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”Advances in Neural Information Processing Systems, 2017

    2017

  13. [21]

    Sequence to sequence learning with neural networks,

    I. Sutskever, “Sequence to sequence learning with neural networks,” arXiv preprint arXiv:1409.3215, 2014

    2014 arXiv

  14. [22]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    Raffel, Colin, Shazeer, Noam, Roberts, and e. a. Adam, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  15. [23]

    Bert: Pre- training of deep bidirectional transformers for language understanding,

    Kenton, J. Devlin, M.-W. Chang, Toutanova, and L. Kristina, “Bert: Pre- training of deep bidirectional transformers for language understanding,” inProceedings of naacL-HLT, vol. 1, no. 2. Minneapolis, Minnesota, 2019

    2019

  16. [24]

    Get to the point: Summarization with pointer-generator networks,

    A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with pointer-generator networks,” inProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), R. Barzilay and M.-Y . Kan, Eds. Vancouver, Canada: Assoc...

    2017

  17. [25]

    Bridging textual and tabular data for cross-domain text-to-SQL semantic parsing,

    X. V . Lin, R. Socher, and C. Xiong, “Bridging textual and tabular data for cross-domain text-to-SQL semantic parsing,” in Findings of the Association for Computational Linguistics: EMNLP 2020, T. Cohn, Y . He, and Y . Liu, Eds. Online: Association for Computational Linguistic...

    2020

  18. [26]

    Gemini: A family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, and e. a. Radu Soricut, “Gemini: A family of highly capable multimodal models,” 2024. [Online]. Available: https://arxiv.org/abs/2312.11805

    2024 arXiv

  19. [27]

    Gpt-4 technical report,

    OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, and e. a. Ilge Akkaya, “Gpt-4 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2303.08774

    2024 arXiv

  20. [28]

    Starcoder: may the source be with you!

    R. Li, L. B. Allal, Y . Zi, N. Muennighoff, D. Kocetkov, and e. a. Chenghao Mou, “Starcoder: may the source be with you!” 2023. [Online]. Available: https://arxiv.org/abs/2305.06161

    2023 arXiv

  21. [29]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, and e. a. Yasmine Babaei, “Llama 2: Open foundation and fine-tuned chat models,” 2023. [Online]. Available: https://arxiv.org/abs/2307.09288

    2023 arXiv

  22. [30]

    Din-sql: Decomposed in-context learning of text-to-sql with self-correction,

    M. Pourreza and D. Rafiei, “Din-sql: Decomposed in-context learning of text-to-sql with self-correction,” inADVANCES IN NEURAL INFORMA- TION PROCESSING SYSTEMS 36 (NEURIPS 2023), ser. Advances in Neural Information Processing Systems, A. Oh, T. Neumann, A. Glober- son, K. Saen...

    2023

  23. [31]

    C3: Zero-shot text-to-sql with chatgpt,

    X. Dong, C. Zhang, Y . Ge, Y . Mao, Y . Gao, lu Chen, J. Lin, and D. Lou, “C3: Zero-shot text-to-sql with chatgpt,” 2023. [Online]. Available: https://arxiv.org/abs/2307.07306

    2023 arXiv

  24. [32]

    Self-consistency improves chain of thought reasoning in language models,

    X. Wang, J. Wei, D. Schuurmans, Q. V . Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” inICLR 2023, 2023. [Online]. Available: https://arxiv.org/abs/2203.11171

    2023 arXiv

  25. [33]

    Text- to-sql empowered by large language models: A benchmark evaluation,

    D. Gao, H. Wang, Y . Li, X. Sun, Y . Qian, B. Ding, and J. Zhou, “Text- to-sql empowered by large language models: A benchmark evaluation,” PROCEEDINGS OF THE VLDB ENDOWMENT, vol. 17, no. 5, pp. 1132–1145, JAN 2024

    2024

  26. [34]

    Self-Polish: Enhance reasoning in large language models via problem refinement,

    Z. Xi, S. Jin, Y . Zhou, R. Zheng, S. Gao, J. Liu, T. Gui, Q. Zhang, and X. Huang, “Self-Polish: Enhance reasoning in large language models via problem refinement,” inFindings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali, Eds. ...

    2023

  27. [35]

    Msc-sql: Multi-sample critiquing small language models for text-to-sql translation,

    S. K. Gorti, I. Gofman, Z. Liu, J. Wu, N. V ouitsis, G. Yu, J. C. Cresswell, and R. Hosseinzadeh, “Msc-sql: Multi-sample critiquing small language models for text-to-sql translation,” 2024. [Online]. Available: https://arxiv.org/abs/2410.12916

    2024 arXiv

  28. [36]

    Mpnet: Masked and permuted pre-training for language understanding,

    K. Song, X. Tan, T. Qin, J. Lu, and T.-Y . Liu, “Mpnet: Masked and permuted pre-training for language understanding,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 202...

    2020

  29. [37]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    Guo, Daya, Yang, Dejian, Zhang, and e. a. Haowei, “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  30. [38]

    Deepseek-v3 technical report,

    DeepSeek-AI, A. Liu, B. Feng, B. Xue, B. Wang, and e. a. Bochao Wu, “Deepseek-v3 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2412.19437

    2024 arXiv

  31. [39]

    Resdsql: decoupling schema linking and skeleton parsing for text-to-sql,

    H. Li, J. Zhang, C. Li, and H. Chen, “Resdsql: decoupling schema linking and skeleton parsing for text-to-sql,” inProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty- Fifth Conference on Innovative Applications of Artificial Intelligence and...

    2023 doi

  32. [2024]

    Available: https://arxiv.org/abs/2410.01943

    [Online]. Available: https://arxiv.org/abs/2410.01943

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.