Pith. sign in

REVIEW 5 major objections 5 minor 50 references

CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B model trained on execution-reward RL beats SQL baselines up to 30 times larger.

desk verdict Solid RL recipe and useful released datasets, but the headline BIRD-dev number is a best-on-filtered-dev selection compared against inconsistent baselines. read the letter →

arxiv 2507.06013 v1 pith:WSLRWPK6 submitted 2025-07-08 cs.AI

Kushal Gajjar , Harshit Sikchi , Arpit Singh Gautam , Marc Hammons , Saurabh Jha This is my paper
classification cs.AI
keywords Text-to-SQLreinforcementlearningGRPOexecutionrewardBIRDbenchmarknaturallanguagetoSQLefficientfine-tuningreasoningtraces
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that CogniSQL-R1-Zero, a 7B-parameter model, reaches 59.97% execution accuracy on the BIRD development set — a standard benchmark for translating natural language questions into SQL — by training with Group Relative Policy Optimization (GRPO), a reinforcement learning rule that compares groups of candidate outputs, on a sparse reward that asks only whether the generated SQL executes to the same result as the ground truth. This matters because it suggests a low-compute recipe — structured prompts, execution feedback, and a small backbone on four A100s — can close the gap with models that are tens of times larger. If the claim holds, reinforcement learning aligned to execution correctness becomes a credible substitute for raw scale in Text-to-SQL. The paper also releases two datasets, 5,024 verified reasoning traces and a 36,356-example corpus of correct sampled SQL with reasoning paths, to support the same style of training on other models.

What carries the argument

Group Relative Policy Optimization (GRPO) is the training rule at the center: for each prompt the policy samples G candidate SQLs, each gets a reward $R_{total}=\alpha_f R_f + \alpha_{sf} R_{sf} + \alpha_c R_c + \alpha_l R_l$ (format, soft format, execution correctness, length) with correctness weighted highest, and the update uses the best candidate's reward minus a group-mean baseline through a clipped surrogate objective with a KL penalty to the reference policy. This removes the value network, stabilizes sparse-reward updates, and lets execution feedback propagate. The second load-bearing piece is the structured prompt — DDL with comments, external knowledge, the question, and enforced <reasoning>/<answer> tags — which reduces ambiguity so the sparse reward can shape behavior.

What would settle it

Run the released checkpoint on the full official BIRD development split with the paper's prompt template and execution harness, re-evaluating the larger baselines under the same conditions; the central claim fails if single-sample accuracy does not reproduce near 59.97% or if the baselines match or beat it in that controlled run.

Watch

Extended reading notes

Core claim

The paper's central claim is that a 7B-parameter model, CogniSQL-R1-Zero, reaches 59.97% execution accuracy on the BIRD development set by training with Group Relative Policy Optimization (GRPO) on a reward that is almost entirely execution correctness, with no supervised fine-tuning and no hybrid pipeline. Starting from Qwen2.5-Coder-7B-Instruct at 52.02%, the policy improves steadily over roughly 34,000 RL steps to 59.97% single-sample accuracy, and sampling six candidates and keeping the best brings the number to 69.68%. The authors report that this beats SFT CodeS-7B and larger backbones including DeepSeek-Coder 236B and Mistral 123B on the same benchmark, and they attribute the gain to structured prompts (DDL, external knowledge, format tags) that narrow the search space, plus a sparse reward that aligns training with the end task of producing executable SQL.

Load-bearing premise

The reported 59.97% is a fair apples-to-apples number: the 1,500-prompt subset follows the official BIRD dev protocol, baselines are measured with the same prompts and execution settings, and choosing the best checkpoint on the dev set does not inflate the result.

Editorial extensions

If this is right

  • A 7B model trained this way on four A100s can exceed the BIRD dev accuracy of models 3–30 times larger, so model scale is not the only path to competitive Text-to-SQL accuracy.
  • A sparse execution-only reward, with correctness weighted above format and length, is enough for stable RL convergence; complex reward shaping and intermediate supervision are not required.
  • Test-time scaling works: best-of-6 sampling with execution-based selection lifts accuracy from 59.97% to 69.68% with no further training, at roughly 6x inference cost.
  • Supervised fine-tuning on distilled reasoning traces can hurt (accuracy drops to about 46%), while RL on self-generated correct samples recovers and exceeds the base model — so the paper recommends RL, not SFT alone, for reasoning-style Text-to-SQL.
  • The two released datasets (5,024 verified reasoning traces and 36,356 positive-sampled SQL-plus-reasoning examples) are intended to let other base models be trained with the same alignment-driven RL recipe.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The same recipe — structured prompt, GRPO, and a binary executable-output reward — should transfer to other code-generation tasks with a runnable ground truth (e.g., data-frame transformations or query rewriting), though the paper only demonstrates it on BIRD.
  • Editorial: The best-of-six accuracy of 69.68% is an oracle-style upper bound because the best candidate is selected by execution against the ground truth; in deployment, a learned verifier or majority vote would be needed, and the paper's own majority-vote gain is only 1.5%.
  • Editorial: Because the model's checkpoints are selected by dev-set accuracy, the published single-sample number is an optimistic estimate of performance on a fresh dataset; a separate validation split would likely show a lower, more realistic number.
  • Editorial: The reported roughly 1.6-point gain from decaying the KL penalty invites a controlled study of KL schedules, but the paper does not isolate this effect from group size and temperature changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes CogniSQL-R1-Zero, a 7B-parameter text-to-SQL model trained with GRPO and a sparse execution-correctness reward, and claims 59.97% execution accuracy on the BIRD development set, outperforming much larger baselines such as DeepSeek-Coder 236B, Mistral 123B, and SFT CodeS-7B. The authors also report a best-of-6 test-time scaling result of 69.68%, release two auxiliary datasets, and document several training-strategy ablations.

Significance. If the headline result were established, the paper would make a useful empirical contribution: it would show that a small RL-trained backbone with execution feedback can match or beat much larger models on BIRD dev under low compute, and the released reasoning-trace and positive-sampling corpora could support follow-up work. However, the central number is not currently credible as a controlled comparison. The evaluation uses a filtered 1,500-example subset of BIRD dev, checkpoints are selected on that same subset, baseline numbers disagree between the text and the tables, and the best-of-6 result is an oracle selection. No model weights or evaluation code are released, so the claimed accuracies cannot be independently verified.

major comments (5)
  1. [§5.1, Tables 1–3] The headline comparison is not a valid head-to-head evaluation. Section 5.1 states that evaluation was performed on a 'held-out 1,500-sample dev set' after filtering prompts exceeding 3,000 tokens, whereas the official BIRD dev set contains 1,534 examples. The paper never lists which examples were removed or shows that the remaining subset has the same difficulty distribution. Tables 1–3 compare this filtered subset against official BIRD-dev numbers from prior work. Because the reported margin over SFT CodeS-7B is only 2.8 points, removing hard long-context queries or selecting a favorable subset could invert the headline result.
  2. [§4.6, §6.1] The reported 59.97% is a max-over-checkpoints statistic on the evaluation set, not an unbiased measurement of a fixed final policy. Section 4.6 says the authors evaluate execution accuracy on the BIRD dev set every 1,000 RL steps and select the best checkpoint from the pure-RL run as the final model. Using the same 1,500-example subset both for checkpoint selection and for the reported number inflates the result and makes comparison with single-checkpoint baseline numbers uninterpretable.
  3. [§5.5 vs. Tables 1–3] The baseline numbers are internally inconsistent. Section 5.5 reports SFT CodeS-7B at 50%, Mistral 123B at 52%, DeepSeek-Coder 236B at 54%, and GPT-4 at 55%, while Tables 1–3 list 57.17, 53.52, 56.13, and 46.35 for the same models. The paper does not explain which set of numbers is official or why the two sections disagree by up to 9 points. Without a consistent baseline protocol, the claim of state-of-the-art performance cannot be assessed.
  4. [§6.2, Table 4] The best-of-6 result of 69.68% is an oracle upper bound, not a deployable inference-time scaling method. Section 6.2 says the authors sample six SQL candidates and select 'the one with the highest execution success,' which requires ground-truth execution results to choose the candidate. The Discussion then presents the +9.71% gain as a practical inference-time benefit. The paper should either provide a selection rule that does not use the gold answer and report its accuracy, or explicitly label this number as an upper bound.
  5. [Abstract, §8] No model weights, training code, or evaluation scripts are released; the only provided artifacts are two datasets. Since the central claim is a specific trained model's execution accuracy, the 59.97% and 69.68% numbers cannot be independently reproduced or checked for protocol errors. At minimum, the final checkpoint and the exact dev-subset split should be released.
minor comments (5)
  1. [§4 (introduction)] The phrase 'Group Relativle Policy Optimization' contains a typo: 'Relativle' should be 'Relative.'
  2. [Abstract] The abstract says 'a positive-sampled corpus of 36,356 corpus of weakly supervised queries,' which duplicates 'corpus' and is grammatically incomplete.
  3. [§4.4, §5.4] The reward weights α_c, α_f, α_sf, and α_l are defined symbolically, but Section 5.4 gives only α_c=2, α_f=1, and α_l=-0.5; the value of α_sf is never specified.
  4. [§4.6 vs. §5.1] The gradient-accumulation configuration is inconsistent: Section 4.6 says microbatch size b=2 with accumulation over 4 steps, while Section 5.1 says b=2 with accumulation steps k=2 and an effective batch size of 16. Please clarify which setting was used.
  5. [§3.3 vs. §4.6] The number of QWQ 32B reasoning traces is inconsistent: Section 3.3 says 5,024 prompts produce 4,928 reasoning-SQL pairs, while Section 4.6 says the retained corpus contains 5,024 pairs.

Circularity Check

1 steps flagged · score 2.0 of 10

No circular derivation underlies the 59.97% central claim; the 69.68% best-of-6 figure is an acknowledged oracle upper bound and is definitional.

  1. self definitional [Section 6.2, Test-Time Scaling (Best-of-6)]
    "To further push accuracy at inference time, we apply test-time scaling [50] by sampling six SQL candidates and selecting the one with the highest execution success."

    The reported metric is execution accuracy, and 'execution success' is judged by the same ground-truth execution comparison that defines execution accuracy. Therefore the best-of-6 score is, by construction, the maximum of six executions of the very metric being reported, not an independent prediction of what a deployed system would achieve without access to the ground truth. The later claim that test-time scaling 'boosts accuracy' from 59.97% to 69.68% and the conclusion that 'applying test-time scaling (best-of-6 sampling) boosts accuracy to 69.68%' reduce to the definition of taking the max over the reward signal.

full rationale

The central empirical claim is not circular: CogniSQL-R1-Zero is trained with GRPO using an execution-correctness reward and then evaluated with execution accuracy on the BIRD dev set. Using the same objective for RL training and for evaluation is a legitimate experimental design, not a hidden fit; no fitted constant is renamed as a prediction. The two released corpora are filtered by execution correctness against ground truth, but they are training resources, not the claimed benchmark result. There is no load-bearing self-citation chain: the GRPO, R1-Zero, and related methods cited are external works, and the authors do not import a uniqueness theorem or ansatz from their own prior papers. The primary risk in the 59.97% headline is dev-set checkpoint selection and the non-identical 1,500-sample filtered dev subset, but those are evaluation-protocol and generalization concerns, not circular derivations. The one definitional element is the best-of-6 test-time scaling figure of 69.68%, where candidates are selected using the same ground-truth execution that defines the metric; this makes the result an oracle upper bound. Because the paper explicitly calls it an upper bound and because it is not the central comparison against CodeS-7B, DeepSeek-Coder, or Mistral, the circularity is localized and minor. Overall score 2 reflects one acknowledged definitional upper bound rather than a central derivation that reduces to its inputs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or model entities. Its central claim depends on hand-tuned RL hyperparameters, the assumption that execution correctness is a sufficient reward, and the assumption that the BIRD dev evaluation is comparable across baselines. The absence of released model weights and code leaves the evaluation assumptions unverified.

free parameters (6)
  • Reward weights (alpha_c, alpha_f, alpha_sf, alpha_l) = alpha_c=2, alpha_f=1, alpha_l=-0.5
    Hand-tuned in Sections 4.4 and 5.4 to prioritize correctness; the training signal depends on these weights.
  • GRPO group size G = 6
    Tuned over {4,6,8} in Section 5.4; affects exploration and training stability.
  • Sampling temperature T = 0.9
    Tuned over {0.6,0.9} in Section 5.4.
  • KL penalty coefficient beta = 0.001 with annealing
    Tuned in Section 5.4; decreasing KL was reported to improve accuracy from 58.4% to 59.97%.
  • LoRA adapter rank r = 64
    Tuned over {16,32,64,128} in Section 5.4.
  • Prompt token cutoff = 3000
    Prompts exceeding 3000 tokens are discarded in Sections 4.1 and 5.2, which may remove hard queries and affect results.
assumptions (4)
  • domain assumption Execution comparison Exec(o)=Exec(q_gt) is a valid and sufficient training signal for text-to-SQL on BIRD.
    Section 4.4 defines the reward this way; it assumes ground-truth SQL execution results are available and that optimizing this transfers to unseen queries.
  • domain assumption The BIRD dev set ground truth and database environment are canonical.
    Sections 5.1 and 6 assume official BIRD dev labels and databases, though the paper evaluates on a 1,500-sample subset rather than the official 1,534.
  • domain assumption The base model Qwen2.5-Coder-7B-Instruct provides a reasonable policy prior for GRPO.
    Section 4.2 uses this instruction-tuned model as the starting point, so the 'pure RL' claim still begins from pretrained weights.
  • domain assumption A 30-second execution timeout and VLLM evaluation do not distort reward magnitudes.
    Section 5.1 assigns Rc=0 to timed-out queries, which may bias the reward signal for complex queries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation." pith.science (2026). https://pith.science/paper/WSLRWPK6

@misc{pith2026250706013,
  author       = {Pith},
  title        = {Pith review of: CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WSLRWPK6}},
  note         = {Machine review of arXiv:2507.06013}
}
read the original abstract

Translating natural language into SQL (Text-to-SQL) remains a core challenge at the intersection of language understanding and structured data access. Although large language models (LLMs) have improved fluency, generating correct and executable SQL, especially for complex queries, continues to be challenging. We introduce CogniSQL-R1-Zero, a reinforcement learning (RL) framework and model that produces accurate SQL using a lightweight reward signal based on execution correctness and format-tag compliance. By avoiding intermediate supervision, hybrid pipelines and complex reward shaping, our method encourages stable learning and stronger alignment with the ultimate task objective-producing executable programs. CogniSQL-R1-Zero achieves state-of-the-art execution accuracy on Text2SQL benchmark; BIRD bench, outperforming prior supervised and instruction-tuned baselines including SFT CodeS-7B, DeepSeek-Coder 236B, and Mistral 123B-despite being trained on a significantly smaller 7B backbone. This result underscores the scalability and efficiency of our RL-based approach when trained on just four NVIDIA A100 GPUs (40 GB VRAM each). To support further research in efficient and interpretable Text-to-SQL modeling, we release two curated datasets: (i) a collection of 5,024 reasoning traces with varying context lengths, and (ii) a positive-sampled corpus of 36,356 corpus of weakly supervised queries, each annotated with six semantically diverse reasoning paths. Together, these contributions advance scalable, execution-aligned Text-to-SQL generation.

Figures

Figures reproduced from arXiv: 2507.06013 by the authors.

Figure 1
Figure 1. Component-level architecture of the experi [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 3
Figure 3. Execution accuracy across checkpoints with single-sample generation. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Execution accuracy across checkpoints with test-time scaling (best-of-6). [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 16 canonical work pages

  1. [1]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems 36 (NeurIPS), 2023

    work page 2023

  2. [2]

    Chess: Contextual harnessing for efficient sql synthesis

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. Chess: Contextual harnessing for efficient sql synthesis. InarXiv preprint arXiv:2405.16755, 2024

    arXiv 2024

  3. [3]

    Xiyan-sql: A multi-generator ensemble framework for text-to-sql.arXiv preprint arXiv:2411.08599, 2024

    Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, Jinyang Gao, Liyu Mou, and Yu Li. Xiyan-sql: A multi-generator ensemble framework for text-to-sql.arXiv preprint arXiv:2411.08599, 2024

    arXiv 2024

  4. [4]

    Towards complex text-to-sql in cross-domain database with intermediate representation

    Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. Towards complex text-to-sql in cross-domain database with intermediate representation. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4524–4535, 2019

    work page 2019

  5. [5]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  6. [6]

    Reasoning-sql: Reinforcement learning with sql tailored partial rewards for reasoning-enhanced text-to-sql.arXiv preprint arXiv:2503.23157, 2025

    Mohammadreza Pourreza, Shayan Talaei, Ruoxi Sun, Xingchen Wan, Hailong Li, Azalia Mirhoseini, Amin Saberi, Sercan Arik, et al. Reasoning-sql: Reinforcement learning with sql tailored partial rewards for reasoning-enhanced text-to-sql.arXiv preprint arXiv:2503.23157, 2025

    arXiv 2025

  7. [7]

    Zhihong Chen, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y .K. Li, Y . Wu, et al. Graph-reward-sql: Execution-free reinforcement learning for text-to-sql via graph matching and stepwise reward.arXiv preprint arXiv:2505.12380, 2025

    work page arXiv 2025

  8. [8]

    Seq2sql: Generating structured queries from natural language using reinforcement learning

    Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1747–1759, 2017

    work page 2017

Show all 50 references
  1. [9]

    Deepspeed: System optimizations enable training of trillion parameter models.arXiv preprint arXiv:2004.08936, 2020

    Jonah Rasley, Samyam Rajbhandari, Minjia He, David Garcia, Deepak Narayanan, Naman Goyal, Sharan Li, Guoqiang Zhou, Raj Puri, Bryan Zhu, et al. Deepspeed: System optimizations enable training of trillion parameter models.arXiv preprint arXiv:2004.08936, 2020. 16

    2004 arXiv

  2. [10]

    Androutsopoulos, G

    I. Androutsopoulos, G. D. Ritchie, and P. Thanisch. Natural language interfaces to databases — an introduction. Journal of Natural Language Engineering, 1(1):29–81, 1995

    1995

  3. [11]

    Text-to-sql empowered by large language models: A benchmark evaluation.arXiv preprint arXiv:2308.00000, 2023

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. Text-to-sql empowered by large language models: A benchmark evaluation.arXiv preprint arXiv:2308.00000, 2023

    2023

  4. [12]

    Unisar: A unified structure-aware autoregressive language model for text-to-sql.arXiv preprint arXiv:2203.07781, 2022

    Longxu Dou, Yan Gao, Mingyang Pan, Dingzirui Wang, Wanxiang Che, Dechen Zhan, and Jian-Guang Lou. Unisar: A unified structure-aware autoregressive language model for text-to-sql.arXiv preprint arXiv:2203.07781, 2022

    2022 arXiv

  5. [13]

    Learning to think: Information-theoretic reinforcement fine-tuning for efficient reasoning.arXiv preprint arXiv:2505.10425, 2025

    Ming Zhang, Jie Wang, et al. Learning to think: Information-theoretic reinforcement fine-tuning for efficient reasoning.arXiv preprint arXiv:2505.10425, 2025

    2025

  6. [14]

    Group relative policy optimization for image captioning.arXiv preprint arXiv:2503.01333, 2025

    Qiang Li, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Group relative policy optimization for image captioning.arXiv preprint arXiv:2503.01333, 2025

    2025 arXiv

  7. [15]

    Hao et al

    Z. Hao et al. Rl-of-thoughts: Navigating llm reasoning with inference-time reinforcement learning.arXiv preprint arXiv:2505.14140, 2025

    2025

  8. [16]

    Zhang et al

    Y . Zhang et al. Rearank: Reasoning re-ranking agent via reinforcement learning.arXiv preprint arXiv:2505.20046, 2025

    2025 arXiv

  9. [17]

    Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution.arXiv preprint arXiv:2502.18449, 2025

    Jason Wei, Zitian Xue, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution.arXiv preprint arXiv:2502.18449, 2025

    2025 arXiv

  10. [18]

    Wan et al

    X. Wan et al. Rema: Learning to meta-think for llms with multi-agent reinforcement learning. InarXiv preprint arXiv:2503.09501, 2025

    2025 arXiv

  11. [19]

    Defining and characterizing reward gaming.Advances in Neural Information Processing Systems, 35:9460–9471, 2022

    Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming.Advances in Neural Information Processing Systems, 35:9460–9471, 2022

    2022

  12. [20]

    Fei Li and H. V . Jagadish. Constructing an interactive natural language interface for relational databases. In Proceedings of the VLDB Endowment, volume 8, pages 73–84, 2014

    2014

  13. [21]

    Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers

    Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers. InProceedings of the 7th International Conference on Learning Representations (ICLR), 2021

    2021

  14. [22]

    Omnisql: Synthesizing high-quality text-to-sql data at scale.arXiv preprint arXiv:2503.02240, 2025

    Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, et al. Omnisql: Synthesizing high-quality text-to-sql data at scale.arXiv preprint arXiv:2503.02240, 2025

    2025 arXiv

  15. [23]

    Semantic decomposition of question and sql for text-to-sql parsing

    Ben Eyal, Moran Mahabi, Ophir Haroche, Amir Bachar, and Michael Elhadad. Semantic decomposition of question and sql for text-to-sql parsing. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 13629–13645, 2023

    2023

  16. [24]

    Multi-reward text-to-sql: Aggregating execution, syntax, and schema feedback.arXiv preprint arXiv:2501.23456, 2025

    Arnav Sipuria, Anshuman Kumar, Shruti Patel, and Ran Zhou. Multi-reward text-to-sql: Aggregating execution, syntax, and schema feedback.arXiv preprint arXiv:2501.23456, 2025

    2025

  17. [25]

    Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Shanelle Yao, Yifan Roman, Qingning Zhang, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. InProceedings of the 2018 Confere...

    2018

  18. [26]

    Synthetic-text-to-sql: A synthetic dataset for training language models to generate sql queries from natural language prompts.arXiv preprint arXiv:2404.00000, 2024

    Yev Meyer, Marjan Emadi, Dhruv Nathawani, Lipika Ramaswamy, Kendrick Boyd, Maarten Van Segbroeck, Matthew Grossman, Piotr Mlocek, Drew Newberry, et al. Synthetic-text-to-sql: A synthetic dataset for training language models to generate sql queries from natural language prompts...

    2024

  19. [28]

    Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls.Advances in Neural Information Processing Systems, 36, 2024

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls.Advances in Neural Information Processing Systems, 36, 2024

    2024

  20. [29]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186, 2024

    2024 arXiv

  21. [30]

    Scaling down to scale up: A guide to parameter-efficient fine-tuning.arXiv preprint arXiv:2303.15647, 2023

    Vladislav Lialin, Vijeta Deshpande, and Anna Rumshisky. Scaling down to scale up: A guide to parameter-efficient fine-tuning.arXiv preprint arXiv:2303.15647, 2023. 17

    2023 arXiv

  22. [31]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  23. [32]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  24. [33]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, p...

    2023

  25. [34]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468, 2025

    2025 arXiv

  26. [35]

    Ignore the kl penalty! boosting exploration on critical tokens to enhance rl fine-tuning.arXiv preprint arXiv:2502.06533, 2025

    Jean Vassoyan, Nathanaël Beau, and Roman Plaud. Ignore the kl penalty! boosting exploration on critical tokens to enhance rl fine-tuning.arXiv preprint arXiv:2502.06533, 2025

    2025 arXiv

  27. [36]

    Stabilizing llm training: Techniques and insights.arXiv preprint arXiv:2501.00000, 2025

    Rohan Paul. Stabilizing llm training: Techniques and insights.arXiv preprint arXiv:2501.00000, 2025

    2025

  28. [37]

    Granite code models: A family of open foundation models for code intelligence.arXiv preprint arXiv:2405.04324, 2024

    Mayank Mishra, Matt Stallone, Gaoyuan Zhang, Yikang Shen, Aditya Prasad, Adriana Meza Soria, Michele Merler, Parameswaran Selvam, Saptha Surendran, Shivdeep Singh, et al. Granite code models: A family of open foundation models for code intelligence.arXiv preprint arXiv:2405.04...

    2024 arXiv

  29. [38]

    Opencoder: The open cookbook for top-tier code large language models.arXiv preprint arXiv:2411.04905, 2024

    Siming Huang, Tianhao Cheng, Jason Klein Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J Yang, JH Liu, Chenchen Zhang, Linzheng Chai, et al. Opencoder: The open cookbook for top-tier code large language models.arXiv preprint arXiv:2411.04905, 2024

    2024 arXiv

  30. [39]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  31. [40]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024

    2024 arXiv

  32. [41]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  33. [42]

    Think2sql: Reinforce llm reasoning capabilities for text2sql.arXiv preprint arXiv:2504.00000, 2025

    Simone Papicchio, Simone Rossi, Luca Cagliero, and Paolo Papotti. Think2sql: Reinforce llm reasoning capabilities for text2sql.arXiv preprint arXiv:2504.00000, 2025

    2025

  34. [43]

    Sft codes: A lightweight supervised approach for sql generation.Proceedings of SIGMOD 2024, 2024

    X Li, Y Chen, Z Zhou, et al. Sft codes: A lightweight supervised approach for sql generation.Proceedings of SIGMOD 2024, 2024

    2024

  35. [44]

    Starcoder 2 and the stack v2: The next generation.arXiv preprint arXiv:2402.19173, 2024

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Noumane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. Starcoder 2 and the stack v2: The next generation.arXiv preprint arXiv:2402.19173, 2024

    2024 arXiv

  36. [45]

    Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence.arXiv preprint arXiv:2406.11931, 2024

    Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence.arXiv preprint arXiv:2406.11931, 2024

    2024 arXiv

  37. [46]

    Codestral: Mistral ai’s first code generation model, 2024

    Mistral AI. Codestral: Mistral ai’s first code generation model, 2024. https://mistral.ai/news/codestral

    2024

  38. [47]

    Codes: Towards building open-source language models for text-to-sql.Proceedings of the ACM on Management of Data, 2024

    Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. Codes: Towards building open-source language models for text-to-sql.Proceedings of the ACM on Management of Data, 2024

    2024

  39. [48]

    Mixtral of experts.arXiv preprint arXiv:2401.04088, 2024

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Deven- dra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts.arXiv preprint arXiv:2401.04088, 2024

    2024 arXiv

  40. [49]

    The dawn of natural language to sql: Are we fully ready?arXiv preprint arXiv:2406.01265, 2024

    Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. The dawn of natural language to sql: Are we fully ready?arXiv preprint arXiv:2406.01265, 2024

    2024 arXiv

  41. [50]

    Scaling laws for reward model overoptimization.arXiv preprint arXiv:2310.12036, 2023

    Yuntao Huang, Hugo Touvron, Yao Xu, et al. Scaling laws for reward model overoptimization.arXiv preprint arXiv:2310.12036, 2023

    2023 arXiv

  42. [51]

    Concrete problems in ai safety.arXiv preprint arXiv:1606.06565, 2016

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety.arXiv preprint arXiv:1606.06565, 2016. 18

    2016 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.