Pith. sign in

REVIEW 18 cited by

CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.01943 v1 pith:7ID2LFJT submitted 2024-10-02 cs.LG cs.AIcs.CLcs.DB

Mohammadreza Pourreza , Hailong Li , Ruoxi Sun , Yeounoh Chung , Shayan Talaei , Gaurav Tarlok Kakkar , Yu Gan , Amin Saberi
show 2 more authors
Fatma Ozcan Sercan O. Arik
This is my paper
classification cs.LGcs.AIcs.CLcs.DB
keywords chase-sqlselectioncandidateexecutiontext-to-sqlcandidatesframeworkgeneration
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In tackling the challenges of large language model (LLM) performance for Text-to-SQL tasks, we introduce CHASE-SQL, a new framework that employs innovative strategies, using test-time compute in multi-agent modeling to improve candidate generation and selection. CHASE-SQL leverages LLMs' intrinsic knowledge to generate diverse and high-quality SQL candidates using different LLM generators with: (1) a divide-and-conquer method that decomposes complex queries into manageable sub-queries in a single LLM call; (2) chain-of-thought reasoning based on query execution plans, reflecting the steps a database engine takes during execution; and (3) a unique instance-aware synthetic example generation technique, which offers specific few-shot demonstrations tailored to test questions.To identify the best candidate, a selection agent is employed to rank the candidates through pairwise comparisons with a fine-tuned binary-candidates selection LLM. This selection approach has been demonstrated to be more robust over alternatives. The proposed generators-selector framework not only enhances the quality and diversity of SQL queries but also outperforms previous methods. Overall, our proposed CHASE-SQL achieves the state-of-the-art execution accuracy of 73.0% and 73.01% on the test set and development set of the notable BIRD Text-to-SQL dataset benchmark, rendering CHASE-SQL the top submission of the leaderboard (at the time of paper submission).

Discussion (0). Sign in to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees

    cs.DB 2025-09 conditional novelty 7.0 of 10

    BARGAIN uses betting-based anytime-valid tests and adaptive, target-aware sampling to set model-cascade thresholds, delivering non-asymptotic quality guarantees and up to 86% greater cost savings than SUPG.

  2. SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning

    cs.CL 2026-08 conditional novelty 6.0 of 10

    SERL-SQL reweights GRPO advantages using teacher-student likelihood gaps computed under execution hindsight, improving multi-turn Text-to-SQL accuracy.

  3. On Evaluating the Adversarial Robustness of Foundation Models for Multimodal Entity Linking

    cs.IR 2025-08 unverdicted novelty 6.0 of 10

    Multimodal entity linking models are vulnerable to visual adversarial perturbations, and the proposed retrieval-augmented LLM method (LLM-RetLink) reportedly improves accuracy by 0.4% to 35.7%.

  4. RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL

    cs.CL 2025-07 conditional novelty 6.0 of 10

    RASL retrieves relevant tables and columns for text-to-SQL by decomposing schemas into semantic entities, calibrating entity-type importance on training data, and using an LLM to rank candidates, beating baselines on ...

  5. Text-to-SQL for Enterprise Data Analytics

    cs.CL 2025-07 conditional novelty 6.0 of 10

    An enterprise Text-to-SQL chatbot augmented with a knowledge graph, context ranking, and query-fixing agents achieves 53% correct or nearly correct answers on LinkedIn's internal benchmark, with knowledge graph compon...

  6. Towards Probabilistic Question Answering Over Tabular Data

    cs.CL 2025-06 reject novelty 6.0 of 10

    LUCARIO is a large-scale benchmark for probabilistic table QA, but Auto-BN's reported gains are circular because ground-truth answers come from the same Bayesian network the method uses for inference.

  7. SEED: Enhancing Text-to-SQL Performance and Practical Usability Through Automatic Evidence Generation

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SEED automatically generates evidence from database schemas, descriptions, and sampled values, improving text-to-SQL accuracy in no-evidence settings.

  8. SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SDE-SQL improves text-to-SQL accuracy by having the model generate and execute exploratory SQL probes to learn database contents before and while writing the final query.

  9. SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SQLens uses weak supervision over database and LLM error signals to detect clause-level semantic errors in text-to-SQL and fixes them iteratively, improving execution accuracy on BIRD and Spider.

  10. RAISE: Reasoning Agent for Interactive SQL Exploration

    cs.AI 2025-06 conditional novelty 6.0 of 10

    An interactive reasoning agent that explores databases with live queries raises text-to-SQL execution accuracy from 44.8% to 56.5% and achieves 81.8% best-of-N coverage on BIRD.

  11. TailorSQL: An NL2SQL System Tailored to Your Query Workload

    cs.DB 2025-05 conditional novelty 6.0 of 10

    Adding workload-derived hints and tailored retrieval from past SQL queries improves NL2SQL execution accuracy and reduces prompt size on three benchmarks.

  12. Automatic Metadata Extraction for Text-to-SQL

    cs.DB 2025-05 conditional novelty 6.0 of 10

    LLM-generated summaries of database profiling statistics outperform human-written metadata for text-to-SQL on BIRD, and a SQL-generation-based schema linker adds further gains.

  13. APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL

    cs.DB 2026-02 conditional novelty 5.0 of 10

    An agentic text-to-SQL pipeline using hypothesis-verification exploration achieves 70.7% execution accuracy on BIRD-Dev and 51.0% on Spider 2.0-Snow.

  14. PaVeRL-SQL: Text-to-SQL via Partial-Match Rewards and Verbal Reinforcement Learning

    cs.AI 2025-09 reject novelty 5.0 of 10

    PaVeRL-SQL reports SOTA execution accuracy on Spider2.0-SQLite using partial-match rewards and verbal RL, but overclaims SOTA on Spider and BIRD.

  15. Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Generating Python programs as intermediate guidance before SQL, then voting on Python execution results to select the fastest matching SQL, improves text-to-SQL execution accuracy and efficiency on BIRD and Archer.

  16. LLM-Driven Data Generation and a Novel Soft Metric for Evaluating Text-to-SQL in Aviation MRO

    cs.DB 2025-06 conditional novelty 4.0 of 10

    Introduces an F1-based soft metric for text-to-SQL evaluation and an LLM-driven data generation pipeline, tested on an aviation MRO database.

  17. Taming SQL Complexity: LLM-Based Equivalence Evaluation for Text-to-SQL

    cs.CL 2025-06 reject novelty 4.0 of 10

    An LLM-based pipeline for judging SQL query equivalence achieves high accuracy on the authors' own data, but test-set fitting and a self-defined ground truth weaken the results.

  18. Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities

    cs.CL 2025-05 conditional novelty 4.0 of 10

    A systematic review organizing LLM-based text-to-SQL methods into pre-processing, in-context learning, fine-tuning, and post-processing paradigms, with a catalog of datasets, metrics, challenges, and future directions.

Pith tools