Pith. sign in

REVIEW 17 cited by

MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.11242 v6 pith:T3D2FRVO submitted 2023-12-18 cs.CL

Bing Wang , Changyu Ren , Jian Yang , Xinnian Liang , Jiaqi Bai , LinZheng Chai , Zhao Yan , Qian-Wen Zhang
show 3 more authors
Di Yin Xing Sun Zhoujun Li
This is my paper
classification cs.CL
keywords frameworkgpt-4mac-sqltext-to-sqlaccuracyagenttoolsachieves
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent LLM-based Text-to-SQL methods usually suffer from significant performance degradation on "huge" databases and complex user questions that require multi-step reasoning. Moreover, most existing methods neglect the crucial significance of LLMs utilizing external tools and model collaboration. To address these challenges, we introduce MAC-SQL, a novel LLM-based multi-agent collaborative framework. Our framework comprises a core decomposer agent for Text-to-SQL generation with few-shot chain-of-thought reasoning, accompanied by two auxiliary agents that utilize external tools or models to acquire smaller sub-databases and refine erroneous SQL queries. The decomposer agent collaborates with auxiliary agents, which are activated as needed and can be expanded to accommodate new features or tools for effective Text-to-SQL parsing. In our framework, We initially leverage GPT-4 as the strong backbone LLM for all agent tasks to determine the upper bound of our framework. We then fine-tune an open-sourced instruction-followed model, SQL-Llama, by leveraging Code Llama 7B, to accomplish all tasks as GPT-4 does. Experiments show that SQL-Llama achieves a comparable execution accuracy of 43.94, compared to the baseline accuracy of 46.35 for vanilla GPT-4. At the time of writing, MAC-SQL+GPT-4 achieves an execution accuracy of 59.59 when evaluated on the BIRD benchmark, establishing a new state-of-the-art (SOTA) on its holdout test set (https://github.com/wbbeyourself/MAC-SQL).

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning

    cs.CL 2026-08 conditional novelty 6.0 of 10

    SERL-SQL reweights GRPO advantages using teacher-student likelihood gaps computed under execution hindsight, improving multi-turn Text-to-SQL accuracy.

  2. SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text

    cs.AI 2026-05 conditional novelty 6.0 of 10

    A 24-dataset benchmark for inducing schema graphs from raw text, plus an auditable LLM-based pipeline that reports the highest scores on the benchmark's four schema-similarity metrics.

  3. RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL

    cs.CL 2025-07 conditional novelty 6.0 of 10

    RASL retrieves relevant tables and columns for text-to-SQL by decomposing schemas into semantic entities, calibrating entity-type importance on training data, and using an LLM to rank candidates, beating baselines on ...

  4. LLM4Hint: Leveraging Large Language Models for Hint Recommendation in Offline Query Optimization

    cs.DB 2025-07 conditional novelty 6.0 of 10

    A hybrid LLM-and-encoder framework recommends query optimization hints offline and reports better speedup and latency than Bao, Lero, and FASTgres.

  5. SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SDE-SQL improves text-to-SQL accuracy by having the model generate and execute exploratory SQL probes to learn database contents before and while writing the final query.

  6. SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SQLens uses weak supervision over database and LLM error signals to detect clause-level semantic errors in text-to-SQL and fixes them iteratively, improving execution accuracy on BIRD and Spider.

  7. RAISE: Reasoning Agent for Interactive SQL Exploration

    cs.AI 2025-06 conditional novelty 6.0 of 10

    An interactive reasoning agent that explores databases with live queries raises text-to-SQL execution accuracy from 44.8% to 56.5% and achieves 81.8% best-of-N coverage on BIRD.

  8. Toward Structured Knowledge Reasoning: Contrastive Retrieval-Augmented Generation on Experience

    cs.CL 2025-06 conditional novelty 6.0 of 10

    CoRE improves structured knowledge reasoning by retrieving both correct and incorrect past examples into the prompt, using MCTS-generated experience memory.

  9. TailorSQL: An NL2SQL System Tailored to Your Query Workload

    cs.DB 2025-05 conditional novelty 6.0 of 10

    Adding workload-derived hints and tailored retrieval from past SQL queries improves NL2SQL execution accuracy and reduces prompt size on three benchmarks.

  10. Knowledge Base Construction for Knowledge-Augmented Text-to-SQL

    cs.CL 2025-05 conditional novelty 6.0 of 10

    KAT-SQL constructs a reusable knowledge base for text-to-SQL by expanding training data with LLM-generated knowledge and retrieving/refining the best entries for each query.

  11. Automatic Metadata Extraction for Text-to-SQL

    cs.DB 2025-05 conditional novelty 6.0 of 10

    LLM-generated summaries of database profiling statistics outperform human-written metadata for text-to-SQL on BIRD, and a SQL-generation-based schema linker adds further gains.

  12. ODIN: A NL2SQL Recommender to Handle Schema Ambiguity

    cs.DB 2025-05 conditional novelty 6.0 of 10

    ODIN uses schema masking, conformal-prediction filtering, and user-preference hints to return a small set of SQL queries that includes the correct one more often than baseline sampling on ambiguous-schema benchmarks.

  13. ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Execution-driven bootstrapping, where a model generates SQL, executes it, and keeps only queries that run, lets a 7B model outperform GPT-4o on PostgreSQL, MySQL, and Oracle text-to-SQL benchmarks.

  14. TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models

    cs.CL 2025-06 conditional novelty 5.0 of 10

    TReB evaluates 26 large language models on 26 table reasoning subtasks using textual, programmatic, and interleaved reasoning modes, finding that the best model reaches only about 70 on a 0-100 judging scale.

  15. SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A three-stage small-model assistant corrects text-to-SQL output by converting SQL into pandas-like action trajectories and refining schema and logic before the LLM rewrites the query.

  16. Design and testing of an agent chatbot supporting decision making with public transport data

    cs.AI 2025-05 conditional novelty 4.0 of 10

    An agent-based LLM chatbot answers natural-language questions over GTFS public transport data, with a GPT-generated test set showing 53% fully correct answers on simple queries and 6 of 34 on complex ones.

  17. Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities

    cs.CL 2025-05 conditional novelty 4.0 of 10

    A systematic review organizing LLM-based text-to-SQL methods into pre-processing, in-context learning, fine-tuning, and post-processing paradigms, with a catalog of datasets, metrics, challenges, and future directions.

Pith tools