REVIEW 8 cited by
DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
There is currently a significant gap between the performance of fine-tuned models and prompting approaches using Large Language Models (LLMs) on the challenging task of text-to-SQL, as evaluated on datasets such as Spider. To improve the performance of LLMs in the reasoning process, we study how decomposing the task into smaller sub-tasks can be effective. In particular, we show that breaking down the generation problem into sub-problems and feeding the solutions of those sub-problems into LLMs can be an effective approach for significantly improving their performance. Our experiments with three LLMs show that this approach consistently improves their simple few-shot performance by roughly 10%, pushing the accuracy of LLMs towards SOTA or surpassing it. On the holdout test set of Spider, the SOTA, in terms of execution accuracy, was 79.9 and the new SOTA at the time of this writing using our approach is 85.3. Our approach with in-context learning beats many heavily fine-tuned models by at least 5%. Additionally, when evaluated on the BIRD benchmark, our approach achieved an execution accuracy of 55.9%, setting a new SOTA on its holdout test set.
Forward citations
Cited by 8 Pith papers
-
Text-to-SQL Oriented to the Process Mining Domain: A PT-EN Dataset for Query Translation
text-2-SQL-4-PM is a bilingual benchmark with 1,655 human utterances and 205 gold SQL statements for text-to-SQL in process mining.
-
Text-to-SQL for Enterprise Data Analytics
An enterprise Text-to-SQL chatbot augmented with a knowledge graph, context ranking, and query-fixing agents achieves 53% correct or nearly correct answers on LinkedIn's internal benchmark, with knowledge graph compon...
-
LLM-based Question-Answer Framework for Sensor-driven HVAC System Interaction
JARVIS, an LLM-based HVAC question-answering framework with an Expert-LLM, a parameterized SQL builder, and bottom-up planning, outperforms a text-to-SQL baseline and its own ablations on a small expert-curated dataset.
-
SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes
SDE-SQL improves text-to-SQL accuracy by having the model generate and execute exploratory SQL probes to learn database contents before and while writing the final query.
-
TailorSQL: An NL2SQL System Tailored to Your Query Workload
Adding workload-derived hints and tailored retrieval from past SQL queries improves NL2SQL execution accuracy and reduces prompt size on three benchmarks.
-
Automatic Metadata Extraction for Text-to-SQL
LLM-generated summaries of database profiling statistics outperform human-written metadata for text-to-SQL on BIRD, and a SQL-generation-based schema linker adds further gains.
-
AttnLink: Turning Attention into Schema Links for Text-to-SQL
Attention from an LLM's generation-start position, pooled over schema candidates, yields millisecond-scale, controllable schema-linking scores that beat prior linkers on three benchmarks.
-
Confidence Estimation for Text-to-SQL in Large Language Models
Consistency-based methods are the most reliable confidence signal for text-to-SQL in black-box LLMs, and executing queries against a database adds a useful correctness signal.
Discussion (0). Continue with ORCID to comment.