{"id":"5214de4b-1f9a-4d45-8edd-4a5d80076d3d","arxiv_id":"2502.15686","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"V-SQL uses hand-crafted database views to simplify schemas for LLMs, generating dummy SQL then reconstructing final SQL, and reports execution accuracy comparable to TA-SQL on the Bird mini-dev set.","lead":"This paper presents V-SQL, a text-to-SQL framework that first generates SQL over simplified database views and then rewrites it into real SQL. Tested on the Bird benchmark, it performs on par with existing LLM-based methods, but its reported gains rely on view definitions hand-built from the benchmark's ground-truth queries.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported V-SQL gains rest on views whose design was fit to the test set's ground-truth join operations; the evaluation does not validate the proposed view-based method as described.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern. I found no reason to disagree. The manual view construction is admitted in Section 6 and in the Conclusion ('Currently, the design of table mappings requires manual effort'), and it directly undermines the causal story: the first contribution is supposed to be a view-based schema strategy generated from the database schema, but the experiments use hand-crafted views informed by the test ground truth. This is a test-set leakage issue, not merely a disagreement with consensus, and it undermines the reported evidence even though the underlying idea is coherent and the paper is transparent. Since the reader's REJECT verdict matches this assessment, I recommend leaving the verdict unchanged unless the training-only construction experiment restores confidence.","tokens_in":8612,"tokens_out":3347,"duration_ms":32222,"concrete_test":"Reproduce V-SQL on the same 500-question split, but construct all view creation syntaxes (D') from each database's schema and from questions in the Bird training split only, never using the 500 test questions' gold SQL, or use the Figure 2 prompt with the same restriction. If total EX drops materially below the Table 3 values (56.0/52.8/45.4), the reported gains are attributable to test-fit views; if EX is stable, the leakage concern is mitigated. The authors should also release the exact view definitions and prompts so this can be checked independently.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that V-SQL's view-based schema produces competitive EX on Bird. Section 6 (Discussion) states: 'we referred to the join operations in the ground truth SQL from the dataset and manually designed the view creation syntax.' Because the 500-question test set is a withheld split of Bird's development set and the ground-truth SQL is available to the authors, the view definitions are effectively fit to the test answers: for each database, the manual design chooses exactly which foreign-key joins to collapse based on the gold queries that will be asked. This removes the schema-linking and join-comprehension difficulty V-SQL is supposed to solve, so Table 3 cannot be read as evidence for the framework. The paper's own Section 4.3 says a prompt is designed to let the LLM generate table mappings, and the Conclusion lists automatic mapping as future work, confirming that the manual test-fit views are not a minor implementation detail. No released code, view definitions, or ablations isolate the view-strategy contribution, so the reported numbers have no independent support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes V-SQL, a two-stage text-to-SQL framework that first maps database tables into simplified views, then has an LLM generate a short 'dummy SQL' against the views, and finally reconstructs an executable SQL query over the original tables. The authors report execution accuracy (EX) on a 500-question subset of the BIRD development set, comparing V-SQL with DIN-SQL, DAIL-SQL, and TA-SQL under GPT-4, GPT-4-turbo, and GPT-3.5-turbo. They report competitive or slightly better EX than TA-SQL on two of the three backbones, and claim the view-based schema reduces LLM schema-based hallucinations.","tokens_in":8772,"tokens_out":3318,"duration_ms":30727,"significance":"If the view-based schema strategy were validated in a leakage-free manner, it would be a simple and potentially useful way to reduce schema complexity for LLM-based text-to-SQL systems. The paper's central idea is clearly presented, the two-stage decomposition is easy to follow, and the appendix provides prompt templates and a worked example. These strengths are real but conditional: the empirical claim is the paper's main contribution, and that claim is undermined by the evaluation design, as detailed below.","major_comments":[{"comment":"The view definitions are explicitly fitted to the test set's ground-truth SQL. Section 6 states: 'we referred to the join operations in the ground truth SQL from the dataset and manually designed the view creation syntax.' Because the test set is a subset of the BIRD development set for which gold SQL is available, the views are constructed using information from the very queries being evaluated. This is test-set leakage, and it means the EX numbers in Table 3 do not measure the effectiveness of an independent schema-simplification method. The manual view design removes exactly the schema-linking and join-comprehension difficulty that V-SQL is supposed to solve, so the central performance claim is unsupported as reported.","section":"6. Discussion"},{"comment":"The method described in Section 4.3 is an LLM-based table-mapping prompt that automatically creates view syntax (Figure 2), but the experiments evaluate manually designed views, not this prompt. The Conclusion explicitly lists automatic mapping as future work. This discrepancy is not a minor implementation detail: it means the evaluated system is a manual procedure fit to the test data, while the actually proposed framework is the automatic procedure. The paper should evaluate the automatic mapping on a validation split disjoint from the test set, or clearly reframe the contribution as a manual view-design strategy with its own validation protocol.","section":"4.3 and 7. Conclusion"},{"comment":"There are no ablations isolating the contribution of the view-based schema. Table 3 only compares the end-to-end V-SQL system against end-to-end baselines. Without an ablation that keeps the two-stage reconstruction but removes the view representation, or that compares automatic versus manual views, or that varies the view granularity, the reported gains cannot be attributed to the view strategy. Given the leakage in the view design, an ablation is especially necessary to establish that any apparent gain comes from schema simplification rather than from information about the gold joins.","section":"5.2, Table 3"},{"comment":"The evaluation lacks statistical and reproducibility details. The test set covers only 11 databases and 500 questions, and the reported differences over TA-SQL are small (for example, 52.8 versus 50.16 for GPT-4-turbo, and 45.4 versus 43.8 for GPT-3.5-turbo), yet no error bars, significance tests, or per-database breakdowns are provided. The paper also does not report the exact view definitions used, the precise prompt versions used in the actual runs, or any code or data release. These omissions are load-bearing because the main claim rests entirely on these aggregate numbers.","section":"5.1 and 5.2"}],"minor_comments":[{"comment":"The subsection titled '4.3 Fine-grained Case Study' appears after Section 5.2 and should be renumbered (e.g., 5.3).","section":"5 (heading)"},{"comment":"The rows labeled 'V-SQLG' should be labeled 'V-SQL' consistently with the rest of the paper.","section":"Table 3"},{"comment":"The phrase 'the Bird datasets' is grammatically awkward; the proper name is the BIRD dataset (or BIRD).","section":"Throughout"},{"comment":"There is a typo in the prompt: 'foramt' should be 'format'.","section":"Figure 5"},{"comment":"The final SQL in the 'output of the second stage' is missing a closing backtick and a semicolon, which makes the example harder to read.","section":"Table 4"},{"comment":"The claim that the view-based strategy 'eliminate[s] foreign key dependencies' is imprecise: the view creation SQL still contains joins, and the foreign keys remain in the underlying tables. What is actually eliminated is the need for the LLM to write those joins when querying the view.","section":"4.1"}],"recommendation":"reject","confidential_remarks":"The paper's central empirical claim is invalid as reported because the manually designed views are constructed from the test set's ground-truth join operations. This is a load-bearing flaw, not a presentational one. The paper could potentially be revised into a valid study by evaluating automatic view generation on a disjoint validation split, but that would require new experiments and likely a different central claim. I recommend rejection of the current submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The stress-test concern holds up. The central claim — that view-based schemas improve text-to-SQL — is not supported by the reported EX numbers, because the views were hand-built from the test set's ground-truth joins. Section 6 admits this explicitly, and the automatic view-generation prompt in the appendix is not the one evaluated. So Table 3 measures hand-fitting, not the framework.\n\nCredit where it's due: the view-based schema idea is distinct from prior schema-linking work, and it's a sensible extension of TA-SQL's dummy-SQL-then-reconstruct pattern. The paper describes the framework clearly, the superhero example is helpful, and the authors are transparent about the manual design. That transparency is rare and worth acknowledging.\n\nWhere it falls apart: no ablations, no error bars, no released code or view definitions. The comparison to DIN-SQL, DAIL-SQL, and TA-SQL uses numbers on a self-created 500-question subset, which is fine, but the own-method numbers are built on leaked views. The method as described requires manual view creation, so the experiment does not validate the automatic pipeline the paper sketches. Minor issues: the section numbering is off (two 4.3s), and the claim that the view strategy reduces token usage isn't measured.\n\nWho this is for: someone working on schema linking for ICL text-to-SQL might get design ideas, but should not rely on the empirical claims. For an editor: this is not a desk-reject in the sense of being incoherent; it has a fixable flaw. A serious referee could push for an evaluation where views are generated from the schema alone or selected on a held-out validation set. If that revision happens, the idea might have merit. As it stands, reject.","headline":"The view-based schema idea is genuinely new, but the reported results are invalid because the views were hand-fit to the test set's ground-truth joins.","tokens_in":9353,"tokens_out":2892,"would_cite":false,"duration_ms":26662,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"V-SQL claims that replacing raw database schemas with hand-built views reduces LLM schema hallucinations in text-to-SQL, yielding execution accuracy on Bird comparable to or better than three in-context-learning baselines.","keywords":["text-to-SQL","large language models","view-based schema","schema linking","hallucination mitigation","in-context learning","Bird dataset","two-stage SQL generation"],"falsifier":"Run V-SQL on the same Bird mini-dev test set with views generated automatically from the schema alone, or from a held-out training split, and compare execution accuracy with the manually designed views; if automatic views do not beat the raw-schema baseline, the claim that view-based schemas reduce hallucinations is unsupported.","tokens_in":8373,"feed_emoji":"🗄️","tokens_out":4852,"duration_ms":45872,"temperature":0.7,"pith_summary":"The paper sets out to show that a major cause of LLM errors in text-to-SQL is the database schema itself: relational tables store foreign keys as ID columns spread across many tables, so even simple questions force the model to plan joins. Its proposed fix is a view-based schema, where tightly coupled tables are pre-joined into flat, low-coupling views, and a two-stage framework called V-SQL that first writes short dummy SQL against those views and then reconstructs it into executable SQL over the original tables. On a Bird mini-dev test set, the framework reaches execution accuracy comparable to or better than three established prompting baselines, with the largest gains on the smaller GPT-3.5-turbo model. If view design could be automated, the same prompt-only idea would offer a way to cut schema-based errors without fine-tuning.","feed_headline":"Views flatten schemas, lifting LLM text-to-SQL accuracy","feed_subtitle":"Two-stage V-SQL matches or beats top prompting methods on Bird; smaller models gain most.","key_machinery":"The machinery is the view-based schema: a SQL view definition that flattens foreign-key chains into a single virtual table with descriptive column names, such as v_superhero exposing eye_colour, hair_colour, gender, and publisher_name. It is paired with a two-stage pipeline in which the LLM first writes dummy SQL using only views, then rewrites that dummy SQL into original-table SQL guided by the relevant mapping rules. The view acts as a compressed intermediate representation that removes join decisions from the first reasoning step, while the reconstruction stage supplies join logic explicitly from the view definition.","core_discovery":"The paper claims that simplifying how the schema is presented, rather than improving schema linking, is what lowers hallucination in LLM text-to-SQL. It maps several foreign-key-linked tables into a single denormalized view, so a query such as listing female superheroes published by Dark Horse Comics can be answered against one virtual table instead of four joined tables. The first stage generates a concise dummy SQL over views; the second stage expands that dummy SQL into final SQL using the view creation rules. Reported execution accuracy on the Bird mini-dev split is 56.0 with GPT-4, 52.8 with GPT-4-turbo, and 45.4 with GPT-3.5-turbo, which is comparable to or better than the TA-SQL, DAIL-SQL, and DIN-SQL baselines under the same no-self-consistency setting.","pith_inferences":["An implication the paper leaves implicit is that the reported results probably depend on the views being hand-designed from ground-truth joins; a fair practical test would generate views from schema statistics or from a separate training split.","Because the view idea is essentially denormalization, it may transfer to enterprise settings where common join patterns are already known from query logs or database administration practice.","A testable extension is to measure whether the accuracy gain comes from fewer join errors in stage one or from simpler prompts overall, by ablating the reconstruction stage and inspecting failure modes.","A second-stage failure analysis would reveal whether errors are merely displaced rather than eliminated, since the final SQL still has to reconstruct the very joins the views hid."],"forward_implications":["If the central claim holds, LLM text-to-SQL accuracy can be improved without retraining or fine-tuning, purely by changing how the database schema is presented to the model.","Smaller and cheaper models gain the most, so the view strategy could narrow the accuracy gap between large and small LLMs on join-heavy questions.","Token cost per query should fall in the first stage because dummy SQL is shorter than final SQL, making the framework cheaper per attempt.","The two-stage design is modular: improvements in view design or reconstruction prompts can be swapped in without changing the underlying model."],"supporting_citations":[{"why":"Supplies the TA-SQL baseline, the dummy-SQL two-stage idea, and the Bird mini-dev test-set construction and execution-accuracy metric that V-SQL builds on and compares against.","marker":"[2]"},{"why":"DIN-SQL is a decomposed in-context-learning baseline whose accuracy is compared in the main results.","marker":"[5]"},{"why":"DAIL-SQL is an in-context-learning benchmark baseline compared in the main results.","marker":"[6]"},{"why":"Provides the GPT-3.5-turbo instruction-tuned model used as one of the three backbones in the experiments.","marker":"[7]"},{"why":"Provides the GPT-4 and GPT-4-turbo models used as the other two backbones in the experiments.","marker":"[8]"}],"fun_headline_variants":["Two-stage view trick boosts LLM text-to-SQL","Denormalized views cut join complexity for text-to-SQL","V-SQL: Simplify schema, lift SQL generation accuracy","Views replace joins to sharpen LLM SQL queries","Low-coupling views help LLMs write better SQL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes it is legitimate to design the views by looking at the join operations in the ground-truth SQL of the test set; if views must instead be created from the schema alone, the reported accuracy may not hold.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage view trick boosts LLM text-to-SQL","Denormalized views cut join complexity for text-to-SQL","V-SQL: Simplify schema, lift SQL generation accuracy","Views replace joins to sharpen LLM SQL queries","Low-coupling views help LLMs write better SQL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1208,"prompt_tokens":892,"completion_tokens":316,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":237}},"tokens_in":508,"tokens_out":316,"duration_ms":3212,"temperature":1.0,"reasoning_tokens":237,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:02:37.994088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run V-SQL on the same Bird mini-dev test set with views generated automatically from the schema alone, or from a held-out training split, and compare execution accuracy with the manually designed views; if automatic views do not beat the raw-schema baseline, the claim that view-based schemas reduce hallucinations is unsupported.","supporting_citations":[{"cited_title":"Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generation","cited_arxiv_id":null,"evidence_quote":"Supplies the TA-SQL baseline, the dummy-SQL two-stage idea, and the Bird mini-dev test-set construction and execution-accuracy metric that V-SQL builds on and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DIN-SQL is a decomposed in-context-learning baseline whose accuracy is compared in the main results."},{"cited_title":"Advances in neural information processing systems, 2022","cited_arxiv_id":null,"evidence_quote":"Provides the GPT-3.5-turbo instruction-tuned model used as one of the three backbones in the experiments."}],"review_version":1}