{"id":"845f5adc-7e10-4daf-9cf8-5bc609b1003f","arxiv_id":"2608.06950","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MIRA decomposes historical SQL corrections into independently reusable, evidence-verified repair memory items and adapts them to new queries, improving execution accuracy by 16.53 and 8.78 points on BIRD and ScienceBenchmark.","lead":"MIRA is a pluggable SQL corrector that breaks past SQL corrections into fine-grained repair items, checks each item against database evidence, and adapts only the items that match the current query. On 1,785 test queries from BIRD and ScienceBenchmark, it lifts execution accuracy by up to 16.5 points while regressing only 1.6% of already-correct queries.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core mechanism depends on an unvalidated LLM judgment that splits historical corrections into independent repair units; end-task accuracy alone cannot rule out systematically wrong memory items.","rationale":"The paper's strongest claim is that MIRA's fine-grained memory items, verified by database evidence, produce a large net gain and a superior repair/regression trade-off. The key differentiator from prior experience-based methods is the decomposition into independently reusable repair units (C1). The only evidence for this decomposition being correct is indirect: the ablation in Table 3 compares against whole-case memory, and whole-case memory could be worse for reasons unrelated to the semantic correctness of the decomposed items. There is no human evaluation of the repair units, no precision/recall analysis of memory item boundaries, and no evidence that the LLM judge's decisions correlate with ground truth. This is load-bearing because if the decomposition is systematically wrong, the method's conceptual contribution is undermined even if the end-task numbers hold; the numbers could be driven by the same GPT-5 model's direct correction ability, with the memory store acting as a noisy prompt template. My proposed check tests the decomposition against human annotation and measures whether correcting the memory items changes end-task accuracy. I do not see evidence strong enough to reject the paper, but the absence of this validation justifies keeping the conditional verdict. The reader identified the same weakest assumption, so no verdict adjustment is needed.","tokens_in":13556,"tokens_out":9829,"duration_ms":100299,"concrete_test":"Select 100 historical corrections from the training splits, stratified across the six settings. Have two senior SQL annotators independently decompose each correction into repair units and write the semantic contract for each unit, following the paper's definitions; measure inter-annotator agreement. Run MIRA's Repair Recovery and Repair Unit Identification on the same corrections and compare the produced units to the human annotation (unit boundaries and contract correctness). Then run the full MIRA online stage on the held-out test set twice: once with the original LLM-generated memory and once with memory items replaced by the human-annotated units, keeping retrieval and adaptation code unchanged.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MIRA's central contribution is the offline decomposition of each historical correction into 'independently judgeable repair units' by an LLM-based semantic judge (Section 3.2.2). A structural group is accepted only if the judge decides, from the historical question, database facts, and execution contrasts, that the validated repair SQL satisfies a required behavior while the SQL with the group reverted conflicts with it. This judgment is never checked against a gold standard. The paper reports only end-task execution accuracy (Table 1) and the ablation in Table 3, which shows that replacing decomposed memory with whole-case records reduces EX by 5.39 pp on one BIRD setting. That ablation does not validate the decomposition itself: a whole-case baseline could be worse for many reasons (prompt clutter, over-activation, irrelevant edits) even if the decomposed items are themselves mislabeled. If the judge systematically splits or merges repair units incorrectly, the memory items encode wrong semantic contracts, and the online LLM (also GPT-5) may still produce correct SQL by direct reasoning, making the end-task results insensitive to memory quality. Thus the paper does not establish that the reported gains actually arise from the proposed memory representation, which is the paper's core claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MIRA, a pluggable post-generation SQL corrector that reuses confirmed historical corrections from the same database. Offline, MIRA recovers a validated repair SQL for each historical correction, groups structural edits, and uses an LLM-based semantic judge to isolate independently judgeable repair units, each stored as a memory item with a semantic contract, structural signature, target local check, and provenance. Online, it retrieves candidate items semantically and structurally, verifies whether the recorded error recurs using the current question, SQL, and bounded database probes, then adapts activated items into a single rewrite that must pass mechanical validation. The evaluation covers 1,785 test queries from BIRD and ScienceBenchmark generated by CHESS, DeepEye-SQL, and OmniSQL-32B, reporting execution-accuracy gains of 16.53 and 8.78 percentage points respectively, with 261 repairs and 18 regressions, plus an ablation on one setting and an analysis of single- versus multi-item repairs.","tokens_in":13791,"tokens_out":4172,"duration_ms":47935,"significance":"If the claims hold, MIRA offers a practical, training-free approach to SQL correction that stores history at a finer granularity than whole-case retrieval, and the evidence-verification step is a sensible response to the regression problem that plagues experience-based correctors. The empirical pattern is consistent across six benchmark-agent settings, and the ablations in Table 3 show that each of the three main components contributes in a directionally plausible way. The clearest strength is the clean problem decomposition and the low regression count relative to comparators. However, the central mechanism---the semantic judge that decides where one repair unit ends and another begins---is never directly validated, and the empirical claims rest on a single unseeded split without confidence intervals. These gaps currently prevent the paper from fully establishing that the observed gains come from the proposed memory representation rather than from the underlying LLM's general correction ability.","major_comments":[{"comment":"The core contribution is the decomposition of historical corrections into independently judgeable repair units by an LLM-based semantic judge, yet the manuscript never validates these unit boundaries against any gold standard. Table 3 shows that replacing decomposed items with whole-case records reduces EX by 5.39 points, but that ablation cannot distinguish a correct decomposition from a systematically wrong one that the online GPT-5 corrector happens to compensate for through direct reasoning. Please add a direct evaluation of memory-item quality: for example, compare judged repair-unit boundaries with human-annotated repair units on a sample of historical corrections, report precision/recall of accepted groups and inter-annotator agreement, and test stability when the semantic judge model is varied.","section":"3.2.2 and Table 3"},{"comment":"All main results come from a single, unseeded 25/75 train/test split. There are no confidence intervals, bootstrap estimates, or paired significance tests, so it is unclear whether key differences such as the 4.26-point overall EX advantage over MAGIC, or the 4.01-point deficit on ScienceBenchmark-CHESS, are stable or within noise. Because every method receives exactly the same current SQL and split, paired comparisons are natural. Please report variability across multiple splits or a bootstrap over test queries, and give paired significance tests for the main EX differences.","section":"4.1 and Table 1"},{"comment":"The paper does not report how many historical corrections actually produce a validated repair SQL and how many structural groups are accepted by the semantic judge. If many corrections are discarded because no candidate matches the reference result or because the judge rejects the group, the memory store is a highly selective subset of the confirmed corrections, and the reported gains may not reflect what would happen under realistic correction distributions. Please report per-database yield statistics: number of corrections yielding at least one item, number of accepted versus rejected groups, the distribution of memory items per correction, and the main reasons for discards.","section":"3.2.1 and 3.2.3"}],"minor_comments":[{"comment":"There is a formatting error in the MAGIC row for ScienceBenchmark: the CHESS and DeepEye entries appear as \"57.1454.91\" with no separator, which makes the table hard to read.","section":"Table 1"},{"comment":"The ablation study is conducted only on the 371 BIRD-DeepEye test cases. The paper motivates all three components as general, so please justify why one setting is representative or add at least one additional setting to show that the ablation pattern is not specific to that agent and database group.","section":"4.4"},{"comment":"The formulation assumes confirmed historical corrections are clean pairs (q_i, s_i^-, s_i^+). In a real deployment, user-confirmed corrections may contain noisy or partial feedback; a short discussion of how MIRA would handle such noise would strengthen the practical framing.","section":"2.2"},{"comment":"The repair budget is mentioned but never specified numerically. Please state the budget (number of rounds or candidates) used in the experiments, since it directly affects both the offline cost in Table 2(b) and the yield of memory items.","section":"3.2.1"},{"comment":"No artifact or code link is provided. Given the number of LLM calls, prompts, and thresholds involved, releasing the exact prompts, memory items, and evaluation pipeline would be important for reproducibility.","section":"4.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a database or NLP systems venue and the empirical story is appealing. The main risk is that the core semantic-judge decomposition is asserted rather than directly validated; I recommend requesting the direct memory-item validation and statistical reporting described in the major comments. I do not see a fatal flaw, but the current evidence is not yet strong enough for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a real contribution, worth a serious referee. The core idea is genuinely finer-grained than MAGIC, Memo-SQL, or TK-Boost: split historical corrections into independently reusable memory items with a semantic contract, structural signature, target local check, and then verify recurrence against the current SQL and database before adapting. That is not a parameter scan. The empirical pattern is consistent: 1,785 test queries across six agent-benchmark settings, 261 repairs versus 18 regressions, and an ablation that isolates each of the three components with distinct effects. The 13.61-point overall net gain and 1.59% regression rate is the strongest repair-regression trade-off in the comparison.\n\nWhat is soft: first, no code or data release in this version, and no confidence intervals or significance tests. The gain is large enough that it is probably not noise, but per-setting counts are small and the random split has no stated seed. Second, the baseline comparison is uneven: MIRA uses GPT-5 for all its LLM stages while two comparators use released smaller specialized models, and TK-Boost is adapted rather than used in its original setting. They disclose this, but it limits clean attribution. Third, and most substantive, the stress-test concern is real: the semantic judge that decomposes historical corrections into repair units is never validated directly against gold repair boundaries. End-task accuracy and the whole-case ablation do not rule out systematic mislabeling, because the online GPT-5 might still fix queries by direct reasoning even when memory items encode wrong contracts.\n\nThat said, the concern is not fatal. The ablation shows decomposed memory beats whole-case records by 5.39 points, 86% of successful repairs use a single item, and the mechanism is coherent. The authors should manually check a sample of memory items or run a probe tracing whether activated items are actually the cause of fixes. Those are fixable experimental rigor issues, not a load-bearing flaw. Bottom line: this deserves peer review. It is a solid within-subfield contribution with a genuine mechanism, I would cite it, and I would bring it to our reading group.","headline":"Fine-grained repair memory with evidence verification gives real gains and deserves a careful referee; the main open question is whether the unvalidated semantic decomposition actually drives the improvements.","tokens_in":14296,"tokens_out":1683,"would_cite":true,"duration_ms":17692,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Historical SQL corrections are reused safely when decomposed into independently verifiable repair units and activated only on database evidence.","keywords":["Text-to-SQL","SQL correction","repair memory","repair unit decomposition","database evidence","execution accuracy","experience-based correction","regression control"],"falsifier":"Inspect the repair unit decomposition directly: for a random sample of the 261 successful repairs, take the source support of the activated memory item and check, against the gold SQL and the historical question, whether the item's semantic contract isolates exactly one required behavior. If a substantial fraction of items encodes multiple fused repairs or an incorrect contract, the decomposition claim is not doing the work attributed to it. Alternatively, permute the memory items across the same database and rerun online reuse; if execution accuracy does not collapse, the evidence-verification and adaptation stages, not the item boundaries, are carrying the result.","tokens_in":13361,"feed_emoji":"🗄️","tokens_out":8805,"duration_ms":85488,"temperature":0.7,"pith_summary":"Text-to-SQL systems often produce SQL that runs correctly but answers the wrong question. MIRA is a post-generation corrector, a component that revises a first-attempt SQL query after the fact, that repairs such errors by reusing confirmed corrections from the same database without retraining a model. Its central proposal is that a historical correction should not be stored as one fused experience: the correction is decomposed into independently reusable repair units, each carrying a semantic contract, a structural signature, and a database check. At reuse time a retrieved unit is activated only when the current question, the current SQL, and database evidence all indicate that the recorded error actually recurs, then the repair is adapted to the current query rather than copied. Across 1,785 test queries from three upstream agents, the paper reports a 16.53-point execution-accuracy gain on BIRD and an 8.78-point gain on ScienceBenchmark, with 261 successful repairs and only 18 regressions.","feed_headline":"SQL repair memory: 261 fixes, only 18 regressions","feed_subtitle":"Splitting historical corrections into verifiable repair units lifts execution accuracy and keeps regressions under 2 percent.","key_machinery":"The load-bearing object is the repair memory item: one independently judgeable repair unit distilled from a historical incorrect-corrected SQL pair, carrying a semantic contract, a structural signature, and a target local check. The argument runs through three mechanisms. First, repair recovery uses generate-and-validate to produce a validated repair SQL whose execution result matches the confirmed correction, so the offline reference supplies the validation target. Second, repair unit identification separates the changes by structural dependency and accepts a group only when the historical question, execution contrast, and database facts show it corrects a required behavior. Third, at inference time evidence-verified memory activation tests whether the current task requires the recorded behavior and whether the current SQL plus bounded database probes exhibit the recorded violation, and target local adaptation binds the activated repairs to the current SQL while preserving unrelated logic.","core_discovery":"The paper's central claim is that confirmed corrections from the same database become a reliable correction resource only when they are stored as independently judgeable repair units and each unit is admitted to a rewrite only after database evidence confirms its recorded error recurs in the current SQL. MIRA builds these units offline by recovering a validated repair SQL from the historical incorrect SQL, grouping AST-level changes by dependency, and accepting a group as a repair unit only when the historical question and objective database facts show the group implements a required behavior that the un-repaired form violates. Online, retrieval is only a candidate gate; activation requires both that the current question demands the recorded behavior and that the current SQL exhibits the recorded violation, optionally confirmed by a bounded read-only probe. The empirical claim is that this decomposition plus evidence gating plus local adaptation is what reconciles repair coverage with low regression: 261 successful repairs against 18 regressions, and the ablations show each component contributes distinctly to coverage or precision.","pith_inferences":["Editorial extension: the paper's decomposition principle should carry to other correction settings where one recorded fix bundles several independent changes, such as code repair or configuration repair, provided a database-like oracle can evaluate each candidate change in isolation.","Editorial extension: a deployment-quality test would be to feed MIRA a stream of corrections in the order they arrive and freeze memory after each one, then measure whether early-firing items stabilize or drift; the paper's fixed 25/75 split cannot reveal this dynamic.","Editorial extension: the paper does not report how many memory items each database yields or how often each item fires, so it remains open whether 261 repairs come from a few high-frequency units or broad coverage; adding these distributions would sharpen the memory-construction design.","Editorial extension: because the same general-purpose correction model performs judging and rewriting, an ablation swapping the base model would separate the contribution of the MIRA scaffolding from the raw reasoning ability of the decoder."],"forward_implications":["MIRA improves execution accuracy on every one of the six benchmark-and-upstream-agent combinations, with a 16.53-point gain on BIRD and an 8.78-point gain on ScienceBenchmark, without updating any model parameters.","Decomposition into repair units is what protects correct SQL: replacing it with whole-pair records cuts repairs from 42 to 29 and raises regressions from 3 to 10 on a 371-query BIRD subset.","Evidence-verified activation is the main regression shield: similarity-only selection keeps almost all repairs but raises regressions from 3 to 14, while the full method regresses only 18 of 1,130 initially correct queries.","Independent memory items can be composed: 36 of the 261 successful repairs use two or three items bound to distinct locations in one rewrite.","Training-free, memory-based correction transfers to a new benchmark: on ScienceBenchmark the two training-dependent comparators lose accuracy while MIRA gains 8.78 points."],"supporting_citations":[{"why":"Supplies the coarse-grained experience-distillation baseline that fuses histories into global self-correction guidelines, which MIRA is designed to beat.","marker":"[2]"},{"why":"Supplies the retrieval-plus-multi-agent comparator that combines similar repair examples with a trained SQL tool.","marker":"[4]"},{"why":"Supplies the trained specialized action-refinement comparator used as a strong correction baseline.","marker":"[21]"},{"why":"Supplies the historical-knowledge baseline that refines SQL through CTE-level feedback, evaluated in adapted form.","marker":"[1]"},{"why":"Supplies the complete-case retrieval approach, the whole-case memory strategy whose granularity MIRA challenges.","marker":"[30]"},{"why":"Supplies the corrected BIRD development collection used as evaluation data.","marker":"[11]"},{"why":"Defines the BIRD benchmark and its official execution-based evaluator used for correctness measurement.","marker":"[18]"},{"why":"Defines the ScienceBenchmark development set and its official execution evaluator used for the second evaluation.","marker":"[33]"},{"why":"Supplies the execution-result equivalence criterion used both for correctness and for validating repaired SQL.","marker":"[34]"},{"why":"Supplies the generate-and-validate automated program repair loop that Repair Recovery adapts.","marker":"[14]"}],"fun_headline_variants":["MIRA: evidence-gated repair memory lifts SQL accuracy","SQL repair units verified by data boost Text-to-SQL","MIRA: reuse repairs only when database evidence confirms","Evidence-checked memory items cut SQL regressions","MIRA: independent repair units, verified reuse, higher accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The memory is only as sound as the LLM-based judge that decides, from the historical question, database facts, and execution contrasts, where one independent repair unit ends and another begins; the paper validates end-task accuracy but never directly audits the memory items themselves, so a systematic mislabeling of repair boundaries could still leave the aggregate numbers intact.","fun_headline_variants_meta":{"raw":{"variants":["MIRA: evidence-gated repair memory lifts SQL accuracy","SQL repair units verified by data boost Text-to-SQL","MIRA: reuse repairs only when database evidence confirms","Evidence-checked memory items cut SQL regressions","MIRA: independent repair units, verified reuse, higher accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000147,"raw_usage":{"total_tokens":1191,"prompt_tokens":953,"completion_tokens":238,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":159}},"tokens_in":569,"tokens_out":238,"duration_ms":2903,"temperature":1.0,"reasoning_tokens":159,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:45:12.169651+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the repair unit decomposition directly: for a random sample of the 261 successful repairs, take the source support of the activated memory item and check, against the gold SQL and the historical question, whether the item's semantic contract isolates exactly one required behavior. If a substantial fraction of items encodes multiple fused repairs or an incorrect contract, the decomposition claim is not doing the work attributed to it. Alternatively, permute the memory items across the same database and rerun online reuse; if execution accuracy does not collapse, the evidence-verification and adaptation stages, not the item boundaries, are carrying the result.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the retrieval-plus-multi-agent comparator that combines similar repair examples with a trained SQL tool."},{"cited_title":"Parameswaran","cited_arxiv_id":null,"evidence_quote":"Supplies the historical-knowledge baseline that refines SQL through CTE-level feedback, evaluated in adapted form."},{"cited_title":"InEMNLP (Findings)","cited_arxiv_id":null,"evidence_quote":"Supplies the complete-case retrieval approach, the whole-case memory strategy whose granularity MIRA challenges."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the corrected BIRD development collection used as evaluation data."},{"cited_title":"Chang, Fei Huang, Reynold Cheng, and Yongbin Li","cited_arxiv_id":null,"evidence_quote":"Defines the BIRD benchmark and its official execution-based evaluator used for correctness measurement."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the execution-result equivalence criterion used both for correctness and for validating repaired SQL."}],"review_version":1}