{"id":"13872d71-c603-42df-a24f-4042f2e534ef","arxiv_id":"2607.19679","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"No single materialized-view pipeline stage or engine dominates across workloads; end-to-end time savings depend on strong interactions between enumeration, selection, and rewriting.","lead":"This paper measures how well database systems speed up repeated analytical queries by precomputing reusable query pieces, comparing the full pipeline across seven systems. It finds that no system or stage wins everywhere; performance depends on how candidate generation, view selection, and query rewriting interact.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline interaction numbers rest on unverified re-implementations of BigSubs and GnnMV; the central ranking-flip example may be an artifact.","rationale":"The reader's weakest assumption — fidelity of the BigSubs and GnnMV re-implementations — is indeed the most load-bearing point. The paper's headline interaction finding is primarily demonstrated through selector-ranking flips involving exactly these two methods. If either re-implementation deviates from the original published method, the specific numbers (43.85 pp, 47.53 pp, 42.4 pp) and the mechanism in §10.3 could be artifacts of the re-implementation rather than properties of the original algorithms. I also note a closely related, compounding issue: GnnMV's utility labels are generated using the same HIV/PostgreSQL pipeline used in evaluation, which gives the learned selector an alignment advantage that a generic cost-based selector like BigSubs does not receive. This makes the comparison of the two methods less clean even if the implementations are faithful. However, the paper's broader claim — that pipeline stages interact and that isolated stage rankings can mislead — is supported by additional evidence beyond these specific numbers, including enumerator-only ablations (ECSE vs Basic under BigSubs), cross-engine rewriter comparisons, and case studies that do not depend on GnnMV fidelity. Therefore the appropriate response is not to reject the paper but to require verification of the re-implementations and sensitivity to the training/evaluation alignment. The reader's CONDITIONAL verdict already captures this need, so no verdict adjustment is required.","tokens_in":36137,"tokens_out":8020,"duration_ms":94340,"concrete_test":"Obtain the original BigSubs and GnnMV source code from the original authors, or commission an independent re-implementation of [46] and [40] without access to this paper's code, then rerun the JOB+1GB and STATS+102MB selector comparisons under the same candidate pools, HIV rewriter, and PostgreSQL engine. If the top-view sets or the 47.53 pp / 42.4 pp gaps change by more than 10 pp, the central interaction example is re-implementation-dependent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative evidence for \"single-stage rankings can be locally valid but globally misleading\" is the JOB selector flip (§8): ECSE→Basic under BigSubs costs 43.85 pp, while BigSubs→GnnMV under Basic gains 47.53 pp; and the STATS+102MB case (§10.3) shows a 42.4 pp gap. Both examples compare BigSubs with GnnMV. Section 5.2 states these are author re-implementations: \"implement BigSubs using its ILP formulation\" and \"implement GnnMV following the model and training procedure described in its paper,\" because \"Most prior MV-selection systems are not publicly available.\" No original code or fidelity ablation is provided. Two implementation details make fidelity non-obvious: (i) Table 9 models BigSubs's utility as u=(creation_cost−scan_cost)×count, a simplified proxy; if the original BigSubs [46] uses different benefit estimation or overlap constraints, its under-ranking of views 1017/3232/906 (Table 8) may be an artifact of this simplification. (ii) Section 7.2 says GnnMV's training labels are generated by HIV rewriting on PostgreSQL — the same rewriter/engine used in evaluation. A learned selector trained on the evaluation pipeline's actual latencies is expected to beat a generic cost-based selector; this alignment alone could create the 47.53/42.4 pp gaps even if the original GnnMV, trained on a different pipeline, would not. Thus the headline interaction and the §10.3 mechanism are not securely attributable to the named methods. The broader qualitative claim of pipeline interactions may survive, but the specific quantitative demonstration is at risk.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a modular benchmark for materialized-view (MV) based query rewriting, jointly evaluating candidate enumeration, budget-constrained view selection, and query rewriting across multiple open-source and commercial engines. It introduces a cross-engine protocol for plan-transparent systems, a set of stage-wise and end-to-end metrics, and controlled ablations over a 4-workload, 7-engine matrix. The central empirical claim is that pipeline stages interact strongly: single-stage rankings of enumerators or selectors are locally valid but globally misleading, with headline examples on JOB (ECSE→Basic costing 43.85 pp under BigSubs, and BigSubs→GnnMV gaining 47.53 pp under Basic) and a STATS+102 MB case showing a 42.4 pp gap between BigSubs and GnnMV. The paper also reports cross-engine variability, rewrite failure modes, and robustness checks under workload drift, data skew, and memory pressure. The artifact is publicly available.","tokens_in":36450,"tokens_out":3982,"duration_ms":48130,"significance":"If the results hold, the paper makes a valuable contribution: it provides the first end-to-end, cross-engine benchmark of MV-based rewriting across all three pipeline stages, and it demonstrates with controlled ablations that isolated stage evaluation can mislead. The modular framework and cross-engine protocol are reusable, the coverage of industrial rewriters is impressive, and the robustness analyses (workload drift, DSB skew, hardware pressure) add useful evidence. The paper ships code and data, which strengthens reproducibility. However, the headline quantitative claims rest on author re-implementations of BigSubs and GnnMV that are not fidelity-validated, and all reported latency savings are point estimates without variance or confidence intervals. The precise magnitudes of the interaction effects are therefore not yet established, even though the qualitative interaction phenomenon is plausible and well supported by the ablations and case studies.","major_comments":[{"comment":"GnnMV is trained using utility labels generated by HIV rewriting on PostgreSQL (Section 7.2), and the evaluation that shows GnnMV beating BigSubs is performed on the same HIV/PostgreSQL pipeline. This alignment gives the learned selector direct access to the exact cost model and rewriter behavior used at test time, which may explain part of the observed gap. The paper calls this a 'fair comparison' but it is a controlled comparison only in the sense that all selectors share the same candidate pool and budget; it does not control for train/test pipeline alignment. Please add an ablation where GnnMV (or any learned selector) is trained on a different rewriter/engine and evaluated on HIV/PG, or report the sensitivity of the selector rankings to the training pipeline. Without this, the claim that GnnMV is systematically better than BigSubs is not fully supported.","section":"§7.2, §8, §9"}],"minor_comments":[{"comment":"The main text and appendix use inconsistent labels: appendix Figure 10 uses 'COM-ii' and 'Hawc' where the main text uses 'Sys-B' and 'Basic' (or 'UniView'). Please harmonize names across all figures and tables.","section":"Figures 3 and 10"},{"comment":"The text states 'switching from HIV to Sys-A's native rewriter reduces workload time saving from ~40% to ~5%' but Figure 4 does not clearly identify the exact pipeline or view set behind this number. Specify the enumerator–selector configuration and provide the precise values in the text.","section":"§9.3, Figure 4"},{"comment":"The end-to-end metric treats a slower rewritten query as 'no rewrite' (i.e., uses original latency). This is a reasonable choice, but it should be stated more prominently and its effect on the reported savings discussed, since it makes the metric more optimistic than one that counts regressions as negative savings.","section":"§7.3"},{"comment":"For the views ranked by BigSubs (MV 121, 263, 7340), the 'Est. benefit (GnnMV)' column is populated even though these views were not selected by GnnMV. It is unclear whether these are GnnMV's estimated utilities for those views; please clarify the interpretation.","section":"Table 8"},{"comment":"References [34] and [35] are duplicates, as are [46] and [47]. Please deduplicate the bibliography.","section":"References"},{"comment":"Sys-B is treated both as an enumerator and as a selector in different parts of the evaluation, but the interface through which its enumerated views are separated from its internal selection is not described. Clarify how the modular framework obtains Sys-B's enumerated candidate set without its selection.","section":"§4.2, §5.2"}],"recommendation":"major_revision","confidential_remarks":"The central risk is the fidelity of the BigSubs and GnnMV re-implementations. If the authors can provide fidelity evidence (e.g., matching published results on the original datasets, or at least a detailed comparison of the utility model and training data pipeline) or reframe the claims as implementation-specific, the paper would be acceptable. The lack of confidence intervals on the headline numbers is also a concern for a benchmark paper; a variance analysis would strengthen it. Overall, the framework and qualitative findings are sound and worth publishing after these revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful core: this is the first benchmark I know that treats MV-based query rewriting as an end-to-end pipeline — enumeration, selection, rewriting — and actually varies all three under one protocol. The cross-engine comparison via portable HIV SQL as a baseline is a smart workaround for plan-transparent engines. The controlled ablations and the case studies (especially the column-pruning story on TPC-DS and the 'risky rewrite' pattern) give real insight. I buy the qualitative claim that single-stage rankings can mislead; the workload-drift robustness check and the stage-wise analyses support that without leaning on any single number.\n\nNow the soft spots. The stress-test note is right about the central numbers. BigSubs and GnnMV are both author re-implementations, and GnnMV's training labels come from actually executing HIV rewrites on PostgreSQL — the same pipeline used for evaluation. That is a meaningful advantage for GnnMV over a cost-proxy selector like BigSubs. If that training protocol differs from the original GnnMV paper, the 47.53 pp and 42.4 pp gaps may not be attributable to the 'method.' The paper does not provide a fidelity ablation or original code. This doesn't sink the paper, but it means the headline interaction example is not yet secure. Separately, there are no repeated-run variances or confidence intervals, so we don't know how stable those percentages are. Minor: appendix Figure 10 labels don't match the main text (COM-i/COM-ii/Hawc vs Sys-A/Sys-B/UniView), which is annoying for traceability.\n\nWho is this for? People building or choosing MV management systems, and researchers who want a reusable benchmark harness. The framework itself is valuable even if some of the absolute numbers shift. I'd want the re-implementations validated or open-sourced before I'd trust the specific rankings, but the benchmark design and failure-mode taxonomy are worth engaging.\n\nRecommendation: send it to peer review. The core question is important, the methodology is mostly sound, and the weaknesses are addressable rather than fatal. I'd ask for a fidelity check and error bars before accepting.","headline":"First full-pipeline MV benchmark; the interaction story is plausible but the headline numbers depend on unverified re-implementations of BigSubs and GnnMV.","tokens_in":36963,"tokens_out":3189,"would_cite":true,"duration_ms":36142,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that materialized-view query rewriting cannot be judged stage by stage: the best enumerator or selector flips depending on the other pipeline stages.","keywords":["materialized views","query rewriting","view selection","candidate enumeration","benchmark","OLAP workloads","cross-engine evaluation","pipeline ablation"],"falsifier":"Re-run the controlled ablations with the original author-provided implementations (or a verified third-party port) of BigSubs and GnnMV; if the 42.4 percentage-point gap on the STATS/Basic/102 MB setting disappears, or BigSubs no longer under-ranks the high-coverage views, the paper's central mechanism is an artifact of re-implementation rather than a property of the methods.","tokens_in":35971,"feed_emoji":"⚙️","tokens_out":7269,"duration_ms":84064,"temperature":0.7,"pith_summary":"The paper argues that materialized-view (MV) based query acceleration is a three-stage pipeline—candidate enumeration, budget-constrained view selection, and optimizer rewriting—and that these stages interact so strongly that single-stage rankings are misleading. Using a modular framework that swaps components independently on four workloads and several engines, it shows that the same enumerator or selector can be a bottleneck or a winner depending on its partners. It also finds that portable SQL rewriting can outperform engine-native optimizer rewriting on complex workloads, while a commercial integrated system's aggressive column pruning wins only when the storage budget is tight relative to database size. The paper identifies a concrete failure mechanism: a cost-only selection utility—creation cost minus scan cost, times query count—systematically under-ranks high-coverage 'heavy tail' views, causing large gaps between selectors.","feed_headline":"No MV pipeline stage wins on its own","feed_subtitle":"Across four workloads, the same enumerator or selector is a bottleneck or a winner depending on the other two stages.","key_machinery":"The modular evaluation framework that decomposes the pipeline into enumeration, selection, and rewriting and allows controlled ablations (fix two stages, vary the third). The cross-engine protocol compares an engine's native optimizer-level rewriting against portable SQL rewrites executed on the same engine. The mechanism identified for the selector failure is a cost-only utility model: u = (creation_cost - scan_cost) × count, which penalizes moderate-cost, high-coverage views and explains why a learned utility model can outperform it under tight budgets.","core_discovery":"The central claim is that pipeline outcomes depend on how candidate generation, selection under budget, and rewriting interact, so no stage can be evaluated or optimized in isolation. Concretely, on the JOB workload at a 1 GB budget, replacing the join-graph enumerator ECSE with the plan-subtree baseline reduces end-to-end time saving by 43.85 percentage points when the selector and rewriter are fixed, while switching the selector from BigSubs to GnnMV under the weaker enumerator increases saving by 47.53 points; both resulting pipelines are high-performing. The same interaction holds across workloads and engines. The paper attributes it to structural properties: enumerators differ in join-o","pith_inferences":["A cheap heuristic approximating the learned selector—for example, ranking views by coverage times per-query saving rather than creation cost times count—could be tested against the reported 42.4 percentage-point gap; if it closes the gap, expensive training may be unnecessary.","The 'Inverse Query Reconstruction' failure mode suggests a static structural checker that flags rewrites introducing UNION ALL with inverse disjunctive predicates could prevent catastrophic regressions; such a checker could be built and validated on the paper's workloads.","Because the paper finds that high-coverage views are not necessarily large, one could hypothesize that join-order-exploring enumerators are more drift-resistant than predicate-specialized ones; a direct test would measure drift robustness on workloads where both enumerator types produce identical candidate counts.","The paper's evidence implies that future MV systems should co-design enumeration and selection around coverage-aware utility rather than optimize them independently; a system that builds workload-level join graphs and feeds coverage-weighted benefits into a budgeted ILP could capture much of the learned-selector gain without training."],"forward_implications":["Single-stage benchmarking of MV enumerators or selectors is insufficient: a ranking from one pipeline context does not transfer to another, so published comparisons that fix one context can be globally misleading.","The bottleneck stage shifts with workload: enumeration caps possible savings when high-coverage views are missing, selection dominates under tight budgets, and rewriting caps realized savings when selected views are not exploited.","Portable SQL rewriting provides a practical cross-engine baseline: on complex workloads it often beats engine-native rewriting by a large margin, so plan-transparent systems can be evaluated fairly against it.","Cost-only selection utilities can systematically omit the very views that matter most, so selectors should weight query coverage or use learned cost models when budgets are tight.","Commercial integrated MV systems do not dominate modular pipelines everywhere; they win mainly when their column-pruned views fit a tight budget and lose where coverage is the limiting factor."],"fun_headline_variants":["MV pipeline stages: no lone winners","Joint pipeline beats isolated MV optimization","Context decides MV rewrite success","For MV speed, interaction beats isolation","Swap a stage, MV gains swing wildly"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper re-implements BigSubs and GnnMV from published descriptions because the original implementations are not public; if either re-implementation deviates from the original in benefit estimation or ILP modeling, the selector rankings it reports may describe the re-implementations rather than the methods themselves.","fun_headline_variants_meta":{"raw":{"variants":["MV pipeline stages: no lone winners","Joint pipeline beats isolated MV optimization","Context decides MV rewrite success","For MV speed, interaction beats isolation","Swap a stage, MV gains swing wildly"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00032,"raw_usage":{"total_tokens":1622,"prompt_tokens":709,"completion_tokens":913,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":854}},"tokens_in":453,"tokens_out":913,"duration_ms":12134,"temperature":1.0,"reasoning_tokens":854,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T12:00:24.282338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the controlled ablations with the original author-provided implementations (or a verified third-party port) of BigSubs and GnnMV; if the 42.4 percentage-point gap on the STATS/Basic/102 MB setting disappears, or BigSubs no longer under-ranks the high-coverage views, the paper's central mechanism is an artifact of re-implementation rather than a property of the methods.","supporting_citations":[],"review_version":1}