{"id":"2fedbc0c-7fd8-4bf3-9ce2-ea5ecbf98080","arxiv_id":"2608.10139","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Compiling SQL and LLM inference into one JAX tensor program removes split-execution data transfers and yields reported speedups on TPUs, with trade-offs at high filter selectivity.","lead":"This paper proposes compiling hybrid SQL and LLM queries into a single tensor program that runs entirely on a TPU, avoiding data movement between a database and a separate inference engine. It reports up to 5.3x lower latency and 9.8x higher throughput on a small benchmark, though the comparison baseline is not optimized and some numbers are internally inconsistent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No output-equivalence check means the headline speedups could be for the wrong answers; the paper needs a fused-vs-split result comparison to support its central claim.","rationale":"In good faith, the paper is a vision/position paper with a plausible architectural idea: lowering SQL and LLM operators into one JAX/XLA tensor program can remove cross-boundary copies and enable compiler-driven sharding. The Q1/Q2 latency results, the linear 1-2-4 device scaling in Table 5, and the honest discussion of Q3's high-selectivity regression all support the feasibility claim. The paper also explicitly admits many limitations, including static shapes, lack of dynamic data skipping, and multi-tenancy bottlenecks. However, the central claim is only as strong as the correctness of the computed query answers. A database query system whose answers are not verified against a reference is not yet a database result; it is a benchmark of a kernel. The reader's weakest-assumption analysis correctly identifies this: the fused program is assumed to produce identical logits, labels, and tokens, but no experiment checks that assumption. The concern is concrete because there are real mechanisms by which outputs could diverge: static padding changes effective attention masks, generative decoding in Q5 may have different sampling or termination behavior, and cross-join duplication in Q4 can expose row-order or deduplication differences. A direct output-comparison test would settle this quickly. If outputs match, the performance claims remain conditionally acceptable; if they do not, the speedups are not yet evidence for the paradigm. The separate throughput/latency inconsistency in Table 4 reinforces the need to re-measure with a clearly defined rows-per-second metric, but it is secondary to the correctness check because even perfect metrics would not validate the system without output equivalence. Therefore the correct verdict remains CONDITIONAL, and no adjustment to the reader's verdict is needed.","tokens_in":10609,"tokens_out":4057,"duration_ms":41446,"concrete_test":"Run Q1-Q5 on a fixed 49,664-row dataset with both the baseline and unified paths, using identical model weights, tokenizer, and decoding settings. Compare exact final SQL results (scalar SUM/COUNT and per-row AI_IF labels and AI_TRANSFORM outputs). For Q5, force deterministic decoding (temperature 0 or greedy) and identical max-length/padding in both paths, then compare the final count and the matched rows. Also recompute Table 4's throughput speedup using the definition rows/s = N / latency; if the 9.87x figure cannot be reproduced with this definition, report the actual definition and rerun.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the absence of any output-equivalence validation between the compiled unified program and the split-execution baseline. The paper's speedup numbers (Tables 3-5) are only meaningful if the fused SQL+LLM program returns exactly the same aggregates, classifications, and transformations as the baseline. This is never checked. The generated kernel in Figure 2 runs the same Flax forward pass, so exact-likelihood agreement is plausible, but there are unexamined failure modes: padding to static sequence length can change attention behavior near boundaries; Q5 calls AI_TRANSFORM with generative decoding, and unless decoding is seeded/argmax and length-capped identically, the COUNT(*) and LIKE match over 'masterpiece' can differ; Q4's cross-join plus self-join conditions depend on row order and duplicate handling that a tensorized program may not preserve. An inconsistent output would make the latency/throughput claims irrelevant to a query engine, regardless of speedup. Additionally, the throughput numbers in Table 4 are not simple inverses of the reported latency speedups (e.g., at Thresh=10.0 latency drops 0.02x but throughput is reported as 9.87x), which further weakens the quantitative claim until the metric definitions are tied down.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a vision for executing hybrid AI queries (SQL plus LLM-based semantic operators) by compiling the entire query into a single JAX/XLA tensor program that runs on TPUs, thereby avoiding PCIe data movement between a CPU database engine and a separate ML serving system. The authors describe a translation from SQL AST to JAX constructs, report preliminary latency and throughput measurements for five queries on SemBench-based movie review data with a Gemma-2B model, and claim up to 5.34x latency and 9.87x throughput speedups, near-linear horizontal scaling on 4 TPU devices, and outline open research challenges.","tokens_in":10827,"tokens_out":5164,"duration_ms":47801,"significance":"If the results hold, this is a useful feasibility demonstration that relational operators and LLM inference can coexist in one compiled accelerator program, enabling automatic sharding and elimination of cross-boundary copies. The paper is honest about limitations, including high-selectivity cases and multi-tenancy serialization, and it articulates a concrete research roadmap. The main contributions are the architectural vision and the preliminary performance characterization, not a full production system. However, the quantitative claims are not yet reliable because the paper omits an output-equivalence validation and contains internally inconsistent speedup metrics.","major_comments":[{"comment":"The paper never verifies that the unified compiled program returns the same query results as the split-execution baseline. The headline speedups are only meaningful if the fused SQL+LLM program yields exactly the same aggregates, classifications, and transformations as the baseline, but no experiment compares outputs. For Q1/Q2 the SUM depends on the binary classification from the LLM; with padding to a static maximum sequence length, the attention computation near sequence boundaries could alter logits and flip classifications. For Q5 the COUNT(*) and the LIKE '%masterpiece%' predicate depend on the generated essay text; unless decoding is seeded/argmax and length-capped identically in both executions, the two pipelines can disagree. For Q4 the cross-join with A.id = B.id AND A.review_id < B.review_id relies on row order and duplicate handling that a tensorized program may not preserve. Add a direct fused-vs-split output-equivalence check (or an explicit correctness argument) for each query.","section":"Section 4, Tables 3-5; Section 3.1"},{"comment":"The throughput speedup numbers are internally inconsistent with the latency speedups. In Table 3, Q3 reports latency speedup 0.47x (unified is slower) yet throughput speedup 4.29x. In Table 4, at Thresh=10.0 the latency speedup is 0.02x but throughput speedup is 9.87x; at Thresh=50.0 they are 1.00x and 4.24x, respectively. For a fixed workload, throughput is N/latency, so the throughput ratio should equal the inverse latency ratio of the two systems; the reported combinations would require the baseline to have both lower latency and lower throughput than the unified system, which is impossible. Clarify the metric definitions (end-to-end query latency vs per-micro-batch latency, steady-state throughput vs batch-inclusive throughput) and recompute the speedups.","section":"Section 4.1, Table 3; Section 4.2, Table 4"},{"comment":"The baseline is described as an eager mixture execution model (CPU DBMS plus TPU inference), but the text also refers to a 'JIT-compiled baseline' and reports very tight standard deviations (e.g., ±0.00 ms). The paper should specify precisely what the baseline does: whether the LLM serving side uses a production server (e.g., vLLM) or a simple Python loop, whether batching is identical between baseline and unified execution, and how the CPU-side SQL execution is timed. This matters because the reported gains may conflate the unification benefit with differences in baseline engineering maturity.","section":"Section 4.1"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and residual LaTeX fragments, including 'subesquently', 'classsificaiton', 'intermedidate', 'executnion', 'databse', 'textitaudienceScore', 'qureis', 'achi-tectures', 'wokrld', 'oeprators', and 'hybird'. A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The text says the results are 'demonstrated in Section??' — an unresolved cross-reference. Also, the figure numbering is confusing: Figure 1 is mentioned as 'High-level view' but the generated JAX program is Figure 2, and the frontend compilation flow is Figure 3; the narrative in Section 3.3 refers to 'Figure 3' for the consolidation, though the intended figure may be Figure 1.","section":"Section 3.3"},{"comment":"The phrase 'completely removes PCIe bug bottlenecks' should read 'PCIe bus bottlenecks'. In addition, the paper claims 'completely alleviate PCIe data movement bottlenecks' in the abstract, but Section 4.2 shows that the unified execution can be much slower at high selectivity; the claim should be qualified accordingly.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's main benchmark, SemBench [16], is co-authored by one of the authors of this submission. This is not disqualifying, but the authors should disclose the relationship and describe any benchmark-specific adaptations. The paper is a vision/position paper with preliminary experimental results; the editors may wish to weigh the absence of an artifact and the missing output-equivalence check against the workshop context. The throughput/latency inconsistencies, if not resolved, would undermine the central quantitative claims even after the correctness issue is addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort take: the core idea here is worth talking about, but the performance claims as written cannot be trusted until the throughput numbers are reconciled with the latency numbers and the system is shown to produce the same answers as the split baseline.\n\nWhat's genuinely new: compiling a hybrid SQL+LLM query into a single JAX/XLA tensor program so that relational operators and LLM inference share one accelerator graph, with GSPMD auto-sharding, is a plausible and useful direction. The SQL-to-JAX mapping in Table 1 and the generated kernel in Figure 2 show the mechanics concretely. The paper also deserves credit for being transparent: it openly reports that the baseline is an unoptimized split execution, that Q3 is slower in latency at high selectivity, and that multi-tenancy throughput is flat. That honesty is welcome in a vision paper.\n\nThe soft spots are real, though. The throughput speedups are internally inconsistent with the latency speedups. For Q3 in Table 3, unified latency is 0.47x the baseline (i.e., slower), yet the throughput speedup is reported as 4.29x. Rows/sec and per-query latency shouldn't move in opposite directions unless the throughput measurement includes some pipelining or concurrency that isn't described. Same problem in Table 4. This needs a clear metric definition or the numbers need to be re-measured.\n\nMore importantly, the paper never verifies that the unified program produces the same outputs as the split baseline. The speedup numbers are only meaningful if the aggregates and transformations match. For Q1/Q2, the forward pass is probably the same, but Q5's generative decoding and Q4's cross-join have row-order and padding sensitivities that could change results. An output-equivalence check is not optional for a query engine claim.\n\nThe baseline being unoptimized is a minor issue for a vision paper, but it means the speedup figures should be framed as 'versus a simple split baseline,' not as general gains. No artifacts are provided either.\n\nBottom line: this deserves a serious referee as a vision paper. The architecture is plausible and the authors are honest about limitations. But the performance claims need re-measurement with consistent metrics and a correctness check before they should be cited. I'd send it to a workshop/vision track and ask for those fixes.","headline":"The unified SQL+LLM compilation idea is a worthwhile vision, but the throughput numbers are internally inconsistent and the missing output-equivalence check makes the speedups untrustworthy as written.","tokens_in":11378,"tokens_out":3638,"would_cite":false,"duration_ms":35357,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single compiled tensor program can run SQL and LLM inference together, eliminating the CPU-to-accelerator data movement penalty.","keywords":["AI query compilation","hybrid SQL/LLM queries","tensor compute graph","JAX/XLA compilation","automatic sharding","semantic operators","TPU query execution","SemBench"],"falsifier":"Run the five benchmark queries from Table 2 over identical data through both the baseline split execution and the unified compiled program, and compare every returned SUM, COUNT, and AI_TRANSFORM string; any difference in those outputs would refute the semantic-equivalence premise even if the latency and throughput speedups reproduce.","tokens_in":10372,"feed_emoji":"⚡","tokens_out":6966,"duration_ms":63480,"temperature":0.7,"pith_summary":"Hybrid AI queries—SQL statements that call an LLM for classification or text transformation—are normally split between a CPU database engine and a separate accelerator, with rows and logits crossing a PCIe bus at each step. The paper's core claim is that the entire hybrid query can be lowered into a single tensor compute graph and JIT-compiled for the accelerator, so filters, LLM inference, and aggregation all execute on-device in one program. If this holds, the data-movement penalty disappears and the compiler can automatically shard the combined relational and neural work across devices, which the paper exhibits with near-linear scaling from one to four TPUs. On SemBench-style movie-review queries the paper reports up to 5.34x latency speedup and 9.8x throughput speedup over a CPU-plus-TPU split baseline, while also noting the split baseline wins when an early relational filter removes most rows.","feed_headline":"SQL and LLM in one tensor graph run queries 5.3x faster","feed_subtitle":"One compiled program removes CPU-to-TPU data transfers and auto-shards across devices.","key_machinery":"The central object is the unified tensor compute graph: a JAX program in which database rows, boolean masks, model parameters, and reductions are all tensor arrays, compiled by XLA into a single accelerator executable. The graph carries the argument through three mechanisms: it fuses relational predicates with the LLM forward pass so no intermediate results leave the device; it uses XLA/GSPMD-style automatic sharding to split rows and model weights across devices without separate CPU/TPU parallelization logic; and it uses structured control flow (jax.lax.scan) to stream micro-batches through the model while keeping intermediate state on the accelerator. Named mappings from SQL AST nodes to JAX implementations (table scans become input arrays, WHERE becomes boolean masks, AI_IF becomes an LLM forward pass with logit comparison, aggregation becomes jnp.where-masked sum) are what make the compilation mechanical.","core_discovery":"By compiling the hybrid AI query as a whole—mapping SQL clauses to JAX tensor primitives and keeping the LLM forward pass inside the same traced program—the authors show that a filter like AI_IF('is this review positive?') plus SUM can run as one @jax.jit-compiled function on TPUs. The result is that no logits or intermediate rows are copied back to the host; only the final scalar aggregate or generated tokens return. The paper reports measured speedups of 5.34x latency and 5.30x throughput for semantic filtering (Q1), 2.28x latency and 5.31x throughput for a hybrid relational-plus-semantic filter (Q2), throughput gains up to 9.8x on a selectivity sweep for Q3, and 4.00x throughput scaling on four devices via automatic data-parallel sharding. The claim is that this unified compiled execution, not specialized kernels, is what removes the CPU-TPU boundary penalty and enables global optimizations like on-device cross-join expansion and fused string matching.","pith_inferences":["The same lowering suggests that any relational operator expressible as tensor algebra—joins, grouping, ordering—could be fused into the model program, so the five queries here are a small slice of a much larger design space.","Because the paper reports no comparison of final query results between the unified program and the split baseline, the most direct next experiment is to assert semantic equivalence: run the same queries both ways and check that sums, counts, and generated strings match exactly.","The selectivity results imply a cost-based optimizer should choose between unified compiled and split execution per query, much as traditional optimizers choose whether to inline a UDF based on estimated cost.","The fixed-throughput multi-tenancy result points toward compiling independent queries into a shared batched program or adding a scheduler that interleaves tokens, rather than compiling each query as a standalone executable."],"forward_implications":["AI queries whose outputs are small—a sum, a count, a handful of generated tokens—can run end-to-end on accelerators, shrinking per-query host-device transfers from gigabytes of logits to kilobytes.","Compiler-driven auto-sharding replaces hand-written CPU/TPU parallelization; the reported 1-to-2-to-4 device scaling for Q3 is roughly 1x, 2x, 4x throughput without manual calibration.","The performance of unified compilation depends on selectivity: the baseline split execution wins for Q3 at threshold 10 (0.02x latency), so the optimal choice between compiled and split execution is query-dependent.","Under concurrent tenants the current compiled single-program design saturates at about 1377 rows/sec aggregate throughput because TPU execution is FIFO-serialized, identifying runtime scheduling of compiled kernels as the next bottleneck.","Long-generation queries with debugging outputs benefit most because on-device string matching means only matching essays are copied back, not the full logits."],"supporting_citations":[{"why":"Supplies the JAX/XLA tracing and JIT compilation infrastructure through which all SQL and LLM operators are lowered into one executable.","marker":"[7]"},{"why":"The SemBench benchmark and Rotten Tomatoes/Movies datasets provide the evaluation workload and baseline query set.","marker":"[16]"},{"why":"Represents a contemporary AI query engine whose split CPU/TPU execution the unified approach targets.","marker":"[5]"},{"why":"A production SQL engine for unstructured data, used as another split-execution baseline contrast.","marker":"[20]"},{"why":"A semantic-operator query engine whose split execution and optimization goals the paper contrasts with unified compilation.","marker":"[23]"},{"why":"The LLM serving engine cited for the serving-side execution model that compiled unified execution would replace.","marker":"[15]"}],"fun_headline_variants":["One compiled graph fuses SQL and LLM for 5.3x faster TPU queries","Compile hybrid SQL+LLM queries to one JAX graph, skip PCIe, 5.3x speedup","Unified tensor graph ends CPU-TPU shuffling, 5.3x latency win","Fusing SQL and LLM into a single trace cuts latency 5.3x on TPUs","Auto-sharded whole-query compilation: 9.8x throughput, no data copies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire argument assumes the compiled unified program produces exactly the same model outputs and therefore exactly the same query answers (sums, counts, generated strings) as the split CPU-plus-accelerator execution; no experiment in the paper checks that equivalence.","fun_headline_variants_meta":{"raw":{"variants":["One compiled graph fuses SQL and LLM for 5.3x faster TPU queries","Compile hybrid SQL+LLM queries to one JAX graph, skip PCIe, 5.3x speedup","Unified tensor graph ends CPU-TPU shuffling, 5.3x latency win","Fusing SQL and LLM into a single trace cuts latency 5.3x on TPUs","Auto-sharded whole-query compilation: 9.8x throughput, no data copies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000712,"raw_usage":{"total_tokens":3166,"prompt_tokens":867,"completion_tokens":2299,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":2173}},"tokens_in":483,"tokens_out":2299,"duration_ms":14904,"temperature":1.0,"reasoning_tokens":2173,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:11:10.426235+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the five benchmark queries from Table 2 over identical data through both the baseline split execution and the unified compiled program, and compare every returned SUM, COUNT, and AI_TRANSFORM string; any difference in those outputs would refute the semantic-equivalence premise even if the latency and throughput speedups reproduce.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the JAX/XLA tracing and JIT compilation infrastructure through which all SQL and LLM operators are lowered into one executable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Represents a contemporary AI query engine whose split CPU/TPU execution the unified approach targets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A production SQL engine for unstructured data, used as another split-execution baseline contrast."}],"review_version":1}