REVIEW 3 major objections 3 minor 32 references
AI Query Compilation for Unified and Optimized Execution
T0 review · 3 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A single compiled tensor program can run SQL and LLM inference together, eliminating the CPU-to-accelerator data movement penalty.
desk verdict The unified SQL+LLM compilation idea is a worthwhile vision, but the throughput numbers are internally inconsistent and the missing output-equivalence check makes the speedups untrustworthy as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the unified tensor compute graph: a JAX program in which database rows, boolean masks, model parameters, and reductions are all tensor arrays, compiled by XLA into a single accelerator executable. The graph carries the argument through three mechanisms: it fuses relational predicates with the LLM forward pass so no intermediate results leave the device; it uses XLA/GSPMD-style automatic sharding to split rows and model weights across devices without separate CPU/TPU parallelization logic; and it uses structured control flow (jax.lax.scan) to stream micro-batches through the model while keeping intermediate state on the accelerator. Named mappings from SQL AST nodes to JAX implementations (table scans become input arrays, WHERE becomes boolean masks, AI_IF becomes an LLM forward pass with logit comparison, aggregation becomes jnp.where-masked sum) are what make the compilation mechanical.
What would settle it
Run the five benchmark queries from Table 2 over identical data through both the baseline split execution and the unified compiled program, and compare every returned SUM, COUNT, and AI_TRANSFORM string; any difference in those outputs would refute the semantic-equivalence premise even if the latency and throughput speedups reproduce.
Extended reading notes
Core claim
By compiling the hybrid AI query as a whole—mapping SQL clauses to JAX tensor primitives and keeping the LLM forward pass inside the same traced program—the authors show that a filter like AI_IF('is this review positive?') plus SUM can run as one @jax.jit-compiled function on TPUs. The result is that no logits or intermediate rows are copied back to the host; only the final scalar aggregate or generated tokens return. The paper reports measured speedups of 5.34x latency and 5.30x throughput for semantic filtering (Q1), 2.28x latency and 5.31x throughput for a hybrid relational-plus-semantic filter (Q2), throughput gains up to 9.8x on a selectivity sweep for Q3, and 4.00x throughput scaling on four devices via automatic data-parallel sharding. The claim is that this unified compiled execution, not specialized kernels, is what removes the CPU-TPU boundary penalty and enables global optimizations like on-device cross-join expansion and fused string matching.
Load-bearing premise
The entire argument assumes the compiled unified program produces exactly the same model outputs and therefore exactly the same query answers (sums, counts, generated strings) as the split CPU-plus-accelerator execution; no experiment in the paper checks that equivalence.
Editorial extensions
If this is right
- AI queries whose outputs are small—a sum, a count, a handful of generated tokens—can run end-to-end on accelerators, shrinking per-query host-device transfers from gigabytes of logits to kilobytes.
- Compiler-driven auto-sharding replaces hand-written CPU/TPU parallelization; the reported 1-to-2-to-4 device scaling for Q3 is roughly 1x, 2x, 4x throughput without manual calibration.
- The performance of unified compilation depends on selectivity: the baseline split execution wins for Q3 at threshold 10 (0.02x latency), so the optimal choice between compiled and split execution is query-dependent.
- Under concurrent tenants the current compiled single-program design saturates at about 1377 rows/sec aggregate throughput because TPU execution is FIFO-serialized, identifying runtime scheduling of compiled kernels as the next bottleneck.
- Long-generation queries with debugging outputs benefit most because on-device string matching means only matching essays are copied back, not the full logits.
Reading between the lines
- The same lowering suggests that any relational operator expressible as tensor algebra—joins, grouping, ordering—could be fused into the model program, so the five queries here are a small slice of a much larger design space.
- Because the paper reports no comparison of final query results between the unified program and the split baseline, the most direct next experiment is to assert semantic equivalence: run the same queries both ways and check that sums, counts, and generated strings match exactly.
- The selectivity results imply a cost-based optimizer should choose between unified compiled and split execution per query, much as traditional optimizers choose whether to inline a UDF based on estimated cost.
- The fixed-throughput multi-tenancy result points toward compiling independent queries into a shared batched program or adding a scheduler that interleaves tokens, rather than compiling each query as a standalone executable.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a vision for executing hybrid AI queries (SQL plus LLM-based semantic operators) by compiling the entire query into a single JAX/XLA tensor program that runs on TPUs, thereby avoiding PCIe data movement between a CPU database engine and a separate ML serving system. The authors describe a translation from SQL AST to JAX constructs, report preliminary latency and throughput measurements for five queries on SemBench-based movie review data with a Gemma-2B model, and claim up to 5.34x latency and 9.87x throughput speedups, near-linear horizontal scaling on 4 TPU devices, and outline open research challenges.
Significance. If the results hold, this is a useful feasibility demonstration that relational operators and LLM inference can coexist in one compiled accelerator program, enabling automatic sharding and elimination of cross-boundary copies. The paper is honest about limitations, including high-selectivity cases and multi-tenancy serialization, and it articulates a concrete research roadmap. The main contributions are the architectural vision and the preliminary performance characterization, not a full production system. However, the quantitative claims are not yet reliable because the paper omits an output-equivalence validation and contains internally inconsistent speedup metrics.
major comments (3)
- [Section 4, Tables 3-5; Section 3.1] The paper never verifies that the unified compiled program returns the same query results as the split-execution baseline. The headline speedups are only meaningful if the fused SQL+LLM program yields exactly the same aggregates, classifications, and transformations as the baseline, but no experiment compares outputs. For Q1/Q2 the SUM depends on the binary classification from the LLM; with padding to a static maximum sequence length, the attention computation near sequence boundaries could alter logits and flip classifications. For Q5 the COUNT(*) and the LIKE '%masterpiece%' predicate depend on the generated essay text; unless decoding is seeded/argmax and length-capped identically in both executions, the two pipelines can disagree. For Q4 the cross-join with A.id = B.id AND A.review_id < B.review_id relies on row order and duplicate handling that a tensorized program may not preserve. Add a direct fused-vs-split output-equivalence check (or an explicit correctness argument) for each query.
- [Section 4.1, Table 3; Section 4.2, Table 4] The throughput speedup numbers are internally inconsistent with the latency speedups. In Table 3, Q3 reports latency speedup 0.47x (unified is slower) yet throughput speedup 4.29x. In Table 4, at Thresh=10.0 the latency speedup is 0.02x but throughput speedup is 9.87x; at Thresh=50.0 they are 1.00x and 4.24x, respectively. For a fixed workload, throughput is N/latency, so the throughput ratio should equal the inverse latency ratio of the two systems; the reported combinations would require the baseline to have both lower latency and lower throughput than the unified system, which is impossible. Clarify the metric definitions (end-to-end query latency vs per-micro-batch latency, steady-state throughput vs batch-inclusive throughput) and recompute the speedups.
- [Section 4.1] The baseline is described as an eager mixture execution model (CPU DBMS plus TPU inference), but the text also refers to a 'JIT-compiled baseline' and reports very tight standard deviations (e.g., ±0.00 ms). The paper should specify precisely what the baseline does: whether the LLM serving side uses a production server (e.g., vLLM) or a simple Python loop, whether batching is identical between baseline and unified execution, and how the CPU-side SQL execution is timed. This matters because the reported gains may conflate the unification benefit with differences in baseline engineering maturity.
minor comments (3)
- [Throughout] The manuscript contains numerous typos and residual LaTeX fragments, including 'subesquently', 'classsificaiton', 'intermedidate', 'executnion', 'databse', 'textitaudienceScore', 'qureis', 'achi-tectures', 'wokrld', 'oeprators', and 'hybird'. A careful proofreading pass is needed.
- [Section 3.3] The text says the results are 'demonstrated in Section??' — an unresolved cross-reference. Also, the figure numbering is confusing: Figure 1 is mentioned as 'High-level view' but the generated JAX program is Figure 2, and the frontend compilation flow is Figure 3; the narrative in Section 3.3 refers to 'Figure 3' for the consolidation, though the intended figure may be Figure 1.
- [Section 3.3] The phrase 'completely removes PCIe bug bottlenecks' should read 'PCIe bus bottlenecks'. In addition, the paper claims 'completely alleviate PCIe data movement bottlenecks' in the abstract, but Section 4.2 shows that the unified execution can be much slower at high selectivity; the claim should be qualified accordingly.
Circularity Check
No circularity: the speedup claims rest on measured execution times against a split-execution baseline, not on a self-referential derivation or fitted target.
full rationale
The paper is an experimental vision paper: it translates SQL AST constructs to JAX primitives (Table 1), embeds a Flax causal LM forward pass (Figure 2), and reports measured latencies/throughputs on TPUs against an eager CPU-SQL plus TPU-LLM baseline. The central claim (up to 5.3x latency and 9.8x throughput speedup) is an empirical measurement, not the output of a derivation whose inputs include that claim or whose parameters are fitted to that target. No parameter is fitted to a target result, and no prediction is obtained by renaming fitted inputs as conclusions. The only self-citations are SemBench [16] and the authors' AI-query approximation paper [5]; SemBench is used as a query/dataset workload, not as a proof or uniqueness argument, and the related-work citation [5] is not load-bearing for the compilation architecture. Section 4 explicitly scopes the evaluation to feasibility and extra-data-copy elimination, which further separates the measured speedups from any imported conclusion. The absence of a fused-vs-split output-equivalence check is a correctness/robustness risk, not a circularity step, because output equivalence is never defined in terms of, or assumed equivalent to, the measured speedups. No circular step can be exhibited from the paper's equations or construction, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- micro-batch size (bsz) =
128
- maximum padded sequence length =
not reported
- baseline configuration =
single-process CPU database engine plus separate TPU inference
assumptions (3)
- domain assumption The compiled JAX program produces the same LLM outputs as the original model, preserving query semantics.
- domain assumption The baseline split execution with CPU database and TPU inference is representative of contemporary AI query engines.
- domain assumption XLA's GSPMD/pmap auto-sharding correctly and efficiently partitions the fused program across TPU devices.
Cite this review
Pith. "Pith review of AI Query Compilation for Unified and Optimized Execution." pith.science (2026). https://pith.science/paper/YWXF5SFV
@misc{pith2026260810139,
author = {Pith},
title = {Pith review of: AI Query Compilation for Unified and Optimized Execution},
year = {2026},
howpublished = {\url{https://pith.science/paper/YWXF5SFV}},
note = {Machine review of arXiv:2608.10139}
}
read the original abstract
In this vision paper, we propose a novel architectural paradigm for accelerated AI query execution via a unified compiled execution strategy. By compiling the hybrid AI Query as a whole -- integrating both standard SQL relational constructs and LLM inference layers into a single, unified tensor compute graph -- we completely alleviate PCIe data movement bottlenecks across execution boundaries and enable global compiler optimizations and efficient automatic sharding. We demonstrate the viability of this unified execution paradigm on select and extended AI queries on SemBench Reviews and Movies datasets, achieving up to 5.3x latency speedup and 9.8x throughput speedup on TPUs, and outline a research roadmap of open technical challenges to realize this vision.
Figures
Reference graph
Works this paper leans on
-
[1]
Samuel Arch, Yuchen Liu, Todd C Mowry, Jignesh M Pate, and Andrew Pavlo
-
[2]
Matthias Boehm, Matteo Interlandi, and Chris Jermaine. 2023. Optimizing tensor computations: From applications to compilation and runtime techniques. In Companion of the 2023 International Conference on Management of Data. 53–59
work page 2023
-
[3]
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, et al. 2024. An introduction to vision-language modeling.arXiv preprint arXiv:2405.17247(2024)
arXiv 2024
-
[4]
Periklis Chrysogelos, Panagiotis Sioulas, and Anastasia Ailamaki. 2019. Hardware-conscious query processing in gpu-accelerated analytical engines. InProceesings of the 9th Biennial Conference on Innovative Data Systems Research
work page 2019
-
[5]
Yeounoh Chung, Rushabh Desai, Jian He, Yu Xiao, Thibaud Hottelier, Yves- Laurent Kom Samo, Pushkar Khadilkar, Xianshun Chen, Sam Idicula, Fatma Özcan, et al. 2026. 100x Cost & Latency Reduction: Performance Analysis of AI Query Approximation using Lightweight Proxy Models. InProceedings of the 2026 ACM SIGMOD International Conference on Management of Data...
work page 2026
-
[6]
Yeounoh Chung, Thibaud Hottelier, Cosmin Arad, Brenton Milne, Per Jacobsson, Sam Idicula, Fatma Özcan, and Alon Halevy. 2026. Architecting the AI-Powered Agentic Data Cloud.IEEE Data Engineering Bulletin49, 1 (March 2026), 22–31
work page 2026
-
[7]
Roy Frostig, Matthew James Johnson, and Chris Leary. 2019. Compiling machine learning programs via high-level tracing. InSysML conference 2018
work page 2019
-
[8]
Amir Gholami, Zhewei Yao, Sehoon Kim, Michael W Mahoney, and Kurt Keutzer
Show all 32 references
-
[9]
2026.TPU v5e
Google Cloud. 2026.TPU v5e. Google. https://docs.cloud.google.com/tpu/docs/ v5e Accessed: 2026-05-11
2026
-
[10]
Google Cloud. 2026. Vertex AI: Google Cloud’s Unified Machine Learning Platform. https://docs.cloud.google.com/vertex-ai/docs. Accessed: May 8, 2026
2026
-
[11]
Dong He, Supun Nakandala, Dalitso Banda, Rathijit Sen, Karla Saur, Kwanghyun Park, Carlo Curino, Jesús Camacho-Rodríguez, Konstantinos Karanasos, and Matteo Interlandi. 2022. Query processing on tensor computation runtimes. arXiv preprint arXiv:2203.01877(2022)
2022 arXiv
-
[12]
Pedro Holanda and Hannes Mühleisen. 2019. Relational queries with a ten- sor processing unit. InProceedings of the 15th International Workshop on Data Management on New Hardware. 1–3
2019
-
[13]
Yu-Ching Hu, Yuliang Li, and Hung-Wei Tseng. 2022. TCUDB: Accelerating data- base with tensor processors. InProceedings of the 2022 International Conference on Management of Data. 1360–1374
2022
-
[14]
Wentao Huang, Mian Lu, and Kian-Lee Tan. 2026. Hash Joins Meet CXL: A Fresh Look. InProceedings of the 2026 Conference on Innovative Data Systems Research (CIDR ’26). https://www.vldb.org/cidrdb/papers/2026/p1-huang.pdf
2026
-
[15]
Gonzalez, Hao Zhang, and Ion Stoica
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAtten- tion. InProceedings of the ACM SIGOPS 29th Symposium on Operating ...
2023
-
[16]
Jiale Lao, Andreas Zimmerer, Olga Ovcharenko, Tianji Cong, Matthew Russo, Gerardo Vitagliano, Michael Cochez, Fatma Özcan, Gautam Gupta, Thibaud Hottelier, H. V. Jagadish, Kris Kissel, Sebastian Schelter, Andreas Kipf, and Im- manuel Trummer. 2026. SemBench: A Benchmark for Se...
2026
-
[17]
Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasilache, and Olek- sandr Zinenko. 2021. MLIR: Scaling compiler infrastructure for domain specific computation. In2021 IEEE/ACM International Sympo...
2021
-
[18]
Guanghua Li, Hao Zhang, Xibo Sun, Qiong Luo, and Yuanyuan Zhu. 2024. Ten- graph: A tensor-based graph query engine.Proceedings of the VLDB Endowment 17, 13 (2024), 4571–4584
2024
-
[19]
Yinan Li, Bailu Ding, Ziyun Wei, Lukas M Maas, Momin Al-Ghosien, Spyros Blanas, Nicolas Bruno, Carlo Curino, Matteo Interlandi, Craig Peeper, et al. 2025. Scaling GPU-Accelerated Databases beyond GPU Memory Size.Proceedings of the VLDB Endowment18, 11 (2025), 4518–4531
2025
-
[20]
Paweł Liskowski, Benjamin Han, Paritosh Aggarwal, Bowei Chen, Boxin Jiang, Nitish Jindal, Zihan Li, Aaron Lin, Kyle Schmaus, Jay Tayade, et al. 2026. Cortex AISQL: A Production SQL Engine for Unstructured Data. InProceedings of the 2026 ACM SIGMOD International Conference on M...
2026
-
[21]
Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baille Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano
-
[22]
Vasilis Mageirakos, Joel André, Marko Kabić, Bowen Wu, Yannis Chronis, and Gustavo Alonso. 2026. To GPU or Not to GPU: Vector Search in Relational Engines.arXiv preprint arXiv:2605.15957(2026). arXiv:2605.15957 [cs.DB] https: //arxiv.org/abs/2605.15957
2026 arXiv
-
[23]
Liana Patel, Siddharth Jha, Melissa Pan, Harshit Gupta, Parth Asawa, Carlos Guestrin, and Matei Zaharia. 2025. Semantic Operators and Their Optimization: Enabling LLM-Based Data Processing with Accuracy Guarantees in LOTUS. Proceedings of the VLDB Endowment (PVLDB)18, 11 (2025...
2025
-
[24]
A declarative system for optimizing ai workloads.arXiv preprint arXiv:2405.14696(2024)
2024 arXiv
-
[25]
Peter Benjamin Volk, Dirk Habich, and Wolfgang Lehner. 2010. GPU-Based Speculative Query Processing for Database Operations.. InADMS@ VLDB. 51– 60
2010
-
[26]
Kaibo Wang, Kai Zhang, Yuan Yuan, Siyuan Ma, Rubao Lee, Xiaoning Ding, and Xiaodong Zhang. 2014. Concurrent Analytical Query Processing with GPUs. Proc. VLDB Endow.7, 11 (2014), 1011–1022
2014
-
[27]
Viktor Rosenfeld, Sebastian Breß, and Volker Markl. 2022. Query processing on heterogeneous CPU/GPU systems.ACM Computing Surveys (CSUR)55, 1 (2022), 1–38. 7
2022
-
[28]
Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung- Gon Chun. 2022. Orca: A distributed serving system for{Transformer-Based} generative models. In16th USENIX symposium on operating systems design and implementation (OSDI 22). 521–538
2022
-
[29]
Shuhao Zhang, Jiong He, Bingsheng He, and Mian Lu. 2013. OmniDB: Towards portable and efficient query processing on parallel CPU/GPU architectures. Proceedings of the VLDB Endowment6, 12 (2013), 1374–1377. 8
2013
-
[30]
Wulf and Sally A
Wm A. Wulf and Sally A. McKee. 1995. Hitting the Memory Wall: Implications of the Obvious.ACM SIGARCH Computer Architecture News23, 1 (1995), 20–24
1995
-
[2024]
AI and Memory Wall.IEEE Micro44, 3 (2024), 20–32
2024
-
[2026]
Partial UDF Inlining.ACM SIGMOD Record55, 1 (2026), 74–83
2026
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.