Semantic Operators: A Declarative Model for Rich, AI-based Data Processing
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:OUV7UOKTrecord.jsonopen to challenge →
read the original abstract
The semantic capabilities of large language models (LLMs) have the potential to enable rich analytics and reasoning over vast knowledge corpora. Unfortunately, existing systems either empirically optimize expensive LLM-powered operations with no performance guarantees, or serve a limited set of row-wise LLM operations, providing limited robustness, expressiveness and usability. We introduce semantic operators, the first formalism for declarative and general-purpose AI-based transformations based on natural language specifications (e.g., filtering, sorting, joining or aggregating records using natural language criteria). Each operator opens a rich space for execution plans, similar to relational operators. Our model specifies the expected behavior of each operator with a high-quality gold algorithm, and we develop an optimization framework that reduces cost, while providing accuracy guarantees with respect to a gold algorithm. Using this approach, we propose several novel optimizations to accelerate semantic filtering, joining, group-by and top-k operations by up to $1,000\times$. We implement semantic operators in the LOTUS system and demonstrate LOTUS' effectiveness on real, bulk-semantic processing applications, including fact-checking, biomedical multi-label classification, search, and topic analysis. We show that the semantic operator model is expressive, capturing state-of-the-art AI pipelines in a few operator calls, and making it easy to express new pipelines that match or exceed quality of recent LLM-based analytic systems by up to $170\%$, while offering accuracy guarantees. Overall, LOTUS programs match or exceed the accuracy of state-of-the-art AI pipelines for each task while running up to $3.6\times$ faster than the highest-quality baselines. LOTUS is publicly available at https://github.com/lotus-data/lotus.
This paper has not been read by Pith yet.
Forward citations
Cited by 17 Pith papers
-
SemCEB: A Cardinality Estimation Benchmark for Semantic Operators
SemCEB is the first benchmark for cardinality estimation over semantic operators, evaluating sampling methods and Semantic Histograms on accuracy, cost, latency, and memory using 102 queries on a real-world dataset.
-
Cost-Aware Optimization for Agentic Query Execution
EnumGRPO is a self-improving optimizer for agentic query execution that reduces LLM-operator costs by ~317x while improving accuracy by 18% over a hybrid baseline across four databases.
-
AnnoRetrieve: Efficient Structured Retrieval for Unstructured Document Analysis
AnnoRetrieve uses auto-generated structured schemas and queries to retrieve information from unstructured documents more efficiently and accurately than embedding-based methods.
-
Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows
Spider 2.0-AIFunc is a 465-instance benchmark for evaluating text-to-SQL systems on queries that incorporate Snowflake Cortex AI functions, with evaluations of ten models showing proprietary models reach 67-70% accuracy.
-
SemJoin: Semantic Join Optimization
SemJoin dynamically routes semantic joins via an LLM advisor to Cluster Join or Classifier strategies, achieving 20-33 higher F1 than ABJ and lower token cost than FDJ on three datasets.
-
Larch: Learned Query Optimization for Semantic Predicates
Larch uses a GNN-MDP formulation and a selectivity predictor plus dynamic programming to reorder semantic filter evaluation, cutting token usage 3x-19x versus prior systems on real and synthetic workloads.
-
MLSkip: Data Skipping for ML Filters via Lightweight Metadata
MLSkip demonstrates that lightweight metadata enables data skipping for ReLU-based ML filters, with 27.4% average pruning using min-max and 38.31% using 2D convex hulls on TPC benchmarks, for a 1.07x end-to-end speedup.
-
Natural Language to What? A Vision for Intermediate Representations in NL-to-X Querying
NLIQ frames natural language querying around target adequacy to classify when the output representation must be constructed, treating intermediate representations as first-class semantic objects rather than mere imple...
-
Semantic Data Processing with Holistic Data Understanding
HoldUp uses LLM-guided clustering to provide holistic dataset context for semantic operators, yielding up to 33% higher classification accuracy and 30% higher scoring accuracy than row-by-row LLM processing across 15 ...
-
Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
A unified framework for LLM agent memory is benchmarked, with a new hybrid method outperforming state-of-the-art on standard tasks.
-
iPDB -- Optimizing Semantic SQL Queries
iPDB adds a predict operator and semantic query optimizations to SQL so that LLM and ML calls run efficiently inside the database, delivering 2.5x average and up to 30x speedup over prior systems.
-
ScaleDoc: Scaling LLM-based Predicates over Large Document Collections
ScaleDoc achieves over 2x end-to-end speedup and up to 85% fewer LLM invocations for semantic predicates on large document collections via offline LLM representations, contrastive-trained proxy filtering, and adaptive...
-
In-depth Analysis of Graph-based RAG in a Unified Framework
A unified framework and large-scale comparison of graph-based RAG methods on QA tasks yields new high-performing variants obtained by recombining existing components.
-
Blue Data Intelligence Layer: Streaming Data and Agents for Multi-source Multi-modal Data-Centric Applications
Blue DIL is a new architecture that unifies structured enterprise data, LLM world knowledge, and personal context through declarative query plans and agents for multi-source multi-modal applications.
-
Making Prompts First-Class Citizens for Adaptive LLM Pipelines
SPEAR proposes structured prompt views, runtime adaptive refinement, and policy rules to make prompts first-class, versioned, and evolvable components in complex LLM applications.
-
Large Databases Need Small, Open-Weight Language Models
Quantized open-weight LMs on consumer hardware match closed-source API accuracy for LM-enhanced relational operators while delivering 390x lower cost and 3.8x lower latency in the BlendSQL framework.
-
Cardinality Estimation for High Dimensional Similarity Queries with Adaptive Bucket Probing
An LSH-based system with adaptive bucket probing, progressive sampling, and product quantization estimates cardinality for high-dimensional similarity queries efficiently.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.