Pith. sign in

REVIEW 3 major objections 5 minor 69 references

OpenMLDB: A Real-Time Relational Data Feature Computation System for Online ML

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read OpenMLDB compiles one extended-SQL feature script into shared offline/online C++ execution, and reports 10x-20x online speedups over Flink and DuckDB via long-window pre-aggregation and self-adjusting window unions.

desk verdict Credible industrial feature computation system with real deployments, but the pre-aggregation path silently drops late tuples and the reported speedups are not uniformly reproduced by the figures. read the letter →

arxiv 2501.08591 v1 pith:WRLR3HYN submitted 2025-01-15 cs.DB cs.AIcs.LG

Xuanhe Zhou , Wei Zhou , Liguo Qi , Hao Zhang , Dihao Chen , Bingsheng He , Mian Lu , Guoliang Li
show 2 more authors
Fan Wu Yuqiang Chen
This is my paper · ORCID
classification cs.DBcs.AIcs.LG
keywords featurecomputationonlinemachinelearningoffline-onlineconsistencypre-aggregationwindowaggregationdataskewin-memorystructureSQLcompilation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

OpenMLDB is a feature-computation system built to close the gap between the two stages where ML features are made: offline batch computation for training and online real-time computation for serving. The paper argues that current practice—writing features in a batch engine, then manually re-implementing them in a streaming or database engine—creates semantic drift and deployment delays of months. OpenMLDB's answer is a unified plan generator that compiles one extended-SQL feature script into C++ machine code shared by its offline and online executors, making consistency structural rather than a verification burden. The system then targets the performance bottlenecks unique to ML features, namely long windows, multi-table window unions, and skewed partitions, with pre-aggregation, a self-adjusting union scheduler, and time-aware repartitioning. If the reported results hold, the payoff is automatic offline/online feature consistency at millisecond latency, with 10x-20x online speedups over Flink and DuckDB and roughly half the memory of in-memory key-value stores.

What carries the argument

The load-bearing mechanism is the unified plan generator plus the online long-window pre-aggregator. The plan generator compiles one extended-SQL feature script into C++ machine code shared by the offline and online executors, so offline/online consistency is produced by construction rather than checked after deployment. The pre-aggregator maintains a hierarchy of aggregators at chosen time granularities, updated asynchronously in the order enforced by a monotonic update log; a long-window request merges a few pre-aggregated levels with the unaggregated head of the stream instead of scanning raw tuples. Supporting the online engine are a refined skiplist that pre-ranks tuples by key and timestamp, a compact row encoding that stores variable-length strings by offset, and a self-adjusting window-union scheduler that rebalances key-to-thread mappings and applies subtract-and-evict increments.

What would settle it

Insert a tuple whose timestamp lies in the past after the daily or monthly pre-aggregation level covering that time has already been computed, then run a long-window feature query over that interval and compare its result with a recomputation over raw data; a mismatch would show that low latency came at the cost of feature correctness.

Watch

Extended reading notes

Core claim

The central claim is that one relational system can serve both the offline and online stages of ML feature computation without sacrificing either consistency or latency, by treating ML features as first-class SQL constructs. The unified query plan generator compiles one extended-SQL feature script—including window functions, WINDOW UNION, and LAST JOIN—into optimized machine code shared by both engines, so the feature definition used at training time is the one executed at serving time. For online requests, the decisive mechanism is multi-level pre-aggregation: long-window aggregations are rewritten into merges of precomputed aggregates, and multi-table window unions are executed with dynamic key-to-thread rebalancing plus subtract-and-evict incremental updates. For offline jobs, multi-window parallelism and data-aware repartitioning remove serial window evaluation and skew bottlenecks. The paper reports 10x-20x higher online performance than Flink and DuckDB, 6x faster offline performance than Spark and GreenPlum, and roughly half the memory usage of in-memory key-value stores.

Load-bearing premise

The online speedups rest on the assumption that updates reach the long-window pre-aggregator in timestamp order; the paper enforces this with a lock-protected monotonic update log, but does not specify how a late tuple with an old timestamp is applied after a coarser aggregate level has already advanced, so such a tuple could be silently missed.

Editorial extensions

If this is right

  • Feature deployment time drops from months to days or less, because the offline feature script is the online feature script; no separate streaming re-implementation is needed.
  • Long-window features (years of history or hotspot keys) go from hundreds of milliseconds to single-digit milliseconds by merging aggregator levels instead of rescanning raw tables.
  • Multi-table window unions keep stable throughput near 1 million tuples per second as window sizes grow, where static key-based streaming degrades toward about 1,000 tuples per second.
  • Offline feature jobs with multiple windows or skewed partitions run several times faster, reducing the number of machines needed for daily batch computation.
  • Memory savings of roughly half make it feasible to hold larger working sets in RAM, lowering hardware cost and reducing spilling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the correctness of pre-aggregation under out-of-order arrival is not settled; a minimal test is to insert an old-timestamp tuple after a coarse aggregate level has advanced and check whether a long-window query still includes it.
  • Beyond the paper: the unified-plan approach may generalize beyond SQL; the same one-definition, two-executors compilation strategy could apply to Python or DataFrame feature DSLs, with the same consistency guarantee.
  • Beyond the paper: combining long-window pre-aggregation with self-adjusting window unions should be tested jointly, since both optimize overlapping time windows and both must agree on how late tuples are reflected in precomputed aggregates.
  • Beyond the paper: if the compact encoding and skiplist indexing carry the memory savings, they could be adopted as a storage substrate for existing feature stores, decoupling retrieval from computation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. OpenMLDB is an industrial feature computation system that unifies offline and online feature extraction through a single SQL plan generator and shared C++ execution functions, and adds an online real-time executor with long-window pre-aggregation and self-adjusting multi-table window unions, plus an offline batch executor with multi-window parallelization and skew-aware repartitioning, and compact in-memory data encoding and indexing. The evaluation reports microbenchmark and production workload results against Spark, Flink, DuckDB, GreenPlum, MySQL, and Redis, claiming 10-20x online speedups, 6x offline speedups, and 40-75% memory savings, supported by deployments in 100+ scenarios. The paper's core technical pitch is that consistency is obtained architecturally, by using the same query plan and C++ libraries offline and online, rather than by reconciliation, and that performance is obtained by compilation plus specialized data structures.

Significance. If the results hold, the paper addresses a real and expensive operational gap: offline/online feature skew in ML pipelines. The strengths are concrete: the system is open source, has production deployments, and the consistency story, a shared SQL compiler and shared C++ implementations, is a credible architectural mechanism rather than an empirical coincidence. The pre-aggregation and self-adjusting window-union designs are also relevant to streaming systems beyond ML. However, the quantitative claims as currently presented are weaker than the abstract suggests: most comparisons are single-point latency and throughput numbers without repetitions or error bars, and several abstract claims do not correspond to directly reported experiments. The correctness of the fast long-window path under out-of-order event-time data is not established. With those gaps closed, this would be a solid systems contribution.

major comments (3)
  1. [§5.1 (Aggregator Update)] The pre-aggregation update protocol only guarantees a total order on binlog entries, not on event-time order. As the text states, 'aggregator updates are designed by assuming the binlog_offset increases monotonically' and are protected by the replicator lock; this is an arrival/ingestion order. Section 5.2 explicitly motivates out-of-order stream data, but Section 5.1 gives no mechanism for revising an aggregate level after its bucket has already been merged into a coarser level. A tuple with an old event timestamp that arrives late will therefore be silently absent from the materialized long-window aggregate, while an offline batch computation over the same final relation would include it. That directly undermines the central offline/online consistency claim in the long-window regime that pre-aggregation is designed to accelerate. The paper should either describe the late-data handling for pre-aggregated windows, for example compensation, invalidation, or lazy re-materialization, or explicitly restrict pre-aggregation to ordered event-time streams and explain why the offline/online consistency guarantee still covers that restriction.
  2. [Section 9 and Abstract] The abstract's headline numbers are not all directly supported by the reported experiments. The claimed '10x-20x higher online performance than Flink and DuckDB' is not shown as a Flink-versus-DuckDB comparison in the online microbenchmark; Figure 6 compares OpenMLDB with MySQL(in-mem), DuckDB, and Trino+Redis, while Flink appears only in the RTP workload (Figure 7) and in Section 9.3.2 as a throughput discussion without reported measurements. Likewise, '6x faster offline performance than Spark and MPP databases like GreenPlum' is supported only by the 6.3x Spark multi-window number in Figure 8; no GreenPlum offline experiment appears. In addition, all latency figures (Figures 6-17, Table 3) are single-point measurements with no repetitions, error bars, or significance testing, making it impossible to assess variance or whether the improvements are robust. The evaluation should be aligned with the claims and should include repeated runs with variance or confidence information.
  3. [§5.2 (Incremental Computation)] The Subtract-and-Evict mechanism in Section 5.2 inherits the same ordering assumption as Section 5.1 but is presented as supporting out-of-order data. The description says outdated tuples are subtracted as they leave the window and new tuples are added incrementally; this is only correct if tuples are evicted and inserted in timestamp order. The paper cites [66] for load balancing and [58] for Subtract-and-Evict, but it does not state how a late tuple whose timestamp falls inside an already-evicted or already-merged interval is handled. Without that, the self-adjusting window union's high throughput in Section 9.3.2 may come at the cost of omitted tuples, and the claimed 'out-of-order stream data' support in Section 3.1 is not demonstrated. Please specify the event-time semantics precisely, including any watermarking or buffering, for both the pre-aggregation path and the window-union path.
minor comments (5)
  1. [References] References [65] and [66] are the same paper with identical title and venue; one duplicate should be removed.
  2. [Contributions list, Section 1] The contribution bullet 'reduces feature deployment time from months to days or even less [?]' contains an unresolved placeholder; please either remove it or replace it with measured numbers.
  3. [Figures 10, 14-17] Several evaluation figures, especially Figure 10 and Figures 14-17, lack readable axis labels and units in the provided text; adding them is necessary for the results to be interpretable.
  4. [Section 8.1, Eq. (1)] The memory estimation model defines C by table type and says K varies between 1 and n_index, but it gives no calibration procedure or validation against measured memory usage, so the example calculation (1.568 GB) cannot be reproduced or checked; please provide calibration guidance or measured validation.
  5. [Section 9.3.1] The long-window pre-aggregation experiment reports a 45x latency reduction but only says the optimization introduces a 'slightly higher data loading overhead' without quantifying it; please report the loading overhead so readers can assess the trade-off.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: performance claims are measured, consistency is an architectural construction, and the paper's self-citations are not load-bearing.

full rationale

No derivational circularity found. The paper's central consistency claim is architectural rather than empirical: Sections 3.1 and 4.2 state that offline and online execution share one query plan generator and the same compiled C++ library functions, so consistency is a construction (same code produces same results), not a fitted prediction; this is a design guarantee, not a circular derivation. Performance claims in Section 9 are measurements against external baselines (Flink, DuckDB, Spark, GreenPlum, Trino+Redis) on MicroBench, TalkingData, and Akulaku workloads, with ablations (Figures 10-13) that turn the proposed optimizations on and off; no parameter is fitted to a target result and then reported as a prediction. The self-citations ([65]/[66] for interval-join/load-balancing motivation, [67] FEBench, and [16] for the memory-estimation blog) are present but not load-bearing: the dynamic scheduler and pre-aggregation are evaluated in this paper's own experiments rather than resting solely on the cited prior work. The one substantive weakness is a correctness gap, not circularity: Section 5.1's 'binlog_offset increases monotonically' assumption orders ingestion rather than event time, and Section 5.2's Subtract-and-Evict incremental updates do not describe how late event-time tuples are retroactively reflected in already-merged pre-aggregation levels; if such tuples occur, long-window online features could diverge from offline recomputation, undermining the consistency claim. That is a semantic/robustness risk that should be weighed separately and does not make the derivation circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central performance claims rest on standard engineering assumptions rather than new mathematical objects. The main free parameters are configuration knobs, memory constants and pre-aggregation granularity, that are hand-set or workload-specific. No new physical or formal entities are introduced.

free parameters (3)
  • C (memory estimation coefficient) = 70 or 74 depending on table type
    Introduced in Section 8.1 to estimate per-row memory cost in the empirical memory model. The value is hand-chosen by table category, not learned, and affects the memory estimation guidance rather than the core latency claims.
  • K (number of data copies in memory model) = between 1 and n_index
    User-configured parameter in Section 8.1's memory estimation formula, representing how many index copies a table maintains. It is an input to the model, not a fitted result.
  • Pre-aggregation hierarchy granularity = e.g., 'w1:1d' in OPTIONS
    The 45x speedup in Section 9.3.1 is obtained only after the user manually declares long_windows='w1:1d'. This workload-specific knob is part of the system's configuration, not an automatically discovered setting.
assumptions (4)
  • domain assumption Online ML feature computation is dominated by relational window aggregates, window unions, and last joins over time-series data.
    Motivates the entire design in Sections 1 and 2.1. If general-purpose systems already handled these patterns efficiently, the performance motivation would be much weaker.
  • domain assumption Using a unified SQL plan generator and shared C++ execution functions guarantees semantic consistency between offline and online results.
    The consistency claim in Sections 3.1 and 4 rests on this architectural assumption. The paper does not provide a formal semantics proof or a differential test on all supported operations.
  • domain assumption The binlog_offset in the pre-aggregation pipeline is monotonically increasing, and replicator locks preserve this invariant.
    Stated in Section 5.1 (Aggregator Update). This is load-bearing for correctness of pre-aggregated long-window results; late or out-of-order tuples that violate the ordering would not be reflected correctly.
  • domain assumption Benchmark results on four 512GB servers and on the Akulaku/SF production workloads generalize to the 100+ claimed deployment scenarios.
    The evaluation in Section 9.1 uses one hardware configuration for microbenchmarks and one set of production workloads; generalization to the claimed breadth of deployments is assumed, not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenMLDB: A Real-Time Relational Data Feature Computation System for Online ML." pith.science (2026). https://pith.science/paper/WRLR3HYN

@misc{pith2026250108591,
  author       = {Pith},
  title        = {Pith review of: OpenMLDB: A Real-Time Relational Data Feature Computation System for Online ML},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WRLR3HYN}},
  note         = {Machine review of arXiv:2501.08591}
}
read the original abstract

Efficient and consistent feature computation is crucial for a wide range of online ML applications. Typically, feature computation is divided into two distinct phases, i.e., offline stage for model training and online stage for model serving. These phases often rely on execution engines with different interface languages and function implementations, causing significant inconsistencies. Moreover, many online ML features involve complex time-series computations (e.g., functions over varied-length table windows) that differ from standard streaming and analytical queries. Existing data processing systems (e.g., Spark, Flink, DuckDB) often incur multi-second latencies for these computations, making them unsuitable for real-time online ML applications that demand timely feature updates. This paper presents OpenMLDB, a feature computation system deployed in 4Paradigm's SageOne platform and over 100 real scenarios. Technically, OpenMLDB first employs a unified query plan generator for consistent computation results across the offline and online stages, significantly reducing feature deployment overhead. Second, OpenMLDB provides an online execution engine that resolves performance bottlenecks caused by long window computations (via pre-aggregation) and multi-table window unions (via data self-adjusting). It also provides a high-performance offline execution engine with window parallel optimization and time-aware data skew resolving. Third, OpenMLDB features a compact data format and stream-focused indexing to maximize memory usage and accelerate data access. Evaluations in testing and real workloads reveal significant performance improvements and resource savings compared to the baseline systems. The open community of OpenMLDB now has over 150 contributors and gained 1.6k stars on GitHub.

Figures

Figures reproduced from arXiv: 2501.08591 by the authors.

Figure 1
Figure 1. Relational Data Feature Computation. top-N frequency counts) and feature signatures (e.g., export format labeling for high-dimension features like product items [47]), as well as computation patterns (e.g., multi-table windows) distinct from those in standard streaming and analytical queries [65, 67]. Example. To illustrate the complexity, consider the product recom￾mendation scenario that generates personalized adv… view at source ↗
Figure 2
Figure 2. Architecture of OpenMLDB. and windowed aggregations over relative time intervals. These limitations often force users to pre-process features using separate online storage systems and to re-synchronize whenever new fea￾tures are introduced, resulting in operational overhead and delayed feature availability for online ML applications. 3 Overview of OpenMLDB 3.1 System Architecture As shown in [PITH_FULL_IMAGE:figure… view at source ↗
Figure 3
Figure 3. Workflow of Different Execution Modes. and (2) resolves the data skew problem by dynamically re-assigning window data based on the key columns and data distribution. Compact Time-Series Data Management. Compared with ex￾isting systems like Spark and Redis, OpenMLDB supports a more compact data encoding format, which saves unnecessary memory consumption and enhances performance by allowing more data to be processed w… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Long Window Pre-Aggregation. illustrated in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: In-Memory Data Encoding Format. tuples to distinguish them from the original data. (3) Union of Data: We merge the expanded data tuples with the original partition data, ensuring each partition has all the data for accurate results. Redistribute Data by New Identifiers…
Figure 6
Figure 6. Figure 6: Online MicroBench Performance Comparison. N=1 N=2 N=4 N=8 0.2 2.0 20.0 200.0 2000.0 Latency (ms) GreenPlum Flink OpenMLDB 14.35 40.53 140.76 350.27 9.33 20.04 38.62 68.89 0.98 1.34 1.99 4.03 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: RTP Performance Comparison. hosts the client application using the OpenMLDB Java SDK for test￾ing. For other workloads, experiments are conducted on 16 servers of similar configurations. 9.2 Overall Performance 9.2.1 Online Feature Computation MicroBench Performance. W…
Figure 8
Figure 8. Figure 8: Offline MicroBench Performance Comparison. N=7 N=8 N=9 N=10 1 10 100 1000 10000 Latency (ms) Spark OpenMLDB 140.0 243.0 358.0 999.0 27.0 29.0 35.0 45.0 [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Offline GLQ Performance Comparison. processes more data in-memory, enabling more scalable and lower￾latency feature computations than baseline systems that must either add more hardware or spill to disk. 9.2.2 Offline Feature Computation MicroBench Performance. As show…
Figure 10
Figure 10. Figure 10: Pre-Aggregation Performance. DEPLOY test1 OPTIONS(long_windows="w1:1d") SELECT itemId, count(itemId) OVER w1 AS count_1, actionTime FROM actions WINDOW w1 AS ( PARTITION BY itemId ORDER BY actionTime rows_range BETWEEN 100d preceding AND current row ) (a) Long Window …
Figure 11
Figure 11. Figure 11: Performance of Long-Window Optimization: (a) [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 14
Figure 14. Figure 14: Performance under Different Threads. (a) Latency (b) Throughput [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 17
Figure 17. Figure 17: Performance under Different Join Numbers. [PITH_FULL_IMAGE:figures/full_fig_p012_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 59 canonical work pages

  1. [66]

    Hao Zhang, Xianzhi Zeng, Shuhao Zhang, Xinyi Liu, Mian Lu, and Zhao Zheng

    work page

  2. [58]

    Kanat Tangwongsan, Martin Hirzel, and Scott Schneider. 2017. Low-Latency Sliding-Window Aggregation in Worst-Case Constant Time (DEBS ’17). Associa- tion for Computing Machinery, New York, NY, USA, 66–77. https://doi.org/10. 1145/3093742.3093925

    work page arXiv 2017

  3. [1]

    https://cassandra.apache.org

    [n.d.]. https://cassandra.apache.org. Last accessed on 2024-12

    work page 2024

  4. [2]

    https://doris.apache.org

    [n.d.]. https://doris.apache.org. Last accessed on 2024-12

    work page 2024

  5. [3]

    https://druid.apache.org

    [n.d.]. https://druid.apache.org. Last accessed on 2024-12

    work page 2024

  6. [4]

    https://github.com/4paradigm/OpenMLDB/tree/main/benchmark

    [n.d.]. https://github.com/4paradigm/OpenMLDB/tree/main/benchmark. Last accessed on 2024-12

    work page 2024

  7. [5]

    https://github.com/feast-dev/feast

    [n.d.]. https://github.com/feast-dev/feast. Last accessed on 2024-11

    work page 2024

  8. [6]

    https://github.com/feathr-ai/feathr

    [n.d.]. https://github.com/feathr-ai/feathr. Last accessed on 2024-11

    work page 2024

Show all 69 references
  1. [7]

    https://github.com/logicalclocks/hopsworks

    [n.d.]. https://github.com/logicalclocks/hopsworks. Last accessed on 2024-11

    2024

  2. [8]

    https://github.com/redis/redis

    [n.d.]. https://github.com/redis/redis. Last accessed on 2024-11

    2024

  3. [9]

    https://greenplum.org

    [n.d.]. https://greenplum.org. Last accessed on 2024-12

    2024

  4. [10]

    https://hbase.apache.org

    [n.d.]. https://hbase.apache.org. Last accessed on 2024-12

    2024

  5. [11]

    https://mariadb.org

    [n.d.]. https://mariadb.org. Last accessed on 2024-12

    2024

  6. [12]

    https://medium.com/engineering-varo/feature-store-challenges-and- considerations-d1d59c070634

    [n.d.]. https://medium.com/engineering-varo/feature-store-challenges-and- considerations-d1d59c070634. Last accessed on 2024-12

    2024

  7. [13]

    https://nvidia.github.io/spark-rapids

    [n.d.]. https://nvidia.github.io/spark-rapids. Last accessed on 2024-12

    2024

  8. [14]

    https://oap-project.github.io/latest

    [n.d.]. https://oap-project.github.io/latest. Last accessed on 2024-11

    2024

  9. [15]

    https://openmldb.ai/en/akulaku-real-time-feature-extraction-for-ai- powered-risk-control

    [n.d.]. https://openmldb.ai/en/akulaku-real-time-feature-extraction-for-ai- powered-risk-control. Last accessed on 2024-12

    2024

  10. [16]

    https://openmldb.ai/en/blog/

    [n.d.]. https://openmldb.ai/en/blog/. Last accessed on 2024-12

    2024

  11. [17]

    https://openmldb.ai/en/the-commercial-bank-driving-the-ai-based-anti- fraud-with-efficient-feature-extraction/

    [n.d.]. https://openmldb.ai/en/the-commercial-bank-driving-the-ai-based-anti- fraud-with-efficient-feature-extraction/. Last accessed on 2024-12

    2024

  12. [18]

    https://opentsdb.net

    [n.d.]. https://opentsdb.net. Last accessed on 2024-12

    2024

  13. [19]

    https://tdengine.com

    [n.d.]. https://tdengine.com. Last accessed on 2024-12

    2024

  14. [20]

    https://www.alibabacloud.com/product/emapreduce

    [n.d.]. https://www.alibabacloud.com/product/emapreduce. Last accessed on 2024-12

    2024

  15. [21]

    https://www.databricks.com/product/photon

    [n.d.]. https://www.databricks.com/product/photon. Last accessed on 2024-11

    2024

  16. [22]

    https://www.influxdata.com

    [n.d.]. https://www.influxdata.com. Last accessed on 2024-12

    2024

  17. [23]

    https://www.kaggle.com/competitions/talkingdata-adtracking-fraud- detection

    [n.d.]. https://www.kaggle.com/competitions/talkingdata-adtracking-fraud- detection. Last accessed on 2024-12

    2024

  18. [24]

    https://www.mongodb.com

    [n.d.]. https://www.mongodb.com. Last accessed on 2024-12

    2024

  19. [25]

    https://www.mysql.com

    [n.d.]. https://www.mysql.com. Last accessed on 2024-12

    2024

  20. [26]

    https://www.postgresql.org

    [n.d.]. https://www.postgresql.org. Last accessed on 2024-12

    2024

  21. [27]

    https://www.starrocks.io

    [n.d.]. https://www.starrocks.io. Last accessed on 2024-12

    2024

  22. [28]

    https://www.tecton.ai

    [n.d.]. https://www.tecton.ai. Last accessed on 2024-12

    2024

  23. [29]

    Forecast: The Business Value of Artificial Intelligence

    2018. Forecast: The Business Value of Artificial Intelligence. In Gartner. https: //www.gartner.com/en/documents/3868267

    2018

  24. [30]

    Abdiansah Abdiansah and Retantyo Wardoyo. 2015. Time complexity analysis of support vector machines (SVM) in LibSVM. Int. J. Comput. Appl 128, 3 (2015), 28–34

    2015

  25. [31]

    Adrian-Ioan Argesanu and Gheorghe-Daniel Andreescu. 2021. Accelerating Data Ingress for Range-Scan Optimized HBase Instances. In 15th IEEE International Symposium on Applied Computational Intelligence and Informatics, SACI 2021, Timisoara, Romania, May 19-21, 2021 . IEEE, 341–...

    2021

  26. [32]

    Michael Armbrust, Reynold S Xin, Cheng Lian, Yin Huai, Davies Liu, Joseph K Bradley, Xiangrui Meng, Tomer Kaftan, Michael J Franklin, Ali Ghodsi, et al. 2015. Spark sql: Relational data processing in spark. In Proceedings of the 2015 ACM SIGMOD international conference on mana...

    2015

  27. [33]

    Tiemo Bang, Ismail Oukid, Norman May, Ilia Petrov, and Carsten Binnig. 2020. Robust Performance of Main Memory Data Structures by Configuration. In Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, ...

    2020

  28. [34]

    Edmon Begoli, Jesús Camacho-Rodríguez, Julian Hyde, Michael J Mior, and Daniel Lemire. 2018. Apache calcite: A foundational framework for optimized query processing over heterogeneous data sources. In Proceedings of the 2018 International Conference on Management of Data . 221–230

    2018

  29. [35]

    Ionut Boicu. 2019. Adaptive on-the-fly compressed execution in spark . Ph.D. Dissertation. Master’s Thesis. Vrije Universiteit Amsterdam

    2019

  30. [36]

    Jie Cai, Jiawei Luo, Shulin Wang, and Sheng Yang. 2018. Feature selection in machine learning: A new perspective. Neurocomputing 300 (2018), 70–79. https://doi.org/10.1016/J.NEUCOM.2017.11.077

    2018 doi

  31. [37]

    Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache Flink ™: Stream and Batch Processing in a Single Engine. IEEE Data Eng. Bull. 38, 4 (2015), 28–38. http://sites.computer. org/debull/A15dec/p28.pdf

    2015

  32. [38]

    Aldana-Martín, Antonio M

    Cristian Cardas, José F. Aldana-Martín, Antonio M. Burgueño-Romero, Antonio J. Nebro, Jose Manuel Mateos, and Juan Jesús Sánchez Sánchez. 2023. On the performance of SQL scalable systems on Kubernetes: a comparative study. Clust. Comput. 26, 3 (2023), 1935–1947. https://doi.or...

    2023 doi

  33. [39]

    Mark de Berg, Marc van Kreveld, Mark Overmars, Otfried Cheong Schwarzkopf, Mark de Berg, Marc van Kreveld, Mark Overmars, and Otfried Cheong Schwarzkopf. 2000. More geometric data structures: Windowing. Computa- tional Geometry: algorithms and applications (2000), 211–233

    2000

  34. [40]

    Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM 51, 1 (2008), 107–113

    2008

  35. [41]

    Philippe Flajolet, Éric Fusy, Olivier Gandouet, and Frédéric Meunier. 2007. Hyper- loglog: the analysis of a near-optimal cardinality estimation algorithm. Discrete mathematics & theoretical computer science Proceedings (2007)

    2007

  36. [42]

    Gartner. [n.d.]. The Logical Feature Store: Data Management for Machine Learn- ing. Last accessed on 2024-02

    2024

  37. [43]

    Michal Gregor and Juraj Spalek. 2017. Using LLVM-based JIT Compilation in Genetic Programming. CoRR abs/1701.05730 (2017). arXiv:1701.05730 http: //arxiv.org/abs/1701.05730

    2017 arXiv

  38. [44]

    Dongxu Huang, Qi Liu, Qiu Cui, Zhuhe Fang, Xiaoyu Ma, Fei Xu, Li Shen, Liu Tang, Yuxing Zhou, Menglong Huang, et al . 2020. TiDB: a Raft-based HTAP database. Proceedings of the VLDB Endowment 13, 12 (2020), 3072–3084

    2020

  39. [45]

    Patrick Hunt, Mahadev Konar, Flavio Paiva Junqueira, and Benjamin C. Reed. 2010. ZooKeeper: Wait-free Coordination for Internet-scale Systems. In Proceedings of the 2010 USENIX Annual Technical Conference, USENIX ATC 2010, Boston, MA, USA, June 23-25, 2010. USENIX Association....

    2010

  40. [46]

    Hongjun Lu, Yuet Yeung Ng, and Zengping Tian. 2000. T-Tree or B-Tree: Main Memory Database Index Structure Revisited. InAustralasian Database Conference, ADC 2000, Canberra, Australia, January 31 - February 3, 2000 . IEEE Computer Society, 65–73. https://doi.org/10.1109/ADC.20...

    2000

  41. [47]

    Lyons and Andrew D

    Terry J. Lyons and Andrew D. McLeod. 2022. Signature Methods in Machine Learning. CoRR abs/2206.14674 (2022). https://doi.org/10.48550/ARXIV.2206. 14674 arXiv:2206.14674

    2022 doi arXiv

  42. [48]

    Miao Ma, Zhengyi Yang, Kongzhang Hao, Liuyi Chen, Chunling Wang, and Yi Jin

  43. [49]

    Jim Melton. 1996. SQL Language Summary. ACM Comput. Surv. 28, 1 (1996), 141–143. https://doi.org/10.1145/234313.234374

    1996

  44. [50]

    Vaivaswatha Nagaraj, Jacob Johannsen, Anton Trunov, George Pîrlea, Amrit Kumar, and Ilya Sergey. 2020. Compiling a Higher-Order Smart Contract Language to LLVM. CoRR abs/2008.05555 (2020). arXiv:2008.05555 https: //arxiv.org/abs/2008.05555

    2020 arXiv

  45. [51]

    William W. Pugh. 1990. Skip Lists: A Probabilistic Alternative to Balanced Trees. Commun. ACM 33, 6 (1990), 668–676. https://doi.org/10.1145/78973.78977

    1990

  46. [52]

    Mark Raasveldt and Hannes Mühleisen. 2019. Duckdb: an embeddable analytical database. In Proceedings of the 2019 International Conference on Management of Data. 1981–1984

    2019

  47. [53]

    Salman Salloum, Ruslan Dautov, Xiaojun Chen, Patrick Xiaogang Peng, and Joshua Zhexue Huang. 2016. Big data analytics on Apache Spark. International Journal of Data Science and Analytics 1 (2016), 145–164

    2016

  48. [54]

    Robert Schulze, Tom Schreiber, Ilya Yatsishin, Ryadh Dahimene, and Alexey Milovidov. 2024. ClickHouse-Lightning Fast Analytics for Everyone. Proceedings of the VLDB Endowment 17, 12 (2024), 3731–3744

    2024

  49. [55]

    Swaminathan Sivasubramanian. 2012. Amazon dynamoDB: a seamlessly scal- able non-relational database service. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data . 729–730

    2012

  50. [56]

    Ji Sun and Guoliang Li. 2019. An End-to-End Learning-based Cost Estimator.Proc. VLDB Endow. 13, 3 (2019), 307–319. https://doi.org/10.14778/3368289.3368296

    2019

  51. [57]

    Rebecca Taft, Irfan Sharif, Andrei Matei, Nathan VanBenschoten, Jordan Lewis, Tobias Grieger, Kai Niemi, Andy Woods, Anne Birzin, Raphael Poss, et al. 2020. Cockroachdb: The resilient geo-distributed sql database. In Proceedings of the 2020 ACM SIGMOD international conference ...

    2020

  52. [59]

    Ashish Thusoo, Joydeep Sen Sarma, Namit Jain, Zheng Shao, Prasad Chakka, Suresh Anthony, Hao Liu, Pete Wyckoff, and Raghotham Murthy. 2009. Hive: a warehousing solution over a map-reduce framework. Proceedings of the VLDB Endowment 2, 2 (2009), 1626–1629

    2009

  53. [60]

    Jiayi Wang, Chengliang Chai, Jiabin Liu, and Guoliang Li. 2021. FACE: A Normalizing Flow based Cardinality Estimator. VLDB 15, 1 (2021), 72–84. http://www.vldb.org/pvldb/vol15/p72-li.pdf

    2021

  54. [61]

    Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. 2009. Feature hashing for large scale multitask learning. InProceedings of the 26th annual international conference on machine learning . 1113–1120

    2009

  55. [62]

    Zhenkun Yang, Chuanhui Yang, Fusheng Han, Mingqiang Zhuang, Bing Yang, Zhifeng Yang, Xiaojun Cheng, Yuzhong Zhao, Wenhui Shi, Huafeng Xi, et al

  56. [63]

    Matei Zaharia, Reynold S Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J Franklin, et al. 2016. Apache spark: a unified engine for big data processing. Commun. ACM 59, 11 (2016), 56–65. 13

    2016

  57. [64]

    Eftim Zdravevski, Petre Lameski, and Andrea Kulakov. 2016. Row Key Designs of NoSQL Database Tables and Their Impact on Write Performance. In 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2016, Heraklion, Crete, Greece, Feb...

    2016 doi

  58. [67]

    Xuanhe Zhou, Cheng Chen, Kunyi Li, Bingsheng He, Mian Lu, Qiaosheng Liu, Wei Huang, Guoliang Li, Zhao Zheng, and Yuqiang Chen. 2023. FEBench: A Benchmark for Real-Time Relational Data Feature Extraction. Proc. VLDB Endow. 16, 12 (2023), 3597–3609. https://doi.org/10.14778/3611...

    2023

  59. [70]

    In 2023 IEEE 39th International Conference on Data Engineering (ICDE)

    Scalable Online Interval Join on Modern Multicore Processors in Open- MLDB. In 2023 IEEE 39th International Conference on Data Engineering (ICDE) . 3031–3042. https://doi.org/10.1109/ICDE55515.2023.00232

    2023

  60. [2022]

    Proceedings of the VLDB Endowment 15, 12 (2022), 3385–3397

    OceanBase: a 707 million tpmC distributed relational database system. Proceedings of the VLDB Endowment 15, 12 (2022), 3385–3397

    2022

  61. [2023]

    An Empirical Analysis of Just-in-Time Compilation in Modern Databases. In Databases Theory and Applications - 34th Australasian Database Conference, ADC 2023, Melbourne, VIC, Australia, November 1-3, 2023, Proceedings (Lecture Notes in Computer Science) , Zhifeng Bao, Renata B...

    2023 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.