Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Parquet's GPU slowness is a configuration problem, not a format flaw.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 22:14 UTC pith:DIU54RAK

load-bearing objection A useful, honest config-tuning study for GPU Parquet scans; the central claim holds on the tested stack, but the page-count mechanism is under-supported and generality is untested. the 3 major comments →

arxiv 2602.17335 v3 pith:DIU54RAK submitted 2026-02-19 cs.DB cs.DC

Do GPUs Really Need New Tabular File Formats?

Jigao Luo , Qi Chen , Carsten Binnig This is my paper
classification cs.DB cs.DC
keywords ParquetGPU data processingcolumnar file formatsscan performancefile configurationdata compressionGPUDirect StorageTPC-H
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that GPU scans of Parquet files are slow not because of the format's design but because Parquet's standard configuration defaults were tuned for CPUs. On an NVIDIA A100 reading TPC-H data from SSDs via direct GPU storage, the authors show that four configuration changes—raising page count to at least 100, enlarging row groups to millions of rows, choosing the smallest per-chunk encoding, and skipping compression when it shrinks data less than 10%—raise effective read bandwidth to 125 GB/s. All changes stay within the existing Parquet specification, so no new file format is required. The claim matters because GPU-accelerated analytics systems currently spend most of their time in Parquet scans, and the fix is a one-time file rewrite rather than an ecosystem-wide format migration.

Core claim

The paper's central claim is that Parquet's poor GPU performance is not inherent to the format but a consequence of suboptimal configuration choices made by CPU-oriented writers. Specifically, it identifies four configuration insights: (1) page count should be 100 or above so the GPU reader's decoding kernel maps pages to the GPU grid and fully utilizes the device; (2) row-group sizes of millions of rows produce column chunks large enough for direct storage reads to saturate SSD bandwidth; (3) allowing each column chunk to pick among Parquet V1 and V2 encodings and keeping the smallest result improves effective bandwidth; and (4) skipping compression when it reduces chunk size by less than 1

What carries the argument

The central mechanism is the configuration knobs of the Apache Parquet format themselves, reinterpreted for GPU execution. The paper maps each knob to a GPU-engine behavior: page count becomes the GPU kernel grid size in the reader's decoder; row-group size determines column-chunk I/O size for direct storage reads; per-chunk encoding flexibility leverages local data distribution to minimize encoded size; and compression selectivity avoids compute-bound decompression. The accompanying Parquet rewriter tool applies these four rules as an offline preprocessing step, transforming existing CPU-default files into GPU-tuned ones.

Load-bearing premise

The whole result rests on the premise that the GPU Parquet reader really maps page count to GPU kernel grid size; if a reader parallelizes differently, the page-count rule and its measured bandwidth gains do not transfer.

What would settle it

Measure a GPU Parquet scan with page counts below 100 on a different GPU Parquet reader that does not map pages to the kernel grid; if bandwidth does not drop as the paper's Figure 2(a) shows, the page-count insight is engine-specific, not a property of the format configuration. Alternatively, run the same four-configuration rewrite on a second table and on a different GPU architecture and check whether 125 GB/s still holds.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • GPU-accelerated databases can get up to 125 GB/s effective scan bandwidth by rewriting Parquet files with these four rules, without waiting for a new file format.
  • This gives a strong, realistic baseline against which any proposed GPU-native format must be measured—the new format must beat an already-optimized Parquet read from SSDs.
  • The rewriting is a one-time offline step that typically completes within minutes for 100 GB datasets, and since files often shrink, storage overhead need not increase.
  • The two I/O-related insights (page count, row-group size) are specific to GPU stacks, while the encoding and compression insights also apply to CPU systems and are already being discussed in the wider Parquet ecosystem.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • It follows, though the paper does not test it, that the 125 GB/s figure likely varies by GPU architecture and reader implementation: the page-count rule is tied to the specific reader's current kernel-launch mapping, so other GPU Parquet readers or future versions may need different page counts.
  • Treating the TPC-H lineitem table as representative is a bet; a wider evaluation across different schemas and data distributions would test whether the four rules generalize.
  • A testable extension is to benchmark the rewriter's output against a GPU-native format on the same SSD+A100 stack; the paper's claim would be strengthened if optimized Parquet remains competitive.
  • The compression threshold of 10% and page-count threshold of 100 are presented as reasonable choices, not optima; a search over thresholds could find small additional gains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates whether Parquet's poor GPU scan performance is inherent to the format or caused by CPU-oriented configuration defaults. Using PystachIO (the authors' GPU query engine on cuDF), an A100, and TPC-H SF300 lineitem read from local SSDs via GPUDirect Storage, the authors report four configuration insights: (1) page count per row group should be at least 100; (2) row-group sizes of millions of rows are preferred; (3) per-column-chunk encoding selection should choose the smallest encoded size; (4) compression should be skipped when size reduction is below a threshold (10% in their experiments). They contribute a Parquet rewriter tool and report an effective read bandwidth of up to 125 GB/s from four SSDs without modifying the Parquet specification. The central claim is that Parquet's poor GPU performance is a configuration problem, not a format problem.

Significance. If the results hold, the paper provides concrete and immediately actionable configuration guidance for GPU Parquet reads and establishes a baseline for evaluating new GPU-oriented file formats. The paper is honest about the limited impact of compression-skipping, which helps only at four SSDs in their setup, and it ships a rewriter tool that is directly reusable. The insights are stated as configuration rules rather than fitted to the measured data, which is a strength. The main limitation is external validity: all experiments use one dataset, one GPU, and the authors' own engine, and the key mechanism in Insight 1 rests on an unverified cuDF implementation detail. Nevertheless, the paper is a useful empirical contribution provided the mechanism claim is substantiated or appropriately scoped.

major comments (3)
  1. [Section 3, 'Increase Page Count'] The premise that 'cuDF maps the number of pages to the grid size in its GPU kernel launch parameters' is stated without citation, code reference, or cuDF version. This is the causal basis for Insight 1 and part of the reported 125 GB/s gain. Because all experiments are run with PystachIO, the authors' own reader layered on cuDF, the reader cannot distinguish a cuDF-wide property from a PystachIO-specific design choice. If other GPU Parquet readers (e.g., Theseus, Sirius, Velox-cuDF) parallelize by row group or column chunk, the page-count rule may not transfer. Please add a cuDF source/version citation or reframe Insight 1 as an empirical finding on this stack, and discuss portability.
  2. [Section 3 and Section 4] The evaluation covers a single dataset (TPC-H SF300 lineitem), a single GPU (A100), and a single engine (PystachIO). The contributions claim 'first guidance on optimizing Parquet for GPU databases,' and Insights 2 and 3 are stated generally. However, Insight 2 (million-row RGs) is tied to GDS saturating this SSD configuration, and Insight 3 depends on the data distribution of one table. To support the central claim that Parquet's poor GPU performance is a configuration problem rather than a format problem, at least one additional dataset and ideally a second reader should be tested; otherwise, the scope of the conclusion should be explicitly narrowed.
  3. [Section 3, 'No Unnecessary Compression'] Insight 4 says 'Skip unnecessary compression if no size reduction,' but the implemented rule is 'only when its size reduction exceeds a chosen threshold (10% in our experiments).' The two statements are inconsistent. Since the measured gain is confined to the four-SSD configuration, the choice of threshold is not innocuous; without a sensitivity analysis or a justification for the 10% threshold, the insight is underdetermined.
minor comments (5)
  1. [Figure 1] The annotation '125x' is confusing; it likely means '125 GB/s' or '125x improvement' but is not explained in the caption or text.
  2. [Figure 2 vs. Figure 3] The metric changes from 'storage bus bandwidth' in Figure 2 to 'effective bandwidth' in Figure 3. The text explains the change, but the figures should label the metric consistently, and both metrics should be defined in a single place.
  3. [References] Reference [24] cites a general Hadoop book rather than the Apache Parquet format specification. Please cite the official specification for the format definitions used in Section 2.
  4. [Rewriter tool availability] The rewriter is available via a URL but no version or commit hash is given. Including a specific version would improve reproducibility.
  5. [Section 3, 'Increase Page Count'] The choice of page count 100 is described as 'reasonable' and tied to the kernel grid size, but no occupancy analysis or A100 grid limits are provided. A brief note would help justify the selected value.

Circularity Check

0 steps flagged

No derivational circularity; empirical config study with external-validity caveats.

full rationale

The paper is an empirical systems study, not a derivation. The four insights are inferred from measurements on a concrete stack (A100, GDS, SSDs, cuDF/PystachIO), and the headline bandwidth gains are reported as measured results, not as predictions obtained by inverting a fitted equation. None of the claimed results are equivalent to their inputs by construction: effective bandwidth is defined independently of the configuration rules, and the page-count, row-group-size, encoding, and compression recommendations are not derived from the same metric they are supposed to explain. The self-citations (PystachIO [17], Arrow issues [3,4,10,27]) provide implementation context and do not serve as the sole justification for the central claim. The unsupported premise that cuDF maps page count to kernel grid size (Section 3, 'Increase Page Count') is an external-validity and correctness risk, not circularity, because it is an asserted mechanism rather than a quantity derived from the asserted conclusion. Similarly, choosing configuration thresholds by inspecting the same workload (page count 100, RG 10M, 10% compression threshold) is a generalization/overfitting risk, but the paper does not present these thresholds as held-out predictions. No circular step can be exhibited where an equation reduces to its own input or a fitted parameter is renamed as a prediction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

This is an empirical tuning study, so its 'free parameters' are configuration values chosen by inspection of the same measurements used to claim improvement: page count 100, RG size 10M rows, the 10% compression threshold, and smallest-encoded-size as the encoding objective. These are legitimate knobs the paper is transparent about, but they are fitted to this workload, not derived. The axioms are domain assumptions about the hardware/software stack: cuDF's page-to-grid mapping (unverified by citation), GDS transfer-size behavior (cited to [6,23]), the stated practical assumption that storage I/O dominates decode cost, representativeness of TPC-H lineitem as the sole dataset, and the attributed property that the authors' own PystachIO fully overlaps I/O and compute. Nothing here is standard math; the burdens are empirical premises that a replication study would need to re-test.

free parameters (4)
  • page count = 100 per column chunk
    Chosen 'as a reasonable choice for kernel grid size' after observing that bandwidth saturates; the paper admits there is no single universally optimal value.
  • row-group size = 10^7 rows
    'We use a larger RG size of 10M rows' so the SSD remains saturated after later optimizations shrink chunk I/O size; picked from the Figure 2(b) sweep on the test SSD.
  • compression-skip threshold = 10% size reduction
    Explicitly a 'chosen threshold (10% in our experiments)'; compression is kept only above this, and the Insight 4 gains are sensitive to it.
  • encoding selection objective = minimum encoded size per column chunk
    Encoding flexibility 'finalize[s] only the one that yields the smallest encoded size'; this optimizes size, and the effective-bandwidth metric rewards size reduction, conflating compression ratio with scan speed.
axioms (5)
  • domain assumption cuDF maps the number of pages to the GPU kernel grid size; page count thus controls GPU utilization.
    Stated in Section 3 'Increase Page Count' with no citation to cuDF source; if a reader parallelizes differently, Insight 1 and the page-count recommendation do not transfer.
  • domain assumption MiB-scale I/O transfers are markedly more efficient than the ~100 KB column chunks implied by DuckDB's row-group size.
    Basis of Insight 2; supported by citations [6,23] and the authors' own Figure 2(b), not by independent derivation.
  • domain assumption For GPU Parquet scans from SSDs, storage I/O is the dominant bottleneck rather than decode/compute.
    Adopted explicitly in Section 4 ('the practical assumption that the dominant bottleneck is storage I/O rather than computation') and used to justify not comparing against new GPU-native encodings.
  • domain assumption TPC-H SF300 lineitem is representative of GPU-analytics Parquet workloads.
    All figures measure this single table; the motivating 85%-scan-time statistic [11] is also TPC-H. No second dataset validates the insights.
  • domain assumption PystachIO fully overlaps I/O and GPU computation, making it a valid evaluation harness.
    Section 2 asserts PystachIO is 'the only system that fully overlaps I/O and GPU computation'; PystachIO is the authors' own system [17], and the assertion is not independently verified in the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 6282 in / 18758 out tokens · 172964 ms · 2026-08-02T22:14:46.907879+00:00 · methodology

0 comments
read the original abstract

Parquet is the de facto columnar file format in modern analytical systems, yet its configuration guidelines have largely been shaped by CPU-centric execution models. As GPU-accelerated data processing becomes increasingly prevalent, Parquet files generated with CPU-oriented defaults can severely underutilize GPU parallelism, turning GPU scans into a performance bottleneck. In this work, we systematically study how Parquet configurations affect GPU scan performance. We show that Parquet's poor GPU performance is not inherent to the format itself but rather a consequence of suboptimal configuration choices. By applying GPU-aware configurations, we increase effective read bandwidth up to 125 GB/s without modifying the Parquet specification.

Figures

Figures reproduced from arXiv: 2602.17335 by Carsten Binnig, Jigao Luo, Qi Chen.

Figure 1
Figure 1. Figure 1: GPU Parquet scan on TPC-H SF300 lineitem with 4 SSDs: file configuration impact on effective read bandwidth. databases, enabling substantial acceleration without requiring a completely new file format. 2 Parquet in GPU Databases Apache Parquet [24] is a widely used columnar storage format for analytical processing. A row group (RG) contains one column chunk (CC) per column, and each CC is divided into page… view at source ↗
Figure 2
Figure 2. Figure 2: GPU Parquet scan on TPC-H SF300 lineitem with one SSD: storage bus bandwidth of different file configura￾tions. Left: varying page counts. Right: varying rows per RG. As shown in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Azim Afroozeh and Peter Boncz. 2025. The FastLanes File Format.Proc. VLDB Endow.18, 11 (2025), 4629–4643. doi:10.14778/3749646.3749718

    arXiv 2025

  2. [2]

    Azim Afroozeh, Lotte Felius, and Peter Boncz. 2024. Accelerating GPU Data Processing using FastLanes Compression. InProceedings of the 20th International Workshop on Data Management on New Hardware, DaMoN 2024, Santiago, Chile, 10 June 2024, Carsten Binnig and Nesime Tatbul (Eds.). ACM, 8:1–8:11. doi:10. 1145/3662010.3663450

    arXiv 2024

  3. [3]

    Andrew Lamb and Xuwei Fu and Jigao Luo. 2025. Apache Arrow Rust Issue: [Parquet] Allow sampling some values to decide the encoding pattern. https: //github.com/apache/arrow-rs/issues/8378. Accessed: 2026-02-01

    2025

  4. [4]

    Andrew Lamb and Xuwei Fu and Jigao Luo. 2025. Apache Arrow Rust Issue: Tuning knobs to tradeoff CPU and compression in parquet. https://github.com/ apache/arrow-rs/issues/8358. Accessed: 2026-02-01

    2025

  5. [5]

    Patel, and Rodrigo Aramburú

    Felipe Aramburú, William Malpica, Kaouther Abrougui, Amin Aramoon, Romulo Auccapuclla, Claude Brisson, Matthijs Brobbel, Colby Farrell, Pradeep Garigipati, Joost Hoozemans, Supun Kamburugamuve, Akhil Nair, Alexander Ocsa, Johan Peltenburg, Rubén Quesada López, Deepak Sihag, Ahmet Uyar, Dhruv Vats, Michael Wendt, Jignesh M. Patel, and Rodrigo Aramburú. 202...

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.05029 2025

  6. [6]

    Nils Boeschen, Tobias Ziegler, and Carsten Binnig. 2024. GOLAP: A GPU-in-Data- Path Architecture for High-Speed OLAP.Proc. ACM Manag. Data2, 6 (2024), 237:1–237:26. doi:10.1145/3698812

    doi:10.1145/3698812 2024

  7. [7]

    DuckDB Documentation. 2026. DuckDB Parquet Tips. https://duckdb.org/docs/ stable/data/parquet/tips. Accessed: 2026-02-01

    2026

  8. [8]

    DuckDB Documentation. 2026. DuckDB Performance Guide: File Formats. https: //duckdb.org/docs/stable/guides/performance/file_formats. Accessed: 2026-02- 01

    2026

  9. [10]

    Jörn Horstmann and Andrew Lamb and Xiangpeng Hao. 2025. Apache Arrow Rust Issue: Parquet LevelEncoder is much too eager to write short rle runs. https://github.com/apache/arrow-rs/issues/7739. Accessed: 2026-02-01

    2025

  10. [11]

    Greg Kimball and Karthikeyan Natarajan. 2025. Accelerating Velox with RAPIDS cuDF - VeloxCon 2025. https://prestodb.io/wp-content/uploads/presto-users/ Accelerating-Velox-with-RAPIDS-cuDF-VeloxCon-April-2025.pdf Accessed: 2026-02-01

    2025

  11. [12]

    Maximilian Kuschewski, David Sauerwein, Adnan Alhomssi, and Viktor Leis

  12. [13]

    Andrew Lamb and Julien Le Dem. 2026. Talk: Column Storage for the AI Era

    2026

  13. [14]

    Andrew Lamb, Yijie Shen, Daniël Heres, Jayjeet Chakraborty, Mehmet Ozan Kabak, Liang-Chi Hsieh, and Chao Sun. 2024. Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine. InCompanion of the 2024 International Conference on Management of Data, SIGMOD/PODS 2024, Santiago, Chile, June 9-15, 2024, Pablo Barceló, Nayat Sánchez-Pi, Alexandr...

    arXiv 2024

  14. [15]

    Harald Lang, Tobias Mühlbauer, Florian Funke, Peter Boncz, Thomas Neumann, and Alfons Kemper. 2016. Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation. InProceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26 - July 01, 2016, Fatma Özcan,...

    arXiv 2016

  15. [16]

    Chunwei Liu, Anna Pavlenko, Matteo Interlandi, and Brandon Haynes. 2023. A Deep Dive into Common Open Formats for Analytical DBMSs.Proc. VLDB Endow.16, 11 (2023), 3044–3056. doi:10.14778/3611479.3611507

    arXiv 2023

  16. [17]

    Jigao Luo, Nils Boeschen, Muhammad El-Hindi, and Carsten Binnig. 2025. Pys- tachIO: Efficient Distributed GPU Query Processing with PyTorch over Fast Networks & Fast Storage.CoRRabs/2512.02862 (2025). arXiv:2512.02862 doi:10.48550/ARXIV.2512.02862

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2512.02862 2025

  17. [18]

    Hamish Nicholson, Konstantinos Chasialis, Antonio Boffa, and Anastasia Aila- maki. 2025. The Effectiveness of Compression for GPU-Accelerated Queries on Out-of-Memory Datasets. InProceedings of the 21st International Workshop on Data Management on New Hardware, DaMoN 2025, Berlin, Germany, June 22-27,

    2025

  18. [19]

    NVIDIA. 2025. NVIDIA GPUDirect Storage. https://docs.nvidia.com/gpudirect- storage/

    2025

  19. [20]

    Pedro Pedreira, Orri Erling, Maria Basmanova, Kevin Wilfong, Laith Sakka, Krishna Pai, Wei He, and Biswapesh Chattopadhyay. 2022. Velox: Meta’s Unified Execution Engine.Proc. VLDB Endow.15, 12 (2022), 3372–3384. doi:10.14778/ 3554821.3554829

    arXiv 2022

  20. [21]

    Mark Raasveldt and Hannes Mühleisen. 2019. DuckDB: an Embeddable Analytical Database. InProceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019, Peter Boncz, Stefan Manegold, Anastasia Ailamaki, Amol Deshpande, and Tim Kraska (Eds.). ACM, 1981–1984. doi:10.1145/3299...

    arXiv 2019

  21. [22]

    RAPIDS Development Team. 2025. NVIDIA RAPIDS libcudf, pylibcudf and cuDF: GPU DataFrame Library. https://github.com/rapidsai/cudf

    2025

  22. [23]

    Torp, Simon A

    Karl B. Torp, Simon A. F. Lund, and Pinar Tözün. 2025. Path to GPU-Initiated I/O for Data-Intensive Systems. InProceedings of the 21st International Workshop on Data Management on New Hardware, DaMoN 2025, Berlin, Germany, June 22-27,

    2025

  23. [24]

    2016.Practical Hadoop Ecosystem: A Definitive Guide to Hadoop- Related Frameworks and Tools(1st ed.)

    Deepak Vohra. 2016.Practical Hadoop Ecosystem: A Definitive Guide to Hadoop- Related Frameworks and Tools(1st ed.). Apress, USA. doi:10.1007/978-1-4842- 2199-0

    work page doi:10.1007/978-1-4842- 2016

  24. [25]

    Robin Vonk, Joost Hoozemans, and Zaid Al-Ars. 2025. GSST: Parallel string decompression at 191 GB/s on GPU.ACM SIGOPS Oper. Syst. Rev.59, 1 (2025), 55–61. doi:10.1145/3759441.3759450

    arXiv 2025

  25. [26]

    doi:10.1145/3736227.3736232

    ACM, 3:1–3:9. doi:10.1145/3736227.3736232

    arXiv

  26. [27]

    Xuwei Fu. 2025. Apache Arrow Rust Pull Request: Parquet: Do not compress v2 data page when compress is bad quality. https://github.com/apache/arrow- rs/pull/8257. Accessed: 2026-02-01

    2025

  27. [28]

    Bobbi Yogatama, Yifei Yang, Kevin Kristensen, Devesh Sarda, Abigale Kim, Adrian Cockcroft, Yu Teng, Joshua Patterson, Gregory Kimball, Wes McKinney, Weiwei Gong, and Xiangyao Yu. 2026. Rethinking Analytical Processing in the GPU Era. In16th Conference on Innovative Data Systems Research, CIDR 2026, Chaminade, CA, USA, January 18-21, 2026. www.cidrdb.org. ...

    2026

  28. [29]

    Will Manning. 2025. FutureData: Vortex: LLVM for File Formats. https://db.cs. cmu.edu/events/futuredata-vortex/. Accessed: 2026-02-01

    2025

  29. [30]

    Patel, Andrew Pavlo, and Huanchen Zhang

    Xinyu Zeng, Ruijun Meng, Martin Prammer, Wes McKinney, Jignesh M. Patel, Andrew Pavlo, and Huanchen Zhang. 2025. F3: The Open-Source Data File Format for the Future.Proc. ACM Manag. Data3, 4 (2025), 245:1–245:27. doi:10. 1145/3749163 3

    2025

  30. [32]

    Xinyu Zeng, Yulong Hui, Jiahong Shen, Andrew Pavlo, Wes McKinney, and Huanchen Zhang. 2023. An Empirical Evaluation of Columnar Storage Formats. Proc. VLDB Endow.17, 2 (2023), 148–161. doi:10.14778/3626292.3626298

    arXiv 2023

  31. [2023]

    ACM Manag

    BtrBlocks: Efficient Columnar Compression for Data Lakes.Proc. ACM Manag. Data1, 2 (2023), 118:1–118:26. doi:10.1145/3589263

    doi:10.1145/3589263 2023