Filtered Vector Search in a Disaggregated Lakehouse: Composing Table-Format Pruning with Per-File ANN

Rakesh Jain; Syed Zawad (IBM Research); Thomas Griffin

REVIEW 4 major objections 5 minor 25 references

Filtered Vector Search in a Disaggregated Lakehouse: Composing Table-Format Pruning with Per-File ANN

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Filtered vector search gets fast when the table's own file pruning runs before per-file ANN, avoiding a new filtering algorithm entirely.

desk verdict Solid systems work with honest disclosure, but the headline filtered-vector-search claim is not supported by the paper's own measurements: the 32x is unfiltered, the pruned filtered query runs at 34s, and the 94x needs a materialized re-partitioned copy. read the letter →

arxiv 2608.05441 v1 pith:GJQ67D37 submitted 2026-08-05 cs.DB cs.DCcs.ETcs.IR

Rakesh Jain , Thomas Griffin , Syed Zawad (IBM Research) This is my paper · ORCID

classification cs.DBcs.DCcs.ETcs.IR

keywords vectorsearchapproximatenearestneighborlakehouseApacheIcebergParquetdataskippingdisaggregatedcomputeper-fileIVF

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that filtered vector search in a disaggregated lakehouse can be made fast by composing the table's existing file-pruning machinery with a per-file ANN index, rather than inventing a new filtering algorithm. The planner prunes data files with the predicate first, using partition pruning, zone-maps, and a bitmap index, and then runs IVF only over the surviving files, assembling the global top-k without re-scanning data. If correct, the lakehouse's own metadata becomes the filtering engine for vector search, so vectors never need to be copied into a separate vector store and the table's access control applies unchanged. On an 11.5M by 768 table, the measured result is about 32 times faster than brute force at recall@10 at least 0.90, with 355 of 444 files pruned before ANN runs. On a 5M-vector real corpus, a filter arriving across a join runs about 94 times faster once the reduction is materialized into a region-partitioned layout.

What carries the argument

The load-bearing mechanism is the composition of the lakehouse's existing file pruning with a per-file IVF index: the planner first runs the query predicate through Iceberg partition pruning, per-column zone-maps, and a scalar bitmap index to get the surviving file set, then runs IVF only over those files. A second distinct piece is the predicate-pushdown purity condition, which states that a residual filter can be pushed into the per-file search only when the filter column is provably pure in a file, meaning an identity partition or a materialized cluster column; a merely sorted or z-ordered column prunes well but is not pure, so it is excluded. The paper also contributes the distributed, non-destructive index build, a metadata-only Iceberg replace that preserves time travel, and the disaggregated warm-set engineering, a rendezvous-hashed per-file cache with stable pod placement, which the ablation shows is what turns correct recall into sub-second latency.

What would settle it

Run the filtered query on a table where the filter column is uniformly mixed across all files with no partition or clustering, keep the same warm cache, and check whether the speedup over brute force remains large; if it does, the claim that file-level locality is required is contradicted.

Watch

Extended reading notes

Core claim

The central discovery is that filtered ANN search does not need a new in-index filtering method when the vectors live inside an open lakehouse table; it needs the table's own file-pruning stack to do the filtering one level up. The paper embeds an IVF index in each Parquet file's footer, then rewrites the query plan so the predicate runs through the table's existing partition, zone-map, and bitmap pruning before any ANN search happens. Only the surviving files are probed, and the per-file search returns its top candidates together with their projected columns, so the final merge needs no second data scan. The composition pays off exactly when the filter column has file-level locality, meaning each value is concentrated in few files, because that is when metadata pruning can drop most files. The paper measures a roughly 32 times warm speedup over brute force on an 11.5M by 768 table at recall@10 at least 0.90, with 355 of 444 files pruned before ANN, and a roughly 94 times speedup on real 5M-vector embeddings when a join filter is materialized into a region-partitioned layout.

Load-bearing premise

The headline speedups assume that the filter column has file-level locality in the physical layout and that the per-pod warm cache stays warm with stable rendezvous placement, because without those conditions the paper's own ablation shows per-file IVF is no faster than brute force.

Editorial extensions

If this is right

Vector search can inherit the lakehouse catalog's access control and single source of truth, because vectors never leave the Iceberg or Parquet table and the index is an ignorable footer artifact.
When the filter dimension is already partitioned or clustered, or is materialized into a partition, filtered vector queries reduce to an ordinary file-pruned scan followed by a small ANN probe, with no dedicated filtered-ANN algorithm needed.
The measured 32 times and 94 times speedups are system-level numbers, while the underlying arithmetic reduction is much larger, so improvements to object-store read latency and cache warmth should translate into proportionally larger end-to-end wins.
Sorted or z-ordered columns can drive file pruning but must not be used for residual-predicate pushdown, because a boundary file holds two values and a purity assumption would corrupt results.
Per-file failures during the distributed build are non-fatal and unindexed files fall back to brute force at query time, so the table stays correct while the index fills in.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the locality precondition is the real lever, practitioners should treat filtered-vector-search tuning as a physical-design task, namely partitioning or clustering tables on the common filter dimensions, rather than as a choice among filtered-ANN algorithms; this generalizes the paper's uniform versus value-local layout comparison.
The gap between the roughly 500 times arithmetic reduction and the roughly 30 times measured speedup implies that on a faster storage tier, such as local NVMe or an object store with lower per-request latency, the same composition could approach a much larger fraction of the algorithmic win, which is a testable prediction using the paper's own cost model.
Because the footer blob is opaque to the table format, the per-file index could be swapped from IVF to product quantization or a small graph without changing the build and commit protocol, and a shared global codebook across files is a natural next experiment.
The sorted-is-not-pure distinction likely applies beyond ANN, meaning any lakehouse optimization that pushes a predicate into a per-file operator based on zone-map pruning alone should re-check whether the column is truly single-valued per file rather than merely locally contiguous.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a genuine systems paper with a real contribution, but the headline claim doesn't survive contact with its own tables. The 32x speedup is measured on unfiltered queries; the filtered query that prunes 355/444 files is never given a latency in the main evaluation, and when the paper does report it in Table 4's footnote, it's 34s p50—slower than brute force. The only fast filtered number is the 157ms join case, which requires a denormalized, region-partitioned physical copy. So the abstract's 'payoff comes entirely from file pruning' is not what the measurements show.

What is actually good: the per-file IVF in Parquet footers with a metadata-only Iceberg replace is a clean way to keep the index in-place and non-destructive. The planner composition with existing file pruning is a sensible idea, and the sorted-vs-pure distinction—excluding SORT/ZORDER from residual pushdown—is a real, non-obvious correctness insight. The semi-join reduction for filters arriving across a join is a nice reuse of existing machinery. The warm-set ablation is the most valuable part: it shows a naive per-file IVF is no faster than brute force, and that the matrix cache, rendezvous placement, and stable pod set are what turn correctness into speed. That is an honest and useful systems result.

The paper is also unusually candid: it reports the 34s failure mode, the Milvus baseline on a small subset, the constructed locality, and the failure to prune customer_id. That transparency is to its credit.

The soft spots are real but fixable. The abstract and intro overstate the evidence. The cost model in §9 compares a sigma=0.2 prediction to the unfiltered 32x measurement, an apples-to-oranges comparison. And crucially, the paper never tests the configuration that would actually validate the composition: a value-local file set registered as materialized-cluster so the residual predicate is pushed and the fused direct top-k path is taken. That measurement is missing. If it shows the filtered query going sub-second, the paper's thesis is solid. If not, the thesis needs serious revision.

Who should read this: anyone building ANN on a lakehouse engine, and anyone interested in how much of an algorithmic win survives the systems tax. The paper deserves peer review, but a serious referee should ask for a revision that aligns the claims with the evidence and adds the missing pushdown measurement.

Referee Report

4 major / 5 minor

Summary. The paper proposes embedding a per-file IVF index in each Parquet footer of an Apache Iceberg table, then composing the table's existing file-pruning tiers (partition pruning, zone-maps, bitmap index) with per-file ANN so that filtered vector queries first prune files and then run IVF only over survivors. It describes a distributed, metadata-only Iceberg build, a planner rewrite that fuses pruning with per-file search and direct top-k assembly, predicate pushdown restricted to provably pure columns, a semi-join reduction for filters arriving across joins, and a rendezvous-hashed per-pod cache that makes warm latency tolerable on disaggregated object storage. The evaluation reports a ~32x unfiltered warm speedup over brute force at recall@10 >= 0.90 on an 11.5M x 768 table, pruning counts for a category-filtered query, and a ~94x speedup (14.7 s to 157 ms) for a region-filtered join on real IBM Granite embeddings, the latter after materializing the reduction into a region-partitioned copy.

Significance. If the central claim held as stated, the paper would make a useful systems contribution: filtered vector search inside an open lakehouse table without a specialized filtered-ANN algorithm, reusing the table format's own pruning and preserving multi-engine readability, time travel, and catalog access control. The paper is honest in its limitations section: it reports failure modes, discloses the warm-set ablation showing that naive per-file IVF is no faster than brute force, and explicitly says the locality it exploits is constructed rather than organic. It also ships a concrete reproducibility plan (open Iceberg datasets, precomputed ground truth, and benchmark drivers), which is a strength. However, the headline claim is not supported by the measurements as written: the 32x figure is unfiltered; the only filtered query on the original value-local layout is reported with pruning counts but no latency, and its latency appears elsewhere as 34 s, worse than brute force; and the fast filtered result is obtained only after a materialized, region-partitioned physical redesign. These are load-bearing gaps for the paper's central claim, not mere presentation issues.

major comments (4)

[Abstract; §8.6 (Table 3) and §8.7 (Table 4, footnote ∥)] The abstract states that "the payoff comes entirely from file pruning," but the only filtered query on the original value-local layout is not shown to be fast. Table 3 reports that WHERE category='tech' prunes 355/444 files before ANN, but it reports no wall-clock latency; Table 4 footnote ∥ reports the same query at p50 34 s because the surviving 89 files take the projection/rescore path rather than the fused direct top-k path, while the unfiltered brute-force p50 is 21.1 s (Table 1). Thus the paper's headline 32x figure is unfiltered, and the filtered query on the layout that is supposed to demonstrate the composition is actually slower than the unfiltered brute-force baseline. The central claim requires either a measured filtered-query latency on the value-local layout with the residual predicate pushed (i.e., with category registered as a pure column so the fast path is taken), or a substantial revision of the claim to separate the unfiltered algorithmic speedup from the filtered-query behavior.
[§5.2, §8.8 (Table 5)] The 94x speedup is achieved only after a materialized physical redesign: the docs_r table is a denormalized copy of the fact table with region partitioned as a partition column. The query-time join path on the original tables, which uses the existing customer_id pruning, runs at 14.7 s p50 and prunes no partitions (Table 5). Therefore the headline filtered speedup does not come from the original table's existing file pruning alone; it comes from the user first performing a schema/materialization change (denormalization plus partition-by-region). The paper should state this prominently in the abstract and contributions, and should not present the 94x result as evidence that composing the table's existing pruning with per-file ANN suffices for filtered vector search. The materialized-layout result is still interesting, but it is a different claim.
[§8.4 (Table 2)] The ablation in Table 2 shows that the naive per-file IVF implementation is no faster than brute force (1.0x), and the speedup to 31.6x is accumulated only after adding the per-file matrix cache, rendezvous placement, and stable pod set. This directly contradicts the abstract's statement that "the payoff comes entirely from file pruning." The paper's own body correctly frames the warm-set engineering as turning correctness into speed, but the abstract and Section 1's framing should be corrected to say that the algorithmic win is due to pruning-plus-IVF while the end-to-end measured speedup is due to pruning-plus-IVF plus the warm-set engineering of Section 7.
[§5.1 and §8.6/§8.7] Section 5.1 defines file_local_cols as partition columns or materialized cluster-spec columns and states that only then can the residual predicate be pushed into the per-file search. The value-local layout of Section 8.6 (per-category files) appears to satisfy the purity condition for category, yet Section 8.7 reports that the same query takes the projection/rescore path and costs 34 s. The paper never explains why the residual predicate was not pushed for this layout, nor does it report the natural experiment: declare category as a partition or materialized cluster column on the original table and measure the filtered query on the fused fast path. Without this experiment, the paper's only fast filtered number comes from the materialized docs_r layout, and the reader cannot tell whether the composition itself delivers filtered speedups or only the combination of composition plus a pure-partition physical design.

minor comments (5)

[§4 title and text] The build is described as "in-place," but it rewrites each file to a new object and commits a metadata-only replace; "in-place" is misleading. Consider "non-destructive per-file rewrite" or similar.
[Abstract and §5.2] The statement that "the vectors never leave the table" should be reconciled with the docs_r materialization, which is a copy (albeit inside the lakehouse) of the fact table with a denormalized region column.
[§8.7 (Table 4, footnote ∥)] The 34 s filtered-query latency is reported only in a footnote; it should be called out in the main text of Section 8.6, since it directly qualifies the pruning-count result in Table 3.
[§8.3] Minor typo: "the recalldistribution" should be "the recall distribution."
[§5.1] The set file_local_cols is introduced informally; a formal definition with an equation or a table would improve precision, especially since the soundness of the pushdown depends on it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the design is evaluated against external and self baselines, and the analytic links (pruning, locality, cost model) are derived from the table format's mechanics rather than from the results they are used to explain.

full rationale

Walking the derivation chain: the paper's central mechanism is a planner rewrite (Alg. 1) that (Step 1) feeds the SQL predicate through the engine's existing partition/zone-map/bitmap file pruning, (Step 2) runs per-file IVF over surviving files, and (Step 3) fuses projection with candidate rescore. None of these steps is defined in terms of the latency or recall numbers it explains: pruning counts come from PlanFiles over Iceberg metadata, IVF is standard k-means/IVF with explicit knobs C, n_probe, s, and the speedups are measured against the system's own brute force and query-time join. The 'locality iff' claim (Section 6) is an analytic consequence of how file pruning proves absence, and it is then verified on uniform versus value-local layouts rather than used as a fitted prediction. The Section 9 cost model is a direct arithmetic count (C/(sigma*n_probe)) explicitly contrasted with the smaller measured speedup, so the model is not presented as a measurement. There is no load-bearing self-citation: references 16, 7, 10, and 23 are external, and the in-footer idea is credited to prior practice. The evaluation's weak points—the 34 s rescore path for the pruned category='tech' query (Table 4 footnote), the 94x result being measured on a materialized region-partitioned copy rather than the original layout (Section 8.8), and Section 10's own admission that the locality is constructed and the Milvus baseline is a 300K-subset stand-in—are evidentiary limitations, not instances of a conclusion being equivalent to its premise. No fitted parameter is renamed as a prediction, and no author-imported uniqueness theorem forces a choice. Verdict: no significant circularity.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

No new entities are posited. Key assumptions are environmental (immutable objects, no atomic rename), semantic (partition purity), and statistical (synthetic latent-cluster structure). C, n_probe, and over-fetch s are tuned knobs reported explicitly, not fitted to make a prediction.

free parameters (3)

C (IVF centroids per file) = 1024
Chosen via rule of thumb 2^ceil(log2(4*sqrt(rows/file))); authors report 4096 over-clustered with no recall gain. Build and query cost depend on it.
n_probe = 10
Query-time knob; swept in Figure 3 to trade recall and latency.
over-fetch safety factor s = not stated
Controls candidate over-fetch for residual filters; its value is not quantified, making the recall-tail argument hard to audit.

assumptions (7)

domain assumption Object-store data files are immutable once written; new writes go to new paths
Used in section 7 to justify cache coherence without invalidation. Holds for S3-compatible object stores in this deployment but is an environmental assumption.
domain assumption Byte-for-byte copy of data pages preserves file-local row ordinals so posting lists stay valid
Required by section 4's non-destructive commit. Depends on the engine's exact rewrite behavior.
domain assumption Partition columns and materialized cluster-spec columns are provably single-valued per file; sorted or ZORDER columns are not
Stated in sections 5.1 and 10 as the soundness condition for residual predicate pushdown. The paper argues this from metadata semantics but does not prove it formally.
domain assumption Synthetic embedding corpus has latent cluster structure so IVF recall is meaningful
Section 8 says the generator produces latent-cluster structure. Recall numbers therefore do not transfer to adversarially distributed real embeddings.
domain assumption Object stores have no atomic rename, so the build uses new paths
Section 4. Environmental property of S3-like stores, not universal.
standard math IVF and k-means provide a valid ANN approximation with the reported parameters
Standard algorithm; recall is validated against brute force in sections 8.2 and 8.3.
domain assumption The Flight-SQL engine and co-located worker process execute shards as described
System under test; not independently verifiable from the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Filtered Vector Search in a Disaggregated Lakehouse: Composing Table-Format Pruning with Per-File ANN." pith.science (2026). https://pith.science/paper/GJQ67D37

@misc{pith2026260805441,
  author       = {Pith},
  title        = {Pith review of: Filtered Vector Search in a Disaggregated Lakehouse: Composing Table-Format Pruning with Per-File ANN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GJQ67D37}},
  note         = {Machine review of arXiv:2608.05441}
}

read the original abstract

Approximate nearest-neighbor (ANN) search increasingly runs alongside structured data - "find the 10 nearest documents where tenant='acme' AND lang='en'" - yet similarity and filtering are usually bolted together: a specialized vector index for one, a separate filter step for the other. We ask what happens when both live inside an open lakehouse table (Apache Iceberg over Parquet on object storage), where the engine already owns a mature file-pruning stack (partition pruning, zone-maps, a bitmap index). We embed an IVF index in place in each Parquet file's footer and make filtered vector queries fast not with a new filtering algorithm but by composing the table's existing file pruning with per-file ANN: the planner prunes data files by the predicate first, then runs IVF only over the survivors. The index is built distributed and non-destructively - a metadata-only Iceberg replace that every other engine still reads - and a rendezvous-hashed per-file cache keeps object-store read latency from swamping the algorithmic win. The payoff comes entirely from file pruning. On an 11.5M x 768 table, warm IVF search is ~32x faster than brute force at recall@10 >= 0.90, a selective predicate having pruned 355 of 444 data files before ANN runs; on 5M real IBM Granite embeddings, a filter arriving across a join prunes four of five region partitions and runs nearly two orders of magnitude (~94x: 14.7 s -> 157 ms) faster than the query-time join at identical top-k, once the reduction is materialized into a region-partitioned layout. We characterize when the composition pays off - it requires file-level locality on the filter column, and the residual predicate is only safe to push into the search over a provably pure (partitioned) column, not a merely sorted one - and report the failure modes we hit bolting ANN onto a lakehouse engine.

Figures

Figures reproduced from arXiv: 2608.05441 by the authors.

**Figure 2.** Warm top-10 latency, 11.5M×768, 𝑛probe=10, result cache off, 200-query sample. Bars are p50; black ticks overlay p95. Latency (𝑦) is log10 seconds. IVF p50/p95 = 668/831 ms; brute-force is bimodal (cached ∼1–2 s vs. cold ∼21 s), p50 21.1 s [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Recall@10 (mean over the 200-query sample) and [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Warm-set ablation (11.5M×768, 𝑛probe=10): each bar adds one fix from §7 to the previous (𝑦-axis: speedup vs. brute force, ×). Correct recall throughout; only the cache and stable placement turn correctness into speed. so the curve requires a dedicated smaller-corpus ha…

**Figure 5.** Figure 5: Files ANN must touch for WHERE category=’tech’ (11.5M×768), same data, two physical layouts (𝑦-axis: data files scanned, of 444). With value-local files the bitmap prunes 355/444 before ANN runs (89 files remain); uniform layout prunes none. Identical, correct results.…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 23 canonical work pages

[1]

Apache Software Foundation. 2023. Apache Iceberg: An open table format for huge analytic datasets. https://iceberg.apache.org

work page 2023
[2]

Apache Software Foundation. 2023. Apache Parquet. (2023). https://parquet.apache.org

work page 2023
[3]

Michael Armbrust, Ali Ghodsi, Reynold Xin, and Matei Zaharia. 2021. Lake- house: A new generation of open platforms that unify data warehousing and advanced analytics.CIDR(2021)

work page 2021
[4]

Alexander Behm, Shoumik Palkar, Utkarsh Agarwal, et al. 2022. Photon: A fast query engine for lakehouse systems. InSIGMOD

work page 2022
[5]

Artur Borycki. 2026. Puffin-Backed Vector Indexes: Attaching Approxi- mate Nearest Neighbor Indexes to Apache Iceberg Snapshots for Compute- Disaggregated Query Engines.arXiv preprint arXiv:2606.04196(2026)

work page Pith review arXiv 2026
[6]

Qi Chen, Bing Zhao, Haidong Wang, et al. 2021. SPANN: Highly-efficient billion- scale approximate nearest neighbor search. InNeurIPS

work page 2021
[7]

Yannis Chronis, Helena Caminal, Yannis Papakonstantinou, Fatma Özcan, and Anastasia Ailamaki. 2025. Filtered Vector Search: State-of-the-art and Research Opportunities.Proceedings of the VLDB Endowment (PVLDB)18, 12 (2025), 5488–

work page 2025
[8]

ClickHouse. 2024. ClickHouse: Approximate nearest neighbor search in- dexes. https://clickhouse.com/docs/en/engines/table-engines/mergetree- family/annindexes

work page 2024

Show all 25 references

[9]

Benoit Dageville, Thierry Cruanes, Marcin Zukowski, et al. 2016. The Snowflake elastic data warehouse. InSIGMOD

2016
[10]

DuckDB Labs. 2024. DuckDB VSS: Vector Similarity Search extension. https://duckdb.org/docs/extensions/vss

2024
[11]

Siddharth Gollapudi, Neel Karia, Varun Sivashankar, Ravishankar Krish- naswamy, Nikit Begwani, Swapnil Raz, Yiyong Lin, Yu Zhang, Neelam Maha- patro, Premkumar Srinivasan, et al. 2023. Filtered-DiskANN: Graph algorithms for approximate nearest neighbor search with filters.Proc...

2023
[12]

Gaurav Gupta, Jonah Yu, Tharun Medini, and Anshumali Shrivastava. 2023. CAPS: A practical partition index for filtered similarity search. InarXiv preprint arXiv:2308.15014

2023 arXiv
[13]

Herve Jégou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search.IEEE TPAMI33, 1 (2011), 117–128

2011
[14]

Andrew Kane. 2024. pgvector: Open-source vector similarity search for Postgres. https://github.com/pgvector/pgvector

2024
[15]

LanceDB. 2024. Lance: modern columnar data format for ML and LLMs. https://lancedb.github.io/lance/

2024
[16]

Mocheng Li, Yue Zhang, Chenhao Ma, Xiao Yan, Baotong Lu, and James Cheng
[17]

Malkov and Dmitry A

Yu A. Malkov and Dmitry A. Yashunin. 2020. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.IEEE TPAMI42, 4 (2020), 824–836

2020
[18]

Liana Patel, Peter Kraft, Carlos Guestrin, and Matei Zaharia. 2024. ACORN: Per- formant and predicate-agnostic search over vector embeddings and structured data. InSIGMOD

2024
[19]

Suhas Jayaram Subramanya, Devvrit, Rohan Kadekodi, Ravishankar Kr- ishaswamy, and Harsha Vardhan Simhadri. 2019. DiskANN: Fast accurate billion-point nearest neighbor search on a single node. InNeurIPS

2019
[20]

Thaler and Chinya V

David G. Thaler and Chinya V. Ravishankar. 1998. Using name-based mappings to increase hit rates.IEEE/ACM Transactions on Networking6, 1, 1–14

1998
[21]

Jianguo Wang, Xiaomeng Yi, Rentong Guo, et al. 2021. Milvus: A purpose-built vector data management system. InSIGMOD

2021
[22]

Mengzhao Wang, Lingwei Xu, Xiaoliang Guo, et al. 2022. Native hybrid queries via structured labels and near neighbor search over vectors.arXiv preprint arXiv:2203.13601(2022)

2022 arXiv
[23]

Peng Xiang. 2024. Vector search with Parquet and DataFusion. https://blog.xiangpeng.systems/posts/vector-search-with-parquet-datafusion/. 12

2024
[2025]

Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study.arXiv preprint arXiv:2508.16263(2025)

2025
[5492]

https://doi.org/10.14778/3750601.3750700

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Apache Software Foundation. 2023. Apache Iceberg: An open table format for huge analytic datasets. https://iceberg.apache.org

work page 2023

[2] [2]

Apache Software Foundation. 2023. Apache Parquet. (2023). https://parquet.apache.org

work page 2023

[3] [3]

Michael Armbrust, Ali Ghodsi, Reynold Xin, and Matei Zaharia. 2021. Lake- house: A new generation of open platforms that unify data warehousing and advanced analytics.CIDR(2021)

work page 2021

[4] [4]

Alexander Behm, Shoumik Palkar, Utkarsh Agarwal, et al. 2022. Photon: A fast query engine for lakehouse systems. InSIGMOD

work page 2022

[5] [5]

Artur Borycki. 2026. Puffin-Backed Vector Indexes: Attaching Approxi- mate Nearest Neighbor Indexes to Apache Iceberg Snapshots for Compute- Disaggregated Query Engines.arXiv preprint arXiv:2606.04196(2026)

work page Pith review arXiv 2026

[6] [6]

Qi Chen, Bing Zhao, Haidong Wang, et al. 2021. SPANN: Highly-efficient billion- scale approximate nearest neighbor search. InNeurIPS

work page 2021

[7] [7]

Yannis Chronis, Helena Caminal, Yannis Papakonstantinou, Fatma Özcan, and Anastasia Ailamaki. 2025. Filtered Vector Search: State-of-the-art and Research Opportunities.Proceedings of the VLDB Endowment (PVLDB)18, 12 (2025), 5488–

work page 2025

[8] [8]

ClickHouse. 2024. ClickHouse: Approximate nearest neighbor search in- dexes. https://clickhouse.com/docs/en/engines/table-engines/mergetree- family/annindexes

work page 2024