Pith. sign in

REVIEW 3 major objections 5 minor 69 references

HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a hybrid row-and-column distribution of vectors lets a distributed database prune most distance computations early, raising throughput 4.63x over a single-node baseline.

desk verdict Harmony's hybrid partition idea is new and the Euclidean pruning argument is correct, but the cosine/inner-product extension is wrong and the headline speedup is measured against single-node Faiss, not a distributed database. read the letter →

arxiv 2506.14707 v1 pith:Q6MBXSBY submitted 2025-06-17 cs.DB

Qian Xu , Feng Zhang , Chengxi Li , Lei Cao , Zheng Chen , Jidong Zhai , Xiaoyong Du This is my paper · ORCID
classification cs.DB
keywords approximatenearestneighborsearchdistributedvectordatabasedimension-basedpartitioninghybridloadbalancingearly-stoppruningquerythroughputskewedworkloads
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the two classic ways of sharding a vector collection—splitting whole vectors across nodes and splitting each vector's dimensions across nodes—are not rivals but complements, and that using both at once is what makes a distributed approximate nearest neighbor (ANN) system both balanced and fast. It argues that when dimensions are split, a candidate's score accumulates block by block, so once the running partial distance falls outside the current top-K threshold the rest of that candidate's dimensions can be skipped; this lets most candidates be pruned before every node has worked on them. On real datasets the implementation reports a 4.63x average throughput gain over a single-node baseline on four nodes, and reports that skewed workloads, which make vector-only sharding collapse, stay stable under the hybrid scheme. That matters because distance computation is the dominant cost in high-dimensional search, so a distribution that prunes and balances that cost addresses the main bottleneck in scaling ANN search to billions of vectors.

What carries the argument

The load-bearing identity is the additivity of squared Euclidean distance over disjoint dimension subsets, $D^2(p,q)=\sum_k D_k^2(p,q)$, where each $D_k^2$ is non-negative. This monotone accumulation lets Harmony keep a max-heap of the current top-K candidates and prune a candidate as soon as a cumulative partial distance crosses the heap threshold; the same mechanism is applied to dot-product similarity in the paper. The second piece is the execution pipeline that runs dimension blocks one after another across machines and propagates the updated threshold, so a prune decision made on one node stops work on all later nodes before it starts.

What would settle it

On a dataset of real-valued vectors under inner-product similarity, build a query q and a base vector p such that the first dimension block contributes a large negative partial product and the remaining blocks contribute a positive total large enough to put p in the true top-K; run Harmony's dimension pipeline and check whether p survives. If the pipeline prunes p after the first block, the system has dropped a true neighbor, which should be visible as a recall drop below brute-force search; conversely, the method is exonerated for squared Euclidean distance, where every block contribution is non-negative.

Watch

Extended reading notes

Core claim

The central discovery is the multi-granularity partition grid: the dataset is cut into vector shards and each shard is further cut into dimension blocks, with the grid cells spread evenly across machines. Because squared Euclidean distance is additive over disjoint dimension blocks and every block contributes a non-negative amount, the cumulative partial distance is a monotone lower bound on the final distance; the moment it exceeds the current top-K threshold, the candidate can be dropped. Harmony couples this with a cost model that chooses the number of vector shards and dimension blocks per workload, balancing the low communication of vector sharding against the load stability of dimension sharding. The reported upshot is throughput beyond the number of machines—4.63x on average with four nodes—and stable query bandwidth under skewed query distributions, where pure vector sharding degrades by roughly half.

Load-bearing premise

The scheme assumes the score already accumulated from earlier dimension blocks is a reliable lower bound on the final score, so crossing the cutoff means the candidate is truly hopeless; that is exactly true for squared Euclidean distance, but for dot-product or cosine similarity later dimensions can contribute negative terms, so an early prune can discard a true nearest neighbor.

Editorial extensions

If this is right

  • If the claims hold, distributed ANN systems can exceed linear scaling: pruning removes work rather than only spreading it, which is how a four-node system reports 4.63x throughput rather than a maximum of 4x.
  • Skewed or hotspot query workloads, a known failure mode for vector-only sharding that drops throughput by about half in the paper's experiments, should no longer collapse performance because dimension blocks distribute hot queries across all nodes.
  • The hybrid grid applies to any cluster-based index unchanged: clustering is built first and only then are clusters cut into vector shards and dimension blocks, so existing inverted-file indexes can be distributed this way.
  • Communication volume stays constant in the paper's accounting: splitting a query into more, smaller pieces does not increase total bytes moved, only the number of messages, so the pruning gains are not bought with extra bandwidth.
  • Because later dimension blocks prune more (over 80% of candidates are gone by the final slice in the reported datasets), high-dimensional vectors—where distance computation is most expensive—stand to gain the most.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The monotone-pruning argument is airtight only for squared Euclidean distance; for dot-product or cosine similarity, per-dimension products can be negative, so a candidate that looks hopeless after the first block could still win after later blocks. A correct extension would need non-negative embeddings or per-block bounds that keep the running estimate a true lower bound.
  • The dimension-block reordering Harmony uses for load balancing is effectively a scheduling policy; one could formalize it as an optimization that assigns later (more pruned) positions to overloaded nodes, and test whether it maximizes throughput under adversarial query streams.
  • A natural testable extension is whether the pruning ratio grows with dimensionality at fixed node count; if so, Harmony's advantage over vector-only sharding should widen for modern embedding models with more than a thousand dimensions, and shrink for low-dimensional data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. HARMONY proposes a distributed ANNS system that partitions vectors along both vector and dimension axes, uses a cost model to choose a hybrid partitioning plan, and pipelines partial distance computations across nodes to prune unpromising candidates early. The paper argues that dimension-level pruning is sound because distance computations are monotone in the partial sums, reports pruning ratios and QPS-recall experiments on ten datasets, and claims 4.63x throughput over Faiss on four nodes and 58% improvement under skewed workloads. The same pruning argument is presented for both squared Euclidean distance and cosine similarity.

Significance. The paper addresses a relevant problem, and the core Euclidean-distance observation is sound: squared-distance partial sums are non-negative, so the early-pruning rule in Section 3.1 is a correct optimization for L2 distance, and Table 3 gives supporting measurements of pruning effectiveness. The ablation study in Section 6.3 is a useful attempt to separate the contributions of load balancing, pipelining, and pruning. If the claims were restricted to L2 distance and the evaluation included a genuine distributed vector database baseline, the system could be a useful contribution. In its current form, however, the paper overclaims in two load-bearing places: the cosine/inner-product pruning extension is not mathematically justified, and the experimental support for 'outperforms leading distributed vector databases' is absent. No code or artifacts are released, so the empirical results are not independently checkable.

major comments (3)
  1. [Section 3.1 and Algorithm 1] Section 3.1 (Dimension-level pruning) and Algorithm 1 (line 9): the pruning argument is mathematically sound for squared Euclidean distance because every partial contribution (p_i - q_i)^2 is non-negative. The same section, however, extends the argument to cosine similarity through the dot-product decomposition p·q = sum_k alpha_k(p,q), and this extension is not valid for general real-valued vectors: per-dimension products p_i q_i may be negative, so a small partial dot product does not imply a small final dot product, and an early-pruned candidate could still be a top-K result. Algorithm 1's condition 'partialDist > q.currentThreshold' is the correct early-stop rule for a distance to be minimized, not for a similarity to be maximized. Because Word2vec, GloVe, and Deep1M embeddings contain negative coordinates, the pruning ratios in Table 3, the no-overhead statement in Section 4.2.2, and the reported 4.63x throughput are only justified for L2 search unless the paper either restricts the system to that metric or derives a monotone bound for inner-product/cosine search; the evaluation does not state which metric was used for each dataset. This is an internal correctness risk for the cosine/inner-product part of the system, not merely a missing benchmark.
  2. [Section 6.1 and Section 6.5.4] The experimental section contains no distributed vector database baseline. The methods compared are Faiss, a single-node engine, and the authors' own Harmony-vector and Harmony-dimension ablations; Auncel is discussed only qualitatively in Section 6.5.4. The abstract's claim that Harmony outperforms 'leading distributed vector databases' and the '4.63x throughput' headline are therefore not supported as stated: 4.63x is a speedup over single-node Faiss, and for SpaceV1B and Sift1B the comparison uses 16 Harmony nodes because Faiss cannot run. The paper also reports both 3.75x and 4.63x average speedups without defining the averaging procedure (abstract vs. Section 6.2). Please add at least one distributed baseline on the smaller datasets or revise the claims to refer specifically to speedup over single-node Faiss, and define how the average is computed.
  3. [Section 4.2.2 and Figure 8] The statement that Harmony 'does not add any communication or computation overhead' relative to traditional partitioning is contradicted by the system's own design and measurements. The pipelined execution in Algorithm 1 transmits partial results between stages and propagates updated heap thresholds (Section 4.3), and Figure 8 shows that Harmony and Harmony-dimension incur communication overhead while Harmony-vector does not. The complexity analysis counts only the total vector payload and ignores per-message overhead, partial-result transfers, and threshold synchronization. The claim should be qualified to something like 'the total vector payload is unchanged,' and the additional communication should be included in the cost model in Section 4.2.1.
minor comments (5)
  1. [Section 4.3 and Section 4.2.1] A distance computation for a D-dimensional vector is O(D), not O(D^2); this error appears in the expressions involving O(Q·N_B·D^2) and in the centroid-assignment cost O(Q·N_C·D^2), and it overstates the naive baseline in the complexity comparison.
  2. [Table 5] The discussion says 'Deep1M with 100 dimensions,' but Table 2 lists Deep1M as 256-dimensional.
  3. [Figure 11(a)] The caption says 'Relationship between pruning ratio and search probes,' but the figure shows speedup as a function of dataset size and dimension; the caption should be corrected.
  4. [Section 4.2.1] The cost model depends on user-specified coefficients c_dim_comp, c_dim_comm, c_vec_comp, c_vec_comm and weight alpha, but the paper does not report how these are set in the experiments or provide a sensitivity analysis for them.
  5. [Throughout] There are several typos, including 'commnuication' (Sections 1 and 6.3.1), 'themsongdataset' (Section 1), and 'centriod' (Figure 4 caption).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the pruning extension to cosine is a soundness issue, not a circular derivation.

full rationale

The main derived claims are decomposable into algebra and measurements rather than fitted inputs renamed as predictions. The 'no communication overhead' statement in Section 4.2.2 is an arithmetic identity: increasing the number of dimension splits by B while reducing each split's size by 1/B keeps total bytes constant, so the conclusion follows from the paper's own definitions and is not equivalent to the performance claim. The pruning ratio and 4.63x throughput are empirical results from Faiss and the paper's own ablations; no parameter is fit to those numbers and then reported as a prediction. The cost model in Section 4.2.1 is a design heuristic, not a derivation of the experimental speedup. There are no load-bearing self-citations: the cited works are external (Faiss, Auncel, standard ANNS literature) and no uniqueness theorem is invoked. The genuine weakness is the Section 3.1 generalization of Euclidean monotone pruning to cosine/inner-product search: for dot products, partial sums are not monotone because per-dimension products can be negative, so Algorithm 1's early-stop is not generally safe for that metric. That is a correctness and soundness flaw in the stated scope, not a circular reduction of the paper's conclusions to its premises; the Euclidean case remains a genuine mathematical consequence. Benchmarking gaps, such as Auncel being described but not run, are evidence-quality issues rather than circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central performance and pruning claims rest on the Euclidean distance decomposition, which is true, on an unjustified extension to cosine and inner-product pruning, which is false without sign constraints, and on the assumption that finer dimension splitting does not add communication overhead, which is contradicted by the paper's own measurements. The cost model that selects partition plans depends on several unreported parameters, including alpha and per-block cost coefficients, so the reported speedups are conditional on unstated tuning choices.

free parameters (3)
  • alpha (α) = not reported
    User-defined weight in the overall cost function C(π,Q) that balances per-query comp/comm cost against the imbalance factor I(π). The paper does not state the value used in any experiment, so reported speedups are conditional on an unreported tuning choice.
  • cost coefficients c_dim_comp, c_dim_comm, c_vec_comp, c_vec_comm = not reported; example values 20/30/15/1 ms
    The cost model in Section 4.2.1 needs per-block computation and communication costs to select a partition plan, but the paper does not specify how these are estimated in experiments. The example values are illustrative and appear hand-chosen.
  • partition grid (Bvec, Bdim) = varies per workload; not specified per dataset
    The number of vector splits and dimension splits is a tunable design parameter selected by the cost model. The experimental configurations are not reported per dataset, yet they strongly affect load balance and communication overhead.
assumptions (4)
  • standard math Squared Euclidean distance decomposes into a sum of non-negative per-dimension squared differences, so cumulative partial distance is monotonically non-decreasing.
    Invoked in Section 3.1 'Dimension-level pruning' to justify safe early stopping. This is true for squared Euclidean distance.
  • ad hoc to paper Partial dot products for cosine and inner-product search accumulate monotonically, so an early cumulative threshold can safely prune candidates.
    Section 3.1 extends pruning to cosine via pre-normalized dot products, but per-dimension products can be negative, so cumulative dot product is not monotone. The paper provides no non-negativity or boundedness assumption, and pruning could discard true neighbors.
  • domain assumption Increasing the number of dimension-based communication chunks by Bdim times does not increase total communication cost because total bytes are unchanged and per-message overhead is negligible.
    Section 4.2.2 'Advantages' and Section 4.3 'Communication Overhead' claim communication remains identical to vector-based partitioning. The paper's own time breakdowns in Figures 2 and 8 show dimension-based partitioning carries larger communication overhead, so this assumption is doubtful and partially contradicted.
  • domain assumption Initial heap populated with centroid distances and a few random vectors provides a pruning threshold that does not harm recall.
    Algorithm 1 'PrewarmHeap' uses distances to centroids and random vectors as the initial top-K threshold. The paper does not analyze the recall cost of this warm-start heuristic, though experiments suggest it is acceptable on the tested datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search." pith.science (2026). https://pith.science/paper/Q6MBXSBY

@misc{pith2026250614707,
  author       = {Pith},
  title        = {Pith review of: HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q6MBXSBY}},
  note         = {Machine review of arXiv:2506.14707}
}
read the original abstract

Approximate Nearest Neighbor Search (ANNS) is essential for various data-intensive applications, including recommendation systems, image retrieval, and machine learning. Scaling ANNS to handle billions of high-dimensional vectors on a single machine presents significant challenges in memory capacity and processing efficiency. To address these challenges, distributed vector databases leverage multiple nodes for the parallel storage and processing of vectors. However, existing solutions often suffer from load imbalance and high communication overhead, primarily due to traditional partition strategies that fail to effectively distribute the workload. In this paper, we introduce Harmony, a distributed ANNS system that employs a novel multi-granularity partition strategy, combining dimension-based and vector-based partition. This strategy ensures a balanced distribution of computational load across all nodes while effectively minimizing communication costs. Furthermore, Harmony incorporates an early-stop pruning mechanism that leverages the monotonicity of distance computations in dimension-based partition, resulting in significant reductions in both computational and communication overhead. We conducted extensive experiments on diverse real-world datasets, demonstrating that Harmony outperforms leading distributed vector databases, achieving 4.63 times throughput on average in four nodes and 58% performance improvement over traditional distribution for skewed workloads.

Figures

Figures reproduced from arXiv: 2506.14707 by the authors.

Figure 1
Figure 1. An example of overhead in different partition gran [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pruning ratio and time overhead breakdown. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Harmony’s query process. 3) Multi-machine pruning with lightweight pipelines. To tackle the challenge of multi-machine pruning, Harmony introduces a lightweight inter-machine pruning mechanism that propagates par￾tial results and prunes irrelevant computations across nodes. By synchronizing nodes with minimal communication overhead, the system reduces redundant computation, accelerating query execu￾tion and leveragi… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Harmony’s query distribution. C𝑖 denotes the id of the cluster, Q𝑖 denotes the query block i, V𝑖 denotes the ith block according to vector-based partition, D𝑖 denotes the ith block according to dimension-based partition, M𝑖 denotes the ith machine. Overall cost functio…
Figure 5
Figure 5. Figure 5: Harmony’s pipeline pruning and querying. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Time-accuracy trade-off. • –Indexing_Parameters [e.g., nlist, nprobe, dim]: Control clustering granularity and search scope to balance recall, latency, and memory usage. • –𝛼 : User-defined parameters to control load balancing and throughput preferences in cost model. …
Figure 7
Figure 7. Figure 7: Impact of load distribution on query performance. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: The contribution of the three optimization tech [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: The contribution of the three optimization tech [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Harmony’s index build time breakdown. 250k 500k 750k 1M Number of vectors 64 128 256 512 Dimension 79.7 173.0 249.2 263.2 169.4 279.8 306.2 361.9 294.0 310.0 327.2 381.4 332.8 348.9 357.0 413.3 100 200 300 400 Speedup (%) (a) Relationship between pruning ratio and sea…
Figure 11
Figure 11. Figure 11: Impact of pruning effectiveness and indexing pa [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 64 canonical work pages

  1. [1]

    https://drill.apache.org/

    Apache drill. https://drill.apache.org/. Accessed: 2025-01-08

    work page 2025

  2. [2]

    https://www.cockroachlabs.com/product/cockroachdb/

    Cockroachdb. https://www.cockroachlabs.com/product/cockroachdb/. Accessed: 2025-01-08

    work page 2025

  3. [3]

    https://prestodb.io/docs/current/connector/ spark.html

    Integrating presto with spark sql. https://prestodb.io/docs/current/connector/ spark.html. Accessed: 2025-01-08

    work page 2025

  4. [4]

    https://neo4j.com/

    Neo4j. https://neo4j.com/. Accessed: 2025-01-08

    work page 2025

  5. [5]

    http://pinecone.io

    Pinecone. http://pinecone.io. Accessed: 2025-01-08

    work page 2025

  6. [6]

    https://prestodb.io/

    Presto: Distributed sql query engine for big data. https://prestodb.io/. Accessed: 2025-01-08

    work page 2025

  7. [7]

    http://qdrant.tech

    Qdrant. http://qdrant.tech. Accessed: 2025-01-08

    work page 2025

  8. [8]

    https://www.tigergraph.com/

    Tigergraph. https://www.tigergraph.com/. Accessed: 2025-01-08

    work page 2025

Show all 69 references
  1. [9]

    http://vald.vdaas.org

    Vald. http://vald.vdaas.org. Accessed: 2025-01-08

    2025

  2. [10]

    https://vespa.ai/

    Vespa. https://vespa.ai/. Accessed: 2025-01-08

    2025

  3. [11]

    https://janusgraph.org/, 2017

    Janusgraph. https://janusgraph.org/, 2017. Accessed: 2025-01-08

    2017

  4. [12]

    https://archive.ics.uci.edu/, 2024

    UCI machine learning repository. https://archive.ics.uci.edu/, 2024

    2024

  5. [13]

    https://www.youtube.com/, 2024

    Youtube. https://www.youtube.com/, 2024

    2024

  6. [14]

    Artem Babenko, Victor Lempitsky, B.Hari Babu, N.Subhash Chandra, and T.V. Gopal. The inverted multi-index.IEEE transactions on pattern analysis and machine intelligence, 37(6):1247–1260, 2014

    2014

  7. [15]

    Revisiting the inverted indices for billion-scale approximate nearest neighbors

    Dmitry Baranchuk, Artem Babenko, and Yury Malkov. Revisiting the inverted indices for billion-scale approximate nearest neighbors. InProceedings ofthe European Conference on Computer Vision (ECCV, pages 202–216„ 2018

    2018

  8. [16]

    Multidimensional binary search trees used for associative searching.Communications ofthe ACM, 18(9):509–517, 1975

    Jon Louis Bentley. Multidimensional binary search trees used for associative searching.Communications ofthe ACM, 18(9):509–517, 1975

    1975

  9. [17]

    Spann: Highly-efficient billion-scale approxi- mate nearest neighborhood search

    Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. Spann: Highly-efficient billion-scale approxi- mate nearest neighborhood search. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J.Wortman Vaughan, editors,Advance...

    2021

  10. [18]

    Clarkson

    Kenneth L. Clarkson. An algorithm for approximate closest-point queries. In Proceedings ofthe tenth annual symposium on Computational geometry, pages 160–164,1994

    1994

  11. [19]

    Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J

    James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J. J. Furman, Sanjay Ghemawat, Andrey Gubarev, Christian Heiser, Pe- ter Hochschild, and et al. Spanner: Google’s globally-distributed database. In Proceedings of the 10th USENIX Symposium on Ope...

    2012

  12. [20]

    Mirrokni

    Mayur Datar, Nicole Immorlica, Piotr Indyk, and Va-hab S. Mirrokni. Locality- sensitive hashing scheme based on p-stable distributions. InProceedings of the Twentieth Annual Symposium on Computational Geometry, SCG ’04, pages 253–262„ 2004

    2004

  13. [21]

    The tail at scale.Communications of the ACM, 56(2):74–80, 2013

    Jeffrey Dean and Luiz André Barroso. The tail at scale.Communications of the ACM, 56(2):74–80, 2013

    2013

  14. [22]

    Mapreduce: Simplified data processing on large clusters

    Jeffrey Dean and Sanjay Ghemawat. Mapreduce: Simplified data processing on large clusters. InCommunications of the ACM, volume 51, pages 107–113. ACM New York, NY, USA, 2008

    2008

  15. [23]

    Delaunay

    B.N. Delaunay. Sur la sphère vide.Bull. Acad. Sci. URSS, 6:793–800, 1934

    1934

  16. [24]

    Efficient k-nearest neighbor graph con- struction for generic similarity measures

    Wei Dong, Moses Charikar, and Kai Li. Efficient k-nearest neighbor graph con- struction for generic similarity measures. InProceedings of the 20th International Conference on World Wide Web, WWW2011, page 577–586, Hyderabad, India,

  17. [25]

    Faiss, 2020

    Facebook. Faiss, 2020. Accessed: 2024-10-15

    2020

  18. [26]

    Fast approximate nearest neighbor search with the navigating spreading-out graph.Proc

    Cong Fu, Chao Xiang, Changxu Wang, and Deng Cai. Fast approximate nearest neighbor search with the navigating spreading-out graph.Proc. VLDB Endow., 12(5):461–474, jan 2019

    2019

  19. [27]

    K.Ruben Gabriel and Robert R. Sokal. A new statistical approach to geographic variation analysis.Systematic zoology, 18, 3:259–278, 1969

    1969

  20. [28]

    High-dimensional approximate nearest neighbor search: with reliable and efficient distance comparison operations.Proc

    Jianyang Gao and Cheng Long. High-dimensional approximate nearest neighbor search: with reliable and efficient distance comparison operations.Proc. ACM Manag. Data, 1(2), jun 2023

    2023

  21. [29]

    Rabitq: Quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search.Proc

    Jianyang Gao and Cheng Long. Rabitq: Quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search.Proc. ACM Manag. Data, 2(3), may 2024

    2024

  22. [30]

    Gray and Deborah A

    Wayne D. Gray and Deborah A. Boehm-Davis. Milliseconds matter: An introduc- tion to microstrategies and to their use in describing and predicting interactive behavior.Journal of experimental psychology: applied, 6(4), 2000

    2000

  23. [31]

    Manu: a cloud native vector database management system.arXiv preprint arXiv:2206.13843, 2022

    Rentong Guo, Xiaofan Luan, Long Xiang, Xiao Yan, Xiaomeng Yi, Jigao Luo, Qianya Cheng, Weizhi Xu, Jiarui Luo, Frank Liu, et al. Manu: a cloud native vector database management system.arXiv preprint arXiv:2206.13843, 2022

    2022 arXiv

  24. [32]

    Fast approximate nearest-neighbor search with k-nearest neighbor graph

    Kiana Hajebi, Yasin Abbasi-Yadkori, Hossein Shahbazi, and Hong Zhang. Fast approximate nearest-neighbor search with k-nearest neighbor graph. InIJCAI 2011, Proceedings ofthe 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, 2011. AAAI...

    2011

  25. [33]

    P. Jain, B. Kulis, and K. Grauman. Fast image search for learned metrics. In2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8„ 2008-06

    2008

  26. [34]

    Speeding up distributed request-response workflows

    Virajith Jalaparti, Peter Bodik, Srikanth Kandula, Ishai Menache, Mikhail Ry- balkin, and Chenyu Yan. Speeding up distributed request-response workflows. ACM SIG-COMM Computer Communication Review, 43(4):219–230, 2013

    2013

  27. [35]

    Diskann: Fast accurate billion-point nearest neighbor search on a single node

    Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. Diskann: Fast accurate billion-point nearest neighbor search on a single node. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, edi...

    2019

  28. [36]

    Product quantization for nearest neighbor search.IEEE transactions on pattern analysis and machine intelligence, 33(1):117–128, 2010

    Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search.IEEE transactions on pattern analysis and machine intelligence, 33(1):117–128, 2010

    2010

  29. [37]

    Searching in one billion vectors: rerank with source coding

    Hervé Jégou, Romain Tavenard, Matthijs Douze, and Laurent Amsaleg. Searching in one billion vectors: rerank with source coding. In2011 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP, page 861–864. IEEE, 2011

    2011

  30. [38]

    Locally optimized product quantization for approximate nearest neighbor search

    Yannis Kalantidis and Yannis Avrithis. Locally optimized product quantization for approximate nearest neighbor search. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR, pages 2321–2328„ 2014

    2014

  31. [39]

    Efficient retrieval of rec- ommendations in a matrix factorization framework

    Noam Koenigstein, Parikshit Ram, and Yuval Shavitt. Efficient retrieval of rec- ommendations in a matrix factorization framework. InProceedings of the 21st ACM international conference on Information and knowledge management, pages 535–544„ 2012

    2012

  32. [40]

    Kernelized locality-sensitive hashing for scal- able image search

    Brian Kulis and Kristen Grauman. Kernelized locality-sensitive hashing for scal- able image search. InComputer Vision, 2009 IEEE 12th International Conference on, page 2130–2137, V., NLD, 2009. IEEE, Elsevier Science Publishers B

    2009

  33. [41]

    Fexipro: fast and exact inner product retrieval in recommender systems

    Hui Li, Tsz Nam Chan, Man Lung Yiu, and Nikos Mamoulis. Fexipro: fast and exact inner product retrieval in recommender systems. InProceedings of the 2017 ACM International Conference on Management of Data, pages 835–850„ 2017

    2017

  34. [42]

    The design and implementation ofa real time visual search system on jd e-commerce platform

    Jie Li, Haifeng Liu, Chuanghua Gui, Jianyu Chen, Zhenyuan Ni, Ning Wang, and Yuan Chen. The design and implementation ofa real time visual search system on jd e-commerce platform. InProceedings ofthe 19th International Mid- dleware Conference Industry (Rennes, France, page 9–1...

    2018

  35. [43]

    Embedding-based product retrieval in taobao search

    Sen Li, Fuyu Lv, Taiwei Jin, Guli Lin, Keping Yang, Xiaoyi Zeng, XiaoMing Wu, and Qianli Ma. Embedding-based product retrieval in taobao search. InProceed- ings ofthe 27th ACMSIGKDD Conference on Knowledge Discovery & Data Mining (KDD ’21, page 3181–3189, New York, NY, USA, 20...

    2021

  36. [44]

    Lightrec: A memory and search-efficient recommender system

    Defu Lian, Haoyu Wang, Zheng Liu, Jianxun Lian, Enhong Chen, and Xing Xie. Lightrec: A memory and search-efficient recommender system. InProceedings of The Web Conference 2020, pages 695–705„ 2020

    2020

  37. [45]

    Moore, Alexander Gray, and Ke Yang

    Ting Liu, Andrew W. Moore, Alexander Gray, and Ke Yang. An investigation of practical approximate nearest neighbor algorithms. InAdvances in Neural Information Processing Systems 17 [Neural Information Processing Systems, NIPS 2004, pages 825–832,. Vancouver, British Columbia,...

    2004

  38. [46]

    Towards software-defined fpga acceleration for big data analytics

    Fangzhou Alec Lu. Towards software-defined fpga acceleration for big data analytics. 2024

    2024

  39. [47]

    Malkov and D

    Yu A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.IEEE Trans. Pattern Anal. Mach. Intell., 42(4):824–836, apr 2020

    2020

  40. [48]

    Efficient estimation of word representations in vector space, 2013

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space, 2013. arXiv:1301.3781 [cs.CL

    2013 arXiv

  41. [49]

    Marius Muja and David G. Lowe. Scalable nearest neighbour algorithms for high dimensional data.IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(11):2227–2240, 2014

    2014

  42. [50]

    Li, Ryan McElroy, Mike Paleczny, Daniel Peek, and Paul Saab

    Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C. Li, Ryan McElroy, Mike Paleczny, Daniel Peek, and Paul Saab. Scaling memcache at facebook. In10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13, pages 385–398„ 2013

    2013

  43. [51]

    iqan: Fast and accurate vector search with efficient intra-query parallelism on multi-core ar- chitectures

    Zhen Peng, Minjia Zhang, Kai Li, Ruoming Jin, and Bin Ren. iqan: Fast and accurate vector search with efficient intra-query parallelism on multi-core ar- chitectures. In Maryam Mehri Dehnavi, Milind Kulkarni, and Sriram Krish- namoorthy, editors,Proceedings of the 28th ACM SIG...

    2023

  44. [52]

    Glove: Global vectors for word representation.EMNLP, 14:1532–1543, 2014

    Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation.EMNLP, 14:1532–1543, 2014

    2014

  45. [53]

    Jie Ren, Minjia Zhang, and Dong Li. Hm-ann: Efficient billionpoint nearest neighbor search on heterogeneous memory.Proceedings ofthe 34th International Conference on Neural Information Processing Systems, 895:20, 2020

    2020

  46. [54]

    F1: A distributed sql database that scales

    Jeff Shute and et al. F1: A distributed sql database that scales. InProceedings of the VLDB Endowment, volume 6, pages 1068–1079, 2013

    2013

  47. [55]

    Toussaint

    Godfried T. Toussaint. The relative neighbourhood graph of a finite planar set. Pattern recognition, 12, 4:261–268, 1980

    1980

  48. [56]

    Milvus: A purpose-built vector data management system

    Jianguo Wang, Xiaomeng Yi, Rentong Guo, Hai Jin, Peng Xu, Shengjun Li, Xi- angyu Wang, Xiangzhou Guo, Chengming Li, Xiaohai Xu, Kun Yu, Yuxing Yuan, Yinghao Zou, Jiquan Long, Yudong Cai, Zhenxiang Li, Zhifeng Zhang, Yihua Mo, Jun Gu, Ruiyi Jiang, Yi Wei, and Charles Xie. Milvu...

    2021

  49. [57]

    Scalable k-nn graph construction for visual descriptors

    Jing Wang, Jingdong Wang, Gang Zeng, Zhuowen Tu, Rui Gan, and Shipeng Li. Scalable k-nn graph construction for visual descriptors. InComputer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, USA, 2012. IEEE, IEEE Computer Society

    2012

  50. [58]

    Trinary-projection trees for approximate nearest neigh- bor search.IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2):388–403, 2014

    Jingdong Wang, Naiyan Wang, You Jia, Jian Li, Gang Zeng, Hongbin Zha, and Xian Sheng Hua. Trinary-projection trees for approximate nearest neigh- bor search.IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2):388–403, 2014

    2014

  51. [59]

    A survey on learning to hash.IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):769–790, 2018

    Jingdong Wang, Ting Zhang, Jingkuan Song, Nicu Sebe, and Heng Tao Shen. A survey on learning to hash.IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):769–790, 2018

    2018

  52. [60]

    Spectral hashing

    Yair Weiss, Antonio Torralba, and Rob Fergus. Spectral hashing. InAdvances in neural information processing systems, pages 1753–1760,. 2009

    2009

  53. [61]

    O’Reilly Media, Inc., 3rd edition, 2012

    Tom White.Hadoop: The Definitive Guide. O’Reilly Media, Inc., 3rd edition, 2012

    2012

  54. [62]

    Progressively optimized bi-granular document representation for scalable embedding based retrieval

    Shitao Xiao, Zheng Liu, Weihao Han, Jianjin Zhang, Yingxia Shao, Defu Lian, Chaozhuo Li, Hao Sun, Denvy Deng, and Liangjie Zhang. Progressively optimized bi-granular document representation for scalable embedding based retrieval. In Proceedings ofthe ACMWeb Conference 2022, pa...

    2022

  55. [63]

    Spfresh: In- cremental in-place update for billion-scale vector search

    Yuming Xu, Hengyu Liang, Jin Li, Shuotao Xu, Qi Chen, Qianxi Zhang, Cheng Li, Ziyue Yang, Fan Yang, Yuqing Yang, Peng Cheng, and Mao Yang. Spfresh: In- cremental in-place update for billion-scale vector search. In Jason Flinn, Margo I. Seltzer, Peter Druschel, Antoine Kaufmann...

    2023

  56. [64]

    Spark: Cluster computing with working sets

    Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. Spark: Cluster computing with working sets. InProceedings of the 2nd USENIX conference on Hot topics in cloud computing (HotCloud), volume 10, pages 10–10, 2010

    2010

  57. [65]

    Df-gas: a distributed fpga-as-a-service architecture towards billion-scale graph-based approximate nearest neighbor search

    Shulin Zeng, Zhenhua Zhu, Jun Liu, Haoyu Zhang, Guohao Dai, Zixuan Zhou, Shuangchen Li, Xuefei Ning, Yuan Xie, Huazhong Yang, and Yu Wang. Df-gas: a distributed fpga-as-a-service architecture towards billion-scale graph-based approximate nearest neighbor search. InProceedings ...

    2023

  58. [66]

    VBASE: unifying online vector similarity search and relational queries via relaxed monotonicity

    Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, Mao Yang, and Lidong Zhou. VBASE: unifying online vector similarity search and relational queries via relaxed monotonicity. In Roxana Geambasu and Ed Nightin...

    2023

  59. [67]

    Composite quantization for approxi- mate nearest neighbor search

    Ting Zhang, Chao Du, and Jingdong Wang. Composite quantization for approxi- mate nearest neighbor search. InProceedings ofthe 31th International Conference on Machine Learning (ICML, volume 32, pages 838–846„ 2014

    2014

  60. [68]

    Fast, approximate vector queries on very large unstructured datasets

    Zili Zhang, Chao Jin, Linpeng Tang, Xuanzhe Liu, and Xin Jin. Fast, approximate vector queries on very large unstructured datasets. In20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 995–1011, Boston, MA, April 2023. USENIX Association

    2023

  61. [2011]

    Association for Computing Machinery

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.