The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform

Chuanghua Gui; Haifeng Liu; Jianyu Chen; Jie Li; Ning Wang; Zhenyun Ni

REVIEW 3 major objections 5 minor 1 cited by

The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 100-billion-image visual search index answers in under a second and ingests a billion updates per day.

desk verdict A credible industrial system paper with real engineering value, but the headline sub-second claim for hundreds of billions of images is an extrapolation from a 100,000-image testbed. read the letter →

arxiv 1908.07389 v1 pith:EVD3BRBT submitted 2019-08-19 cs.IR

Jie Li , Haifeng Liu , Chuanghua Gui , Jianyu Chen , Zhenyun Ni , Ning Wang This is my paper · ORCID

classification cs.IR

keywords visualsearchreal-timeindexingimageretrievale-commercedistributedsystemsinvertedindexscalabilitydatafreshness

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that an e-commerce visual search system can be genuinely real time at internet scale: a shopper photographs a product, and the system returns similar products from a collection of roughly one hundred billion images in under a second, while also absorbing about one billion image updates per day so that new, delisted, or repriced products appear in results immediately. It argues that the path to this is a distributed three-level search architecture combined with an in-memory index built for cheap appends, deletions, and attribute updates rather than for batch rebuilding. The authors report a day of production data with 977 million index updates at an average latency of 132 milliseconds, and a 100,000-image testbed that sustains 1800 queries per second with a 99th-percentile response time of 0.3 seconds. A sympathetic reader would take the contribution to be the engineering pattern—reusing features, flipping validity bits, and appending to inverted lists—that makes freshness affordable at this scale.

What carries the argument

The load-bearing object is the append-only real-time inverted index. Image features are grouped into $N$ clusters by k-means; each cluster owns an inverted list of image IDs, and an auxiliary array stores the position of the last element of every list, so inserting a new image just computes its nearest cluster and appends its ID. A validity bitmap makes deletion a flag flip and lets re-listed products reuse stored features, while a forward index of fixed-length numeric fields (with variable-length URLs in a side buffer) allows in-place attribute updates. When an inverted list reaches capacity, a double-size replacement list is built in the background and swapped in lock-free, so queries never contend with memory reallocation. Around this index, a three-tier blender-broker-searcher fan-out partitions the image set by URL hash, has each searcher own one partition, and merges top-$k$ results upward.

What would settle it

Run the same query workload against the full approximately 100-billion-image production index and record the response-time distribution; if the 99th-percentile latency substantially exceeds the 0.3 seconds measured on the 100,000-image, 20-searcher testbed—or if average latency exceeds the sub-second bound—the paper's scale claim does not hold.

Watch

Extended reading notes

Core claim

The central claim is that real-time visual search over hundreds of billions of product images can be supported by a hierarchical distributed system whose indexing, and not just its search, is designed for high-churn updates. The paper's concrete mechanism is a real-time index that appends a new image's ID to the end of one of $N$ k-means-clustered inverted lists while an auxiliary array records the tail position of each list; deletions are single-bit flips in a validity bitmap, and a product that returns to the market reuses its previously extracted image features instead of paying for feature extraction again. Attribute changes update fixed-length fields in a forward index in place, and inverted-list memory grows by copying to a double-size list in the background so search never blocks. The paper reports that on August 4, 2018 the production system processed 977 million image updates with average latency 132 ms, and that a testbed with 100,000 images and 20 searcher nodes reached 1800 queries per second with a 99th-percentile response time of 0.3 seconds, which it takes as evidence for the sub-second scale claim.

Load-bearing premise

The central scale claim rests on extrapolating query latency from a 100,000-image, 20-searcher testbed to the production system holding roughly 100 billion images; the paper reports production update latency but not production query latency over the full index.

Editorial extensions

If this is right

Search results can track product availability in near real time: the reported production update latency is 132 ms on average and 816 ms at the 99th percentile, so a delisted item can disappear from results within about a second.
Freshness does not have to cost throughput: enabling the real-time index introduced under 10% overhead in the testbed's query throughput, with response times similar to the non-real-time run.
Feature reuse is the main cost saver: 513 million of the 521 million image additions on the reported day were products returning to the market, so their features did not need to be recomputed.
The system scales horizontally: because each searcher owns one URL-hashed partition, the 1800-queries-per-second testbed result can be raised by adding more searcher nodes rather than by changing the search algorithm.
The tested configuration supports about 155 million search requests per day, which the paper argues is more than enough for typical e-commerce visual search demand.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The append-only inverted list plus validity bitmap pattern is a general recipe for any high-churn embedding search problem—recommendation items, ads, documents—where objects appear and disappear but the embedding space is stable.
If the sub-second claim extrapolates to the full index, the design suggests that billion-scale image search need not rely on memory-heavy approximate-nearest-neighbor libraries: a coarse k-means partition with a per-shard linear scan of one cluster can be enough when partitions are small.
The paper's production latency numbers describe index updates, not queries over the full 100-billion-image index; a natural follow-up is a response-time distribution from the live index, which would directly test the extrapolation from the 100,000-image testbed.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

This paper is worth reading, but with a critical eye. It describes JD.com's visual search system: distributed indexing and search with real-time updates, feature reuse via a validity bitmap, and lock-free inverted-list expansion. The engineering is sane, and the production update numbers are concrete: about one billion image updates per day, with an average update latency of 132ms. That is a real data point for anyone building similar systems.

The novelty is not in the algorithms—k-means inverted lists, nearest-cluster search, and product ranking are all known—but in the system integration and the emphasis on data freshness. The authors are honest about focusing on systems rather than algorithms, and they cite the Pinterest/Alibaba/eBay work appropriately.

The soft spot is the evaluation. The abstract claims sub-second search over hundreds of billions of images, but the only query-latency experiment uses 100,000 images on 20 searcher nodes. The production section reports update latency, not query latency. The search path scans the nearest cluster's inverted list, so per-searcher cost grows with the list length. At 100 billion images spread over 20 searchers, each searcher would hold about 5 billion images—a million times more than the testbed's 5,000 per searcher. The paper gives no production searcher count, no number of clusters, and no list-length distribution, so the scaling argument is absent. The sub-second claim for the full index is an extrapolation, not a measurement.

There are also no retrieval quality metrics—no precision/recall, no business impact—so we cannot tell whether the system returns useful results, only that it returns results quickly on a small testbed. That is a separate gap but worth noting.

This is a credible engineering case study, not a scientific breakthrough. The right audience is systems practitioners who want to see how one company handled real-time indexing at scale. It deserves peer review because the claims are important and testable, but a referee should insist on either a production query-latency measurement or a clear scaling model, plus at least a baseline comparison or some quality metric. I would send it to a systems venue with major-revision expectations.

Best, [Your name]

Referee Report

3 major / 5 minor

Summary. The paper describes the design and implementation of JD.com's real-time visual search system. The system uses a three-tier architecture (blenders, brokers, searchers) to distribute search over partitioned image indexes, and a two-track indexing pipeline: periodic full indexing and incremental real-time indexing. The indexing design includes forward and inverted indexes, k-means-based inverted lists, a validity bitmap to reuse previously extracted features, and lock-free in-memory list expansion via copy-on-double. The authors report production operation data for August 4, 2018 (about 100 billion images, 977 million image updates that day, average update latency 132 ms) and a performance testbed evaluation with 100,000 images on 20 searcher nodes, reaching 1800 QPS with a 99th-percentile query response time of 0.3 s. The paper claims the system supports real-time visual search over hundreds of billions of images at sub-second timescales.

Significance. If the central scale claim were established, this would be a valuable systems/experience paper for the database and information-retrieval community. The reported design contains useful engineering ideas: reusing image features for products that leave and re-enter the market, a validity bitmap for cheap deletion, pre-allocated inverted lists with double-size replacement, and a hierarchical search architecture that separates ranking (blender), aggregation (broker), and partition-local search (searcher). The paper also provides production update-throughput and update-latency numbers, which are rarely published. However, the headline claim of sub-second search over hundreds of billions of images is not supported by the measurements in the paper: the only query-latency experiment uses 100,000 images, and the production data cover update latency, not query latency over the full index. The manuscript therefore needs additional evidence or a revised claim before the central contribution can be accepted.

major comments (3)

[Section 3.1/3.2, Abstract] Section 3.1 reports about 100 billion images in production, and Section 3.2 evaluates query scalability on a 100,000-image testbed with 20 searcher nodes. The abstract claims support for "hundreds of billions of product images at sub-second timescales," but no production query latency on the full 100-billion-image index is reported. Because Section 2.4 states that a searcher identifies the nearest cluster and scans that cluster's inverted list, per-searcher query cost grows with the list length; extrapolating from 5,000 images per searcher in the testbed to roughly 5 billion images per searcher at production scale is unsupported without additional evidence. Please provide production query latency measurements, the production searcher count, the number of inverted lists N, the list-length distribution, and a scaling argument or measurement showing that per-searcher scan cost is bounded.
[Section 3.2, last paragraph] The sentence "The results demonstrate the system's ability to support real time visual search with the real production system" overstates the evidence in the paper. Figure 13(b), the only query-response-time distribution, is from the 100,000-image testbed, not from the production 100-billion-image system. The production operation data in Section 3.1 and Figure 11(b) concern index-update latency (average 132 ms), which does not validate the sub-second query claim at full scale. The paper should either add full-scale production query latency measurements or revise the abstract and this sentence to state the tested scale explicitly.
[Sections 2.2-2.4, 3.2] Several parameters that determine the feasibility and reproducibility of the design are not given: the number of inverted lists N, the dimension of the extracted image features, the number of clusters in the k-means training set, the candidate count returned by each searcher, and the exact distance computation method. Without these, the per-searcher complexity of Section 2.4 cannot be analyzed or compared with the related systems in Section 4. Please add a parameter table and a complexity analysis, or a production measurement of these quantities.

minor comments (5)

[Abstract, Section 1] The phrase "deep natural network technology" should read "deep neural network technology."
[Section 3.1, Figure 11(b) caption] "perceptively" should be "respectively" in the discussion of 99th and 90th percentile latencies.
[Section 3.2, Figure 13(a)] The x-axis of Figure 13(a) is labeled "# of Threads" while the text describes concurrent users; please clarify what is being varied.
[Section 2.4] The notation alternates between "top k" and "top N" for the number of results returned by a searcher; please unify the notation.
[References] Several references contain formatting artifacts (for example, "arXiv preprint arXiv:1702.08734 (2017)" and the stray "16" and "1" near references 20 and 22); please clean up the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical systems report; the query-latency extrapolation gap is an evidence limitation, not a definitional or self-citation loop.

full rationale

This is a systems paper with no mathematical derivation chain to inspect. The abstract claim that the system can support sub-second visual search over hundreds of billions of product images is supported by two distinct bodies of evidence: production update volume and update latency in Section 3.1, and testbed query latency on 100,000 images in Section 3.2. The testbed result does not by construction entail the production-scale claim; it is an extrapolation whose validity depends on unstated production parameters such as searcher count and inverted-list length distributions. That is a correctness or evidence gap, not circularity, because no parameter was fitted to a target and then renamed as a prediction, and no load-bearing premise is justified only by a self-citation. The production metrics are self-reported by the deploying authors, but self-reporting is not circular reasoning. No step satisfies the required quoted-reduction test, so the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper's central claims rest on the accuracy of self-reported production telemetry, the perceptual validity of learned feature distances, and the approximation quality of k-means inverted lists. None of these are independently verified in the paper.

free parameters (2)

Number of inverted lists (clusters) in the index = Not reported
The number of clusters is produced by k-means on a training set (Section 2.2). The paper does not state the value or the tuning procedure, and retrieval quality and search latency depend on it.
Candidate count returned by each searcher = Not reported
Section 2.4 says each searcher returns the top N most similar images; the value is unspecified and affects both ranking quality and response time.

assumptions (3)

domain assumption Deep network image features are compared by Euclidean distance to rank visual similarity.
Section 2.4 states that similarity is computed as Euclidean distance in the high-dimensional feature space. The paper does not evaluate whether this distance corresponds to product-level visual similarity.
domain assumption The production metrics recorded on August 4, 2018 are accurate and representative of typical operation.
The system's claims rest on self-reported telemetry from a single day (Table 1, Figure 11), with no independent verification or multi-day statistics.
domain assumption Scanning the single nearest k-means cluster finds the most similar images, so the inverted index approximation does not hurt search quality.
Section 2.2 builds inverted lists by assigning each image to its nearest cluster, and Section 2.4 searches only the nearest cluster. The approximation error is not measured.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform." pith.science (2026). https://pith.science/paper/EVD3BRBT

@misc{pith2026190807389,
  author       = {Pith},
  title        = {Pith review of: The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EVD3BRBT}},
  note         = {Machine review of arXiv:1908.07389}
}

read the original abstract

We present the design and implementation of a visual search system for real time image retrieval on JD.com, the world's third largest and China's largest e-commerce site. We demonstrate that our system can support real time visual search with hundreds of billions of product images at sub-second timescales and handle frequent image updates through distributed hierarchical architecture and efficient indexing methods. We hope that sharing our practice with our real production system will inspire the middleware community's interest and appreciation for building practical large scale systems for emerging applications, such as ecommerce visual search.

Figures

Figures reproduced from arXiv: 1908.07389 by the authors.

**Figure 14.** Real Search Examples on a Mobile Application [PITH_FULL_IMAGE:figures/full_fig_p006_14.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

AddrLLM: Address Rewriting via Large Language Model on Nationwide Logistics Data
cs.CL 2024-11 conditional novelty 6.0 of 10

An LLM fine-tuned on 60 million logistics address examples, with nearby-address retrieval and reinforcement learning from JD's geocoding service, rewrites abnormal Chinese addresses better than prior geocoding and rew...

Reference graph

Works this paper leans on

10 extracted references · 9 canonical work pages · cited by 1 Pith paper

[1]

Each server has 24 CPU cores and 256GB memory

Number of Image Updates on 8/4/2018 workload generation, 1 Nginx server as the front end, 6 servers as blenders and brokers, and 20 servers as searchers. Each server has 24 CPU cores and 256GB memory. A total of 100,000 images are used. The client machine emulates a different number of concurrent users by sending image query requests to the visual search ...

work page 2018
[5]

Billion-scale similarity search with gpus

Johnson, Jeff, Matthijs Douze, and Hervé Jégou. "Billion-scale similarity search with gpus." arXiv preprint arXiv:1702.08734 (2017)

arXiv 2017
[6]

Earlybird: Real-time search at twitter

Busch, Michael, et al. "Earlybird: Real-time search at twitter." Data Engineering (ICDE), 2012 IEEE 28th International Conference on. IEEE,

work page 2012
[7]

A study on different image retrieval techniques in image processing

Ahmed, Gul^ishan Firdose, and Raju Barskar. "A study on different image retrieval techniques in image processing." International Journal of Soft Computing and Engineering 1.4 (2011): 247-251

work page 2011
[9]

Table 1 summarizes the operation data on August 4, 2018 collected from the production environment

Inverted List Expansion 5 users every day. Table 1 summarizes the operation data on August 4, 2018 collected from the production environment. The system contains about 100 billion images and their feature indexes. The visual search system processed a total of 977 millions of image updates on that day. Among them, 315 millions were attribute updates, 521 m...

work page 2018
[10]

Building a web-scale image similarity search system

Batko, Michal, et al. "Building a web-scale image similarity search system." Multimedia Tools and Applications 47.3 (2010): 599-629

work page 2010
[11]

Operation Data on August 4, 2018 020406080100 Time…0:001:002:003:004:005:006:007:008:009:0010:0011:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:0022:00 Millions Time (hour) Number of Real Time Index UpdatesUpdateDeletionAdditionTotal 050010001500200025000:001:002:003:004:005:006:007:008:009:0010:0011:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:...

work page 2018
[12]

Visualrank: Applying pagerank to large-scale image search

Jing, Yushi, and Shumeet Baluja. "Visualrank: Applying pagerank to large-scale image search." IEEE Transactions on Pattern Analysis and Machine Intelligence 30.11 (2008): 1877-1890

work page 2008

Show all 10 references

[13]

Query Performance Scalability 1200 1300 1400 1500 1600 1700 1800 1900 1357911131517192123252729313335 # of Threads QPS 0 0.2 0.4 0.6 0.8 1 0 0.51 1.52 2.5Response Time (seconds) CDF of the Response Time Total Attribute Update Image Addition Image Deletion 977 million 315 milli...

1900
[17]

Visualrank: Applying pagerank to large-scale image search,

Jing, Yushi, and Shumeet Baluja. "Visualrank: Applying pagerank to large-scale image search," IEEE Transactions on Pattern Analysis and Machine Intelligence 30.11 (2008): 1877-1890

2008

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Each server has 24 CPU cores and 256GB memory

Number of Image Updates on 8/4/2018 workload generation, 1 Nginx server as the front end, 6 servers as blenders and brokers, and 20 servers as searchers. Each server has 24 CPU cores and 256GB memory. A total of 100,000 images are used. The client machine emulates a different number of concurrent users by sending image query requests to the visual search ...

work page 2018

[2] [5]

Billion-scale similarity search with gpus

Johnson, Jeff, Matthijs Douze, and Hervé Jégou. "Billion-scale similarity search with gpus." arXiv preprint arXiv:1702.08734 (2017)

arXiv 2017

[3] [6]

Earlybird: Real-time search at twitter

Busch, Michael, et al. "Earlybird: Real-time search at twitter." Data Engineering (ICDE), 2012 IEEE 28th International Conference on. IEEE,

work page 2012

[4] [7]

A study on different image retrieval techniques in image processing

Ahmed, Gul^ishan Firdose, and Raju Barskar. "A study on different image retrieval techniques in image processing." International Journal of Soft Computing and Engineering 1.4 (2011): 247-251

work page 2011

[5] [9]

Table 1 summarizes the operation data on August 4, 2018 collected from the production environment

Inverted List Expansion 5 users every day. Table 1 summarizes the operation data on August 4, 2018 collected from the production environment. The system contains about 100 billion images and their feature indexes. The visual search system processed a total of 977 millions of image updates on that day. Among them, 315 millions were attribute updates, 521 m...

work page 2018

[6] [10]

Building a web-scale image similarity search system

Batko, Michal, et al. "Building a web-scale image similarity search system." Multimedia Tools and Applications 47.3 (2010): 599-629

work page 2010

[7] [11]

Operation Data on August 4, 2018 020406080100 Time…0:001:002:003:004:005:006:007:008:009:0010:0011:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:0022:00 Millions Time (hour) Number of Real Time Index UpdatesUpdateDeletionAdditionTotal 050010001500200025000:001:002:003:004:005:006:007:008:009:0010:0011:0012:0013:0014:0015:0016:0017:0018:0019:0020:0021:...

work page 2018

[8] [12]

Visualrank: Applying pagerank to large-scale image search

Jing, Yushi, and Shumeet Baluja. "Visualrank: Applying pagerank to large-scale image search." IEEE Transactions on Pattern Analysis and Machine Intelligence 30.11 (2008): 1877-1890

work page 2008