{"id":"3e18b72d-8234-4bd4-86f2-0b4196e1eff3","arxiv_id":"2412.06069","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A fuzzy type-2 clustering codebook construction for NEQ improves recall on Netflix, Audio, and Cifar60k by a few points at similar runtime, though without released code or recall error bars.","lead":"This paper replaces the clustering step in norm-explicit product quantization with a fuzzy type-2 clustering routine fused by a Sugeno integral. The authors report higher recall at roughly unchanged running time on three recommender benchmarks, but the supporting experimental details are incomplete.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (12) leaves the Sugeno fuzzy measure g unspecified, so the fusion step that produces the crisp codebook is not a well-defined procedure; the reported recall gains cannot currently be reproduced or attributed to the proposed method.","rationale":"The reader's weakest assumption correctly identifies the undefined fuzzy measure g in Eq. (12) as a central gap. I agree that the fusion rule is not fully defined and that this blocks independent confirmation. I would place even more weight on the procedural consequence: because the Sugeno integral is not a uniquely defined operation without g, Algorithm 2 does not currently specify a deterministic mapping from fuzzy cluster outputs to a crisp codebook. That makes the experimental comparison impossible to reproduce as written, which is a stronger reason for a conditional verdict than the separate Lloyd-optimality concern. The paper does provide internal consistency: Table II's differences are roughly aligned with the abstract's +6%/+5%/+8% claims, and Table IV shows Fuzzy-2 NEQ training time near PQ. Those are real points in its favor. However, no code is released, no recall error bars are reported, and the missing NEQ row in Table IV weakens the running-time claim. A conditional verdict remains appropriate: the method may be valid, but the manuscript must specify the aggregation and provide reproducible experiments before the central claim can be accepted.","tokens_in":19007,"tokens_out":6799,"duration_ms":66613,"concrete_test":"Obtain from the authors a fully specified aggregation, or fix one: for each cluster index i, set h(i) to the normalized IT2FPCM centroid and g to the Sugeno lambda-measure induced by the interval memberships; then rerun the Netflix 8-CB/32-Cl and 128-CB/100-Cl settings from Table VI under the Table II protocol. If the resulting Fuzzy-2 NEQ recall versus NEQ falls within the 10-iteration bootstrapped variability, or the aggregation must be re-tuned per dataset to recover the gain, then the reported +6% advantage is an artifact of the unspecified g rather than a property of the fuzzy codebook.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Algorithm 2, Step 3: fusing the IT2FPCM fuzzy codebooks with the Sugeno integral to obtain a crisp codebook. This step is not actually defined. Eq. (12) writes an integral of h over codebooks with respect to a fuzzy measure g, but g is never specified, h is only called 'cluster centers', and the statement that 'product has been used as a t-norm of Sugeno integral' is inconsistent with the standard Sugeno integral, which uses min/max. The discrete form of the Sugeno integral requires a set function g on the cluster index set; different choices of g give different aggregated codewords and therefore different recall. Without g, a reimplementation cannot know what Fuzzy-2 NEQ is, and the Table II recall advantages over NEQ, PQ and RQ cannot be attributed to a well-defined algorithm. This is the load-bearing gap: even if the fuzzy clustering is a good codebook learner, the claimed fusion mechanism is unspecified. A supporting issue is that Table IV omits NEQ running time, although the abstract claims near-equal time to NEQ.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Fuzzy-2 NEQ, a variant of Norm-Explicit Product Quantization in which the vector codebooks are trained with Interval Type-2 Fuzzy Possibilistic C-Means and the resulting fuzzy codebooks are aggregated with a Sugeno integral. The authors report recall improvements over PQ, OPQ, AQ, RQ, NormRQ, APQ, Simple Hash, and NEQ on Netflix, Audio, and Cifar60k, while claiming that the running time and complexity remain close to the most efficient existing PQ method. The manuscript includes the NEQ background, the proposed two-step codebook construction, experimental tables (Tables II-VI), and an appendix describing the genetic-algorithm tuning of the two fuzzy exponents.","tokens_in":19237,"tokens_out":6458,"duration_ms":63197,"significance":"If the proposed fuzzy codebook construction were fully specified and reproducible, it would offer a low-cost way to replace hard K-means codebooks in MIPS-oriented product quantization, which is a relevant direction for large-scale recommender systems. The paper deserves credit for comparing against seven baselines on three benchmark datasets and for reporting precision/F1 in addition to recall. However, the central aggregation mechanism is currently not defined precisely enough to be implemented, and the experimental evaluation does not provide a valid uncertainty measure for the claimed recall gains. As written, the contribution is therefore not yet verifiable.","major_comments":[{"comment":"Equation (12) does not define the Sugeno integral used in Algorithm 2, Step 3. The fuzzy measure g is never specified, h is only described as \"cluster centers,\" and the domain and range of the integral are unclear. The standard discrete Sugeno integral requires a set function g on the index set of the fuzzy codebooks, and different choices of g will produce different aggregated crisp codewords and therefore different recall values. The statement that \"the product has been used as a t-norm of Sugeno integral\" is also inconsistent with the standard min-max formulation of the Sugeno integral. Consequently, the step that converts the fuzzy codebooks into the crisp codebook consumed by the rest of NEQ is not implementable as written, and the reported recall improvements cannot yet be reproduced or attributed to the proposed method.","section":"Section IV-B, Eq. (12), and Algorithm 2"},{"comment":"The abstract's headline numbers are selected as the largest values in Table II (94.65 for Netflix at 16,384 items, 69.71 for Audio at 32,768 items, and 59.34 for Cifar60k at 32,768 items), so they do not describe a single experimental setting, and the baseline corresponding to each of the claimed '+6%, +5%, +8%' gains is not clearly indicated. Table IV is described in the text as reporting the standard deviation of recall, but the table actually reports only time standard deviations, and Table II contains no error bars for recall. Without an uncertainty measure, the claimed advantages over NEQ, PQ, and RQ are not statistically supported.","section":"Section V, Tables II and IV"},{"comment":"The fuzzy exponents ξ1=8.5 and ξ2=9.1 are tuned with a genetic algorithm on the same benchmark datasets that are later used for the reported recall, and the number of codebooks and clusters is also chosen per benchmark. Because these parameters directly determine the fuzzy partition and hence the aggregated codebook, this tuning protocol does not establish that the method generalizes. A held-out tuning split or a sensitivity analysis over the parameter grid shown in Figure 7 is needed before the improvements can be attributed to the fuzzy mechanism rather than to dataset-specific hyperparameter selection.","section":"Section V and Appendix B"},{"comment":"The abstract claims that the computing time and complexity of Fuzzy-2 NEQ \"nearly equals\" the most computationally efficient existing PQ method, but Table IV contains no NEQ row, and Table V reports CPU and memory usage for NEQ rather than running time. The method used as the reference point for the time claim is therefore not actually compared in the timing table. Please report the training and query time of NEQ alongside the other baselines, or temper the complexity claim.","section":"Section V, Tables IV and V"}],"minor_comments":[{"comment":"The precision formula divides by \"All Relevant Documents,\" which is the same denominator used for recall in Eq. (13); precision should divide by the number of retrieved documents.","section":"Section V-A, Eq. (14)"},{"comment":"The second loop is written as \"while s = m′+1 to s = m,\" which is not a well-formed while condition; it should be a for loop, and the return statement should make clear that l·r is computed after both loops terminate.","section":"Algorithm 1"},{"comment":"Several related-work citations do not match the methods they are meant to support: [15] is not the original product-quantization paper of Jegou et al., and [16] does not correspond to the OPQ work described in the text; the same issue affects the references for RQ and AQ. Please correct the bibliography so that the baseline descriptions can be verified.","section":"Section II and References [15]-[17]"},{"comment":"The table header is confusing: \"Average Time Std Time Std\" does not match the text's statement that the table shows the average standard deviation of recall. The header and caption should be aligned with the values actually tabulated.","section":"Section V, Table IV"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is valid and lands on the central contribution: the fusion via Sugeno integral in Algorithm 2 is not defined because the fuzzy measure g and the aggregation procedure are missing. I would ask the authors to provide either a precise pseudo-code with the full discrete Sugeno integral formula or the source code used in the experiments. The reference list also contains several citations that appear unrelated to the claimed prior work, which raises a reproducibility concern that the editor should ask the authors to address carefully."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read of arXiv:2412.06069. The core idea — replace K-means codebook training in Norm-Explicit Product Quantization with Interval Type-2 Fuzzy Possibilistic C-means and aggregate the fuzzy codebooks with a Sugeno integral — is a legit extension, and the reported recall improvements (a few points over NEQ/PQ/RQ on Netflix, Audio, Cifar60k at roughly unchanged runtime) would be useful if they hold. But as written, the method is not fully specified. The load-bearing step, Algorithm 2's 'aggregate using Sugeno integral,' depends on a fuzzy measure g that is never defined; Eq. (12) is not enough to pin down the aggregation, and the remark that 'product has been used as a t-norm' conflicts with the standard min/max Sugeno integral. A reimplementation cannot know what Fuzzy-2 NEQ is. That is the main soft spot.\n\nWhat I do credit: the application of soft clustering to codebook construction for NEQ is new relative to the cited work, the experiment includes multiple baselines and three datasets, and the runtime numbers suggest the fuzzy training overhead is small. The 10-iteration bootstrapping is a good instinct, though the paper never reports the recall standard deviations, only time standard deviations in Table IV.\n\nOther issues: the headline gains are best-case differences at the largest item counts, not typical operating points. The two fuzzy exponents are tuned by genetic algorithm on what appears to be the same benchmark data used for evaluation, which risks selection bias. And the reference list is seriously broken — [15], [16], [17], [18], and [12] do not point to the methods the text attributes to them (Jegou's PQ, OPQ, RQ, AQ, and Simple LSH are all mis-cited). That is a citation-integrity problem, not a style nit.\n\nWho is this for? People working on PQ-family MIPS might want to try fuzzy codebooks, and the negative lesson about evaluation rigor is also useful. The paper deserves a serious referee only after the authors specify g, add error bars, retune on a validation split, and fix citations. As is, I would not accept the numbers at face value. My recommendation: send it to review with a clear request for major revision, or desk-reject with an invitation to resubmit. If I were the editor, I'd lean toward the latter given the undefined method, but the underlying idea is not silly and could be made solid.","headline":"A plausible fuzzy-clustering extension of NEQ with modest reported recall gains, but an undefined Sugeno fusion step and broken references keep it from being reproducible as submitted.","tokens_in":19766,"tokens_out":4965,"would_cite":false,"duration_ms":46209,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that codebooks built from Interval Type-2 fuzzy clustering, fused by a Sugeno integral, improve the recall of norm-explicit product quantization for maximum inner product search on recommender benchmarks while keeping…","keywords":["product quantization","norm-explicit quantization","maximum inner product search","fuzzy clustering","type-2 fuzzy sets","Sugeno integral","recommender systems","recall"],"falsifier":"Run Fuzzy-2 NEQ on Netflix, Audio, and Cifar60k with the Sugeno-fused fuzzy codebooks replaced by K-means centroids while keeping every other algorithm step identical; if recall at the larger item counts does not drop, the reported gains are not caused by the fuzzy codebook construction.","tokens_in":18788,"feed_emoji":"🎯","tokens_out":6345,"duration_ms":57052,"temperature":0.7,"pith_summary":"The paper tries to establish that product quantization for maximum inner product search, used in recommender systems to find the highest-scoring user-item matches, can be made more accurate by building its codebooks with soft fuzzy clustering instead of hard K-means. Its proposed method, Fuzzy-2 NEQ, replaces K-means in Norm-Explicit Product Quantization with Interval Type-2 Fuzzy Possibilistic C-Means and fuses the resulting fuzzy codebooks with a Sugeno integral, so each sub-vector can belong to more than one codeword. On the Netflix, Audio, and Cifar60k benchmarks, it reports recall of 94%, 69%, and 59%, respectively, outperforming NEQ, PQ, and RQ by up to +6%, +5%, and +8% at the larger item settings. The authors argue that the gain matters because recall improves without a meaningful increase in running time, keeping the method suitable for high-traffic services where query speed and low cost are priorities. If the central claim is right, soft codebook construction is a low-cost way to push the recall of quantization-based recommender retrieval beyond what hard clustering achieves.","feed_headline":"Fuzzy codebooks lift recommendation recall up to 94%","feed_subtitle":"Soft clustering plus Sugeno fusion beats norm-explicit product quantization on three benchmarks with equal runtime.","key_machinery":"The load-bearing machinery is the fuzzy codebook plus fusion step. Interval Type-2 Fuzzy Possibilistic C-Means (IT2FPCM) is a soft clustering algorithm in which each data point has interval-valued memberships in several clusters, so the codebook codewords become type-2 fuzzy sets instead of hard centroids. The Sugeno integral, an ordinal fuzzy aggregation rule, combines those fuzzy codebooks into the single crisp codebook required by the rest of the norm-explicit pipeline. Around this, the method keeps NEQ's separation of each vector into a scalar norm and a unit direction, encoding the direction with the fuzzy-derived codebooks and the relative norm with separate scalar codebooks. The paper's claim is that this combination lowers quantization error at cluster boundaries and thereby raises recall per item retrieved.","core_discovery":"The central discovery, on the paper's own terms, is that fuzzifying the codebook construction step of Norm-Explicit Product Quantization yields better recall on maximum inner product search without sacrificing the method's efficiency. In Fuzzy-2 NEQ, the norm and the direction of each item vector are quantized separately, as in NEQ, but the direction codebooks are trained with a type-2 fuzzy possibilistic clustering algorithm rather than K-means. Because the clustering output is a fuzzy tensor rather than a crisp set of centroids, the method aggregates it with a Sugeno integral into a single crisp codebook before encoding. The paper reports that this procedure satisfies Lloyd's optimality conditions and, in experiments across Netflix, Audio, and Cifar60k, is statistically more accurate than NEQ, PQ, OPQ, RQ, Norm-RQ, AQ, APQ, and a simple locality-sensitive hashing baseline, while its running time stays close to that of plain PQ.","pith_inferences":["The paper does not state the fuzzy measure $g$ used in Eq. (12), so a natural next experiment would be to test whether the recall gain survives under different fuzzy measures or under a simpler weighted average; this is an editorial suggestion, not a result in the paper.","Because the fuzzy clustering step is independent of the norm-explicit decomposition, the same fuzzy codebook construction could plausibly be applied to other vector quantizers such as OPQ, RQ, or AQ; the paper only tests it inside NEQ.","The paper asserts that the Sugeno integral outperforms the Choquet integral for this fusion but reports no direct comparison; a head-to-head aggregation comparison would clarify whether the ordinal nature of Sugeno is essential.","If soft codebooks genuinely help because boundary vectors straddle clusters, then datasets with more cluster overlap should show larger gains; checking whether the gain magnitude correlates with cluster overlap would be a testable extension."],"forward_implications":["If the reported gains hold, recommender systems that already use product quantization can raise recall by swapping K-means for the fuzzy clustering and Sugeno fusion step, with essentially no change in query-time cost.","The method leaves the MIPS-to-nearest-neighbor transformation untouched, so it can be inserted into existing PQ-based retrieval pipelines without changing the distance computation.","At the largest tested settings, Fuzzy-2 NEQ's recall advantage appears without increasing the number of codebooks, implying the memory footprint of the compressed index stays comparable to PQ.","The recall improvements on Cifar60k (image embeddings) and Audio (voice embeddings) suggest the benefit is not specific to user-item rating data, if the paper's benchmark results are representative."],"supporting_citations":[{"why":"Defines Norm-Explicit Product Quantization, the baseline the paper fuzzifies and the main method it compares against.","marker":"[13]"},{"why":"Provides Interval Type-2 Fuzzy Possibilistic C-Means, the soft clustering algorithm used to build fuzzy codebooks.","marker":"[35]"},{"why":"Supplies the Lloyd optimality conditions that the fuzzy codebook construction is claimed to satisfy.","marker":"[38]"},{"why":"Introduces Product Quantization with K-means codebooks, the base retrieval scheme and one of the baselines.","marker":"[15]"},{"why":"Establishes the Euclidean transformation that makes MIPS equivalent to nearest-neighbor search, which the experiments inherit.","marker":"[34]"},{"why":"Defines the Sugeno integral used to fuse fuzzy codebooks into a crisp codebook for norm-explicit quantization.","marker":"[41]"}],"fun_headline_variants":["Fuzzy norm-explicit PQ hits 94% recall","Type-2 fuzzy codebooks boost PQ recall without slowdown","Fuzzy product quantization beats PQ, NEQ, RQ by up to 8%","Fuzzy codebooks lift recall on Netflix, Audio, CIFAR","Soft fuzzy norm quantization sharpens recommendations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole advantage rests on the premise that Sugeno-fused type-2 fuzzy clusters form a codebook that quantizes item vectors better than K-means centroids, and this premise is not independently established because the paper never specifies the fuzzy measure in the Sugeno integral.","fun_headline_variants_meta":{"raw":{"variants":["Fuzzy norm-explicit PQ hits 94% recall","Type-2 fuzzy codebooks boost PQ recall without slowdown","Fuzzy product quantization beats PQ, NEQ, RQ by up to 8%","Fuzzy codebooks lift recall on Netflix, Audio, CIFAR","Soft fuzzy norm quantization sharpens recommendations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000679,"raw_usage":{"total_tokens":3137,"prompt_tokens":1046,"completion_tokens":2091,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":2004}},"tokens_in":662,"tokens_out":2091,"duration_ms":14768,"temperature":1.0,"reasoning_tokens":2004,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:02:34.555297+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Fuzzy-2 NEQ on Netflix, Audio, and Cifar60k with the Sugeno-fused fuzzy codebooks replaced by K-means centroids while keeping every other algorithm step identical; if recall at the larger item counts does not drop, the reported gains are not caused by the fuzzy codebook construction.","supporting_citations":[{"cited_title":"Norm- explicit quantization: Improving vector quantization for maximum inner product search,","cited_arxiv_id":null,"evidence_quote":"Defines Norm-Explicit Product Quantization, the baseline the paper fuzzifies and the main method it compares against."},{"cited_title":"Interval type-2 fuzzy possibilistic c- means clustering algorithm,","cited_arxiv_id":null,"evidence_quote":"Provides Interval Type-2 Fuzzy Possibilistic C-Means, the soft clustering algorithm used to build fuzzy codebooks."},{"cited_title":"Quantization/clustering: when and why does k-means work?","cited_arxiv_id":null,"evidence_quote":"Supplies the Lloyd optimality conditions that the fuzzy codebook construction is claimed to satisfy."},{"cited_title":"Speeding up the xbox recommender system using a euclidean transformation for inner- product spaces,","cited_arxiv_id":null,"evidence_quote":"Establishes the Euclidean transformation that makes MIPS equivalent to nearest-neighbor search, which the experiments inherit."},{"cited_title":"Sugeno integral generalization applied to improve adaptive image binarization,","cited_arxiv_id":null,"evidence_quote":"Defines the Sugeno integral used to fuse fuzzy codebooks into a crisp codebook for norm-explicit quantization."}],"review_version":1}