The reviewed record of science sign in
Pith

arxiv: 2606.23291 · v1 · pith:4MQ62GH7 · submitted 2026-06-22 · cs.IR

URecJPQ: Memory-efficient Multimodal Recommendation Models through RecJPQ in Large-Scale Scenarios

Giuseppe Spillo , Zixuan Yi , Aleksandr Petrov , Cataldo Musto , Craig Macdonald , Iadh Ounis This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-26 06:47 UTCgrok-4.3pith:4MQ62GH7record.jsonopen to challenge →

classification cs.IR
keywords multimodal recommendationproduct quantizationmemory efficiencylarge-scale modelsembedding compressiontop-k rankingparameter reduction
0
0 comments X

The pith

URecJPQ represents users and items as concatenations of shared sub-embeddings to cut memory use in large multimodal recommendation models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that joint product quantization can be adapted for multimodal recommendation to address high memory demands from unique ID embeddings and additional modality features. A sympathetic reader would care because industrial datasets with millions of users and items often exceed available memory, limiting model training and deployment. By replacing full embeddings with concatenations of shared sub-embeddings, the method reduces checkpoint sizes by 86 to 98 percent and trainable parameters by 98 to 99 percent. Experiments across movies, baby products, and sports domains show only average accuracy losses of 8.5 percent on recall and 16 percent on NDCG, with occasional gains up to 85 percent.

Core claim

URecJPQ is a joint product quantization method for large-scale multimodal top-k recommendation where each user and item is encoded not as a unique full embedding but as a concatenation of shared learned sub-embeddings, leading to major reductions in memory and parameters with limited impact on ranking quality.

What carries the argument

The RecJPQ joint product quantization that encodes users and items via concatenations of shared sub-embeddings instead of unique full embeddings.

If this is right

  • Multimodal item features can be added to recommendation models without a proportional increase in memory requirements.
  • Training on large industrial datasets becomes possible with fewer hardware resources.
  • Recommendation accuracy remains competitive, with potential improvements in specific domains such as baby products.
  • Checkpoint sizes are reduced enough to enable more frequent model updates or larger scale experiments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar quantization could be applied to other embedding-based tasks beyond recommendation, such as retrieval or personalization systems.
  • Combining this with other efficiency techniques like pruning might yield even greater savings.
  • The approach may support adding more modalities in future models without hitting memory walls.
  • Performance gains in certain domains suggest that the method could act as a regularizer in some cases.

Load-bearing premise

The shared sub-embeddings retain enough unique signal from the original embeddings to support accurate top-k item ranking.

What would settle it

A test on a fourth large-scale dataset showing accuracy drops exceeding 20 percent on both recall and NDCG would indicate the method does not preserve sufficient signal in general.

read the original abstract

Training state-of-the-art recommendation models on large-scale industrial datasets can be a challenging task due to the high number of users and items which are typically represented through ID embeddings. Such embeddings typically require a large amount of memory resources, which are not always available. This problem is further exacerbated in multimodal recommendation, in which multimodal item features generally improve recommendation performance, but require more resources to encode. In this paper, we introduce URecJPQ, a Joint Product Quantization method specifically designed for large-scale and multimodal top-k recommendation tasks, in which the vast number of users and items, combined with the available modalities, further increases the memory demands for the computation. The core idea is to represent each user/item not as a fully learned, unique embedding, but rather as a concatenation of shared learned sub-embeddings, thereby significantly reducing the total number of trainable parameters. Our experiments on three widely-used datasets across different domains (movies, baby and sports products) show that URecJPQ can be effectively applied to multimodal recommendation settings. In large scale scenarios, we observe a substantial reduction in checkpoint sizes and the number of trainable parameters (ranging from 86% to 98%, and 98% to 99%, respectively), with only a marginal decrease in accuracy (8.5% on recall and 16% on NDCG, on average), and, in some cases, even performance improvements (up to 85%), as in the baby products domain. Our codebase is available at https://anonymous.4open.science/r/large_mmrecjpq-839B/README.md.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces URecJPQ, a Joint Product Quantization approach for large-scale multimodal top-k recommendation. It replaces unique user/item ID embeddings with concatenations of shared learned sub-embeddings to cut memory use. Experiments on three datasets (movies, baby products, sports products) report checkpoint size reductions of 86-98%, trainable parameter reductions of 98-99%, average accuracy drops of 8.5% recall and 16% NDCG, and occasional gains up to 85%.

Significance. If the empirical results hold under broader testing, the work addresses a practical bottleneck in scaling multimodal recommenders to industrial item cardinalities under memory limits. The open codebase is a clear strength for reproducibility.

major comments (2)
  1. [§4 (Experiments)] §4 (Experiments): the description of accuracy drops as 'marginal' (8.5% recall, 16% NDCG on average) is not supported by any reported standard deviations, number of runs, or statistical significance tests; without these, it is impossible to determine whether the observed differences fall within run-to-run variance.
  2. [§4.3 (domain-specific results)] §4.3 (domain-specific results): the central claim that URecJPQ 'can be effectively applied to multimodal recommendation settings' in large-scale scenarios rests on signal preservation via shared sub-embeddings, yet this is demonstrated only on three datasets whose modality densities and item distributions may not be representative; additional datasets with different characteristics (e.g., denser image features or larger scales) are needed to substantiate generalization.
minor comments (1)
  1. [Abstract] Abstract: the provided codebase link is anonymous; a permanent identifier should be supplied upon acceptance.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below, agreeing where revisions are warranted and providing substantive responses on the others.

read point-by-point responses

  1. Referee: [§4 (Experiments)] the description of accuracy drops as 'marginal' (8.5% recall, 16% NDCG on average) is not supported by any reported standard deviations, number of runs, or statistical significance tests; without these, it is impossible to determine whether the observed differences fall within run-to-run variance.

    Authors: We agree this is a valid concern. The manuscript reports average performance across the three datasets without variance estimates or multiple runs. In revision we will rerun all experiments using at least five random seeds, report means with standard deviations, and add paired statistical significance tests (e.g., Wilcoxon) to establish whether the observed drops fall within run-to-run variance. revision: yes

  2. Referee: [§4.3 (domain-specific results)] the central claim that URecJPQ 'can be effectively applied to multimodal recommendation settings' in large-scale scenarios rests on signal preservation via shared sub-embeddings, yet this is demonstrated only on three datasets whose modality densities and item distributions may not be representative; additional datasets with different characteristics (e.g., denser image features or larger scales) are needed to substantiate generalization.

    Authors: The three datasets (MovieLens-25M, Amazon-Baby, Amazon-Sports) are standard large-scale benchmarks spanning different domains, item cardinalities (tens to hundreds of thousands), and modality densities (image + text features with varying sparsity). We will expand §4.3 with a table quantifying modality density, item distribution statistics, and scale for each dataset to strengthen the representativeness argument. While new datasets would be desirable, the current selection already covers the key axes mentioned; we therefore treat the request for entirely new data as a limitation rather than a requirement for the current claims. revision: partial

Circularity Check

0 steps flagged

No circularity: purely empirical method proposal and evaluation

full rationale

The paper introduces URecJPQ as a practical compression technique using Joint Product Quantization to replace unique ID embeddings with concatenations of shared sub-embeddings. All central claims (memory reduction of 86-98%, parameter reduction of 98-99%, and accuracy changes) are supported exclusively by direct experimental results on three datasets. There is no derivation chain, no equations that define a quantity in terms of itself, no fitted parameters renamed as predictions, and no load-bearing self-citations that justify uniqueness or ansatzes. The work is self-contained as an empirical engineering contribution whose validity rests on the reported runs rather than any internal reduction to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; the method implicitly assumes that sub-embedding sharing does not destroy ranking signal, but this is an empirical claim rather than an axiom.

pith-pipeline@v0.9.1-grok · 5840 in / 1143 out tokens · 23885 ms · 2026-06-26T06:47:52.450908+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 5 canonical work pages · 3 internal anchors

  1. [1]

    Proceedings of the 17th ACM International Conference on Web Search and Data Mining , pages=

    RecJPQ: training large-catalogue sequential recommenders , author=. Proceedings of the 17th ACM International Conference on Web Search and Data Mining , pages=

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Recommender systems with generative retrieval , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    IEEE Transactions on Big Data , volume=

    Billion-scale similarity search with GPUs , author=. IEEE Transactions on Big Data , volume=. 2019 , publisher=

    2019

  4. [4]

    Proceedings of the 18th ACM Conference on Recommender Systems , pages=

    Better generalization with semantic ids: A case study in ranking for recommendations , author=. Proceedings of the 18th ACM Conference on Recommender Systems , pages=

  5. [5]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=

    Soundstream: An end-to-end neural audio codec , author=. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=. 2021 , publisher=

    2021

  6. [6]

    arXiv preprint arXiv:2508.05198 , year=

    Balancing Accuracy and Novelty with Sub-Item Popularity , author=. arXiv preprint arXiv:2508.05198 , year=

    work page arXiv

  7. [7]

    Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Multi-modal graph contrastive learning for micro-video recommendation , author=. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  8. [8]

    ACM Transactions on Recommender Systems , volume=

    Enhancing recommender systems: Deep modality alignment with large multi-modal encoders , author=. ACM Transactions on Recommender Systems , volume=. 2025 , publisher=

    2025

  9. [9]

    Proceedings of the 5th ACM International Conference on Multimedia in Asia Workshops , pages=

    Mmrec: Simplifying multimodal recommendation , author=. Proceedings of the 5th ACM International Conference on Multimedia in Asia Workshops , pages=

  10. [10]

    IEEE Assp Magazine , volume=

    Vector quantization , author=. IEEE Assp Magazine , volume=. 1984 , publisher=

    1984

  11. [11]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Product quantization for nearest neighbor search , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2010 , publisher=

    2010

  12. [12]

    Proceedings of the 30th ACM International Conference on Information & Knowledge Management , pages=

    Jointly optimizing query encoder and product quantization to improve retrieval performance , author=. Proceedings of the 30th ACM International Conference on Information & Knowledge Management , pages=

  13. [13]

    Proceedings of the 28th ACM international conference on information and knowledge management , pages=

    BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer , author=. Proceedings of the 28th ACM international conference on information and knowledge management , pages=

  14. [14]

    Session-based Recommendations with Recurrent Neural Networks

    Session-based recommendations with recurrent neural networks , author=. arXiv preprint arXiv:1511.06939 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    2018 IEEE international conference on data mining (ICDM) , pages=

    Self-attentive sequential recommendation , author=. 2018 IEEE international conference on data mining (ICDM) , pages=. 2018 , organization=

    2018

  16. [16]

    BPR: Bayesian Personalized Ranking from Implicit Feedback

    BPR: Bayesian personalized ranking from implicit feedback , author=. arXiv preprint arXiv:1205.2618 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    IEEE Transactions on Multimedia , volume=

    Self-supervised learning for multimedia recommendation , author=. IEEE Transactions on Multimedia , volume=. 2022 , publisher=

    2022

  18. [18]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    VBPR: visual bayesian personalized ranking from implicit feedback , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  19. [19]

    BIT Numerical Mathematics , volume=

    The truncated SVD as a method for regularization , author=. BIT Numerical Mathematics , volume=. 1987 , publisher=

    1987

  20. [20]

    Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders

    Bridging Language and Items for Retrieval and Recommendation , author=. arXiv preprint arXiv:2403.03952 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Proceedings of the 29th ACM international conference on multimedia , pages=

    Mining latent structures for multimedia recommendation , author=. Proceedings of the 29th ACM international conference on multimedia , pages=

  22. [22]

    arXiv preprint arXiv:2302.04473 , year=

    A comprehensive survey on multimodal recommender systems: Taxonomy, evaluation, and future directions , author=. arXiv preprint arXiv:2302.04473 , year=

    work page arXiv

  23. [23]

    Communications of the ACM , volume=

    Recommender systems , author=. Communications of the ACM , volume=. 1997 , publisher=

    1997

  24. [24]

    The adaptive web: methods and strategies of web personalization , pages=

    Collaborative filtering recommender systems , author=. The adaptive web: methods and strategies of web personalization , pages=. 2007 , publisher=

    2007

  25. [25]

    modality-based recommender models revisited , author=

    Where to go next for recommender systems? id-vs. modality-based recommender models revisited , author=. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  26. [26]

    Expert Systems with Applications , volume=

    An embedding and interactions learning approach for ID feature in deep recommender system , author=. Expert Systems with Applications , volume=. 2022 , publisher=

    2022

  27. [27]

    Advances in Neural Information Processing Systems , volume=

    Transformer memory as a differentiable search index , author=. Advances in Neural Information Processing Systems , volume=

  28. [28]

    Proceedings of the web conference 2020 , pages=

    Lightrec: A memory and search-efficient recommender system , author=. Proceedings of the web conference 2020 , pages=

    2020

  29. [29]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

    Additive quantization for extreme vector compression , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

  30. [30]

    Proceedings of the 22nd ACM international conference on Information & Knowledge Management , pages=

    Learning deep structured semantic models for web search using clickthrough data , author=. Proceedings of the 22nd ACM international conference on Information & Knowledge Management , pages=

  31. [31]

    Proceedings of the 31st ACM international conference on multimedia , pages=

    A tale of two graphs: Freezing and denoising graph structures for multimodal recommendation , author=. Proceedings of the 31st ACM international conference on multimedia , pages=

  32. [32]

    Advances in Neural Information Processing Systems , volume=

    MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers , author=. Advances in Neural Information Processing Systems , volume=

  33. [33]

    International Conference on Learning Representations (ICLR) , year=

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , author=. International Conference on Learning Representations (ICLR) , year=

  34. [34]

    fm-2k, and DBbook with multimodal data , author=

    See the movie, hear the song, read the book: Extending movielens-1m, last. fm-2k, and DBbook with multimodal data , author=. Proceedings of the Nineteenth ACM Conference on Recommender Systems , pages=

  35. [35]

    , author=

    Deep matrix factorization models for recommender systems. , author=. IJCAI , volume=. 2017 , organization=

    2017

  36. [36]

    Proceedings of the 26th international conference on world wide web , pages=

    Neural collaborative filtering , author=. Proceedings of the 26th international conference on world wide web , pages=