REVIEW 7 cited by
Beyond Unimodal Boundaries: Generative Recommendation with Multimodal Semantics
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Generative recommendation (GR) has become a powerful paradigm in recommendation systems that implicitly links modality and semantics to item representation, in contrast to previous methods that relied on non-semantic item identifiers in autoregressive models. However, previous research has predominantly treated modalities in isolation, typically assuming item content is unimodal (usually text). We argue that this is a significant limitation given the rich, multimodal nature of real-world data and the potential sensitivity of GR models to modality choices and usage. Our work aims to explore the critical problem of Multimodal Generative Recommendation (MGR), highlighting the importance of modality choices in GR nframeworks. We reveal that GR models are particularly sensitive to different modalities and examine the challenges in achieving effective GR when multiple modalities are available. By evaluating design strategies for effectively leveraging multiple modalities, we identify key challenges and introduce MGR-LF++, an enhanced late fusion framework that employs contrastive modality alignment and special tokens to denote different modalities, achieving a performance improvement of over 20% compared to single-modality alternatives.
Forward citations
Cited by 7 Pith papers
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Recommender systems are moving from raw IDs to semantic IDs, and the next step should be semantic planning that first predicts an exposure's purpose before choosing or generating content.
-
SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation
SynGR is a synergistic generative recommendation method that masks dominant-modal tokens and uses contrastive learning to capture cross-modal synergies, outperforming prior methods on Amazon Arts, Games, and Instruments.
-
Efficient Item ID Generation for Large-Scale LLM-based Recommendation
LLM-based recommenders can treat item IDs as single direct embeddings and decode in one step, with a two-level softmax for efficiency and quality matching or beating multi-token models.
-
Generative Recommendation with Semantic IDs: A Practitioner's Handbook
An open-source framework and ablation study showing which semantic-ID generative recommendation components actually matter, with results that challenge several standard defaults.
-
LLM-Based Generative Retrieval for Snapchat Content Recommendation
SnapLGR, a production LLM-based generative retrieval system for Snapchat short video, lifted View Time 0.37% and related engagement metrics in a 7-day A/B test, with offline ablation attributing most of the gain to de...
-
Stream-aware Side Adaptation for Large Pre-trained Multimodal Embedding Models in Sequential Recommendation
Stream-aware fusion (SHAF) and residual stream adapters (ReSA) stabilize deep side adaptation of frozen multimodal embedding models and improve sequential recommendation over standard side adapters.
-
Semantic IDs for Recommender Systems at Snapchat: Use Cases, Technical Challenges, and Design Choices
Semantic IDs with STE-trained multi-modal RQ-VAE and heuristic collision resolution improve Snapchat ranking and generative retrieval offline and in online A/B tests.
Discussion (0). Continue with ORCID to comment.