REVIEW 9 cited by
Scaling New Frontiers: Insights into Large Recommendation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recommendation systems are essential for filtering data and retrieving relevant information across various applications. Recent advancements have seen these systems incorporate increasingly large embedding tables, scaling up to tens of terabytes for industrial use. However, the expansion of network parameters in traditional recommendation models has plateaued at tens of millions, limiting further benefits from increased embedding parameters. Inspired by the success of large language models (LLMs), a new approach has emerged that scales network parameters using innovative structures, enabling continued performance improvements. A significant development in this area is Meta's generative recommendation model HSTU, which illustrates the scaling laws of recommendation systems by expanding parameters to thousands of billions. This new paradigm has achieved substantial performance gains in online experiments. In this paper, we aim to enhance the understanding of scaling laws by conducting comprehensive evaluations of large recommendation models. Firstly, we investigate the scaling laws across different backbone architectures of the large recommendation models. Secondly, we conduct comprehensive ablation studies to explore the origins of these scaling laws. We then further assess the performance of HSTU, as the representative of large recommendation models, on complex user behavior modeling tasks to evaluate its applicability. Notably, we also analyze its effectiveness in ranking tasks for the first time. Finally, we offer insights into future directions for large recommendation models. Supplementary materials for our research are available on GitHub at https://github.com/USTC-StarTeam/Large-Recommendation-Models.
Forward citations
Cited by 9 Pith papers
-
From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction
FAT specializes attention by semantic field and reports +0.51% AUC over baselines on Taobao data, but its power-law scaling law is an empirical fit, not a derived prediction.
-
FuXi-\beta: Towards a Lightweight and Fast Large-Scale Generative Recommendation Model
FuXi-β shows that removing query-key attention and using a functional relative time bias makes generative recommendation Transformers faster and, on industrial datasets, more accurate.
-
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
RankMixer scales an industrial ranking model to 1B dense parameters with 10x MFU improvement and unchanged latency, gaining 1.08% in app duration in Douyin A/B tests.
-
Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent
TAIRA, a thought-pattern-augmented multi-agent recommender, outperforms prior LLM agents in simulated interactive recommendation, with the largest gains on complex user intents.
-
TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation
TD3 factorizes a synthetic sequence summary into user, time, item, and core factors via Tucker decomposition, and trains recommenders on this summary with a feature-alignment meta-objective.
-
Why Thinking Hurts: Diagnosing and Rectifying Linguistic Inertia in Large Language Models for Recommendation
Chain-of-thought reasoning degrades semantic-ID recommendation accuracy through 'linguistic inertia,' and a training-free compression-plus-contrastive decoding fix restores and often improves accuracy.
-
DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction
DLF is a CTR prediction architecture that combines low-rank, high-rank, and implicit interaction blocks with layer-wise attention fusion, reporting state-of-the-art results on Criteo, Avazu, Movielens, and Frappe.
-
FuXi-$\alpha$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
FuXi-alpha, a sequential recommender with decoupled temporal, positional, and semantic attention channels plus a two-stage FFN, reports gains over HSTU and positive online engagement results.
-
Climber: Toward Efficient Scaling Laws for Large Recommendation Models
Climber reports that splitting user sequences by behavior type, adding adaptive temperature, and co-designed batching enable more efficient Transformer scaling in recommender systems.
Discussion (0). Continue with ORCID to comment.