REVIEW 5 cited by
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Modeling ultra-long user behavior sequences is critical for capturing both long- and short-term preferences in industrial recommender systems. Existing solutions typically rely on two-stage retrieval or indirect modeling paradigms, incuring upstream-downstream inconsistency and computational inefficiency. In this paper, we present LONGER, a Long-sequence Optimized traNsformer for GPU-Efficient Recommenders. LONGER incorporates (i) a global token mechanism for stabilizing attention over long contexts, (ii) a token merge module with lightweight InnerTransformers and hybrid attention strategy to reduce quadratic complexity, and (iii) a series of engineering optimizations, including training with mixed-precision and activation recomputation, KV cache serving, and the fully synchronous model training and serving framework for unified GPU-based dense and sparse parameter updates. LONGER consistently outperforms strong baselines in both offline metrics and online A/B testing in both advertising and e-commerce services at ByteDance, validating its consistent effectiveness and industrial-level scaling laws. Currently, LONGER has been fully deployed at more than 10 influential scenarios at ByteDance, serving billion users.
Forward citations
Cited by 5 Pith papers
-
UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction
UniRank is an open benchmark that standardizes chronological autoregressive supervision, multi-task evaluation, and capacity controls for 15 unified ranking models on five large datasets.
-
Towards Comprehensible Recommendation with Large Language Model Fine-tuning
CURec aligns an LLM with recommendation objectives via RL, generates personalized user patterns and item-reason texts, and corrects them through a chronological loop, improving top-K recall and NDCG on MovieLens and Amazon.
-
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
RankMixer scales an industrial ranking model to 1B dense parameters with 10x MFU improvement and unchanged latency, gaining 1.08% in app duration in Douyin A/B tests.
-
Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models
DIVER trains an LLM with synthetic data plus multi-objective RL to emit several diverse, style-tagged ad headlines in one pass, reporting online ADVV and CTR gains.
-
Large Foundation Model for Ads Recommendation
Tencent's LFM4Ads transfers user, item, and user-item cross representations from a pre-trained foundation model into downstream ad models via feature, module, and model-level mechanisms, reporting a 2.45% platform-wid...
Discussion (0). Continue with ORCID to comment.