REVIEW 3 cited by
External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.
Forward citations
Cited by 3 Pith papers
-
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
ROCS restructures recommendation models so user-side computation is shared across all candidate items, yielding up to 3x serving throughput at equal or better prediction quality.
-
Building a User Foundation Model for the Open Web
A self-supervised Transformer on short open-web browsing sequences improves production CTR and win-rate models and delivers +2.13% live CTR under RTB latency and privacy constraints.
-
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
A single pretrained model over user activity sequences improves save rates in Pinterest's Home Feed and Related Items ranking when fine-tuned per application, while deduplication and quantization keep serving costs neutral.
Discussion (0). Continue with ORCID to comment.