Pith. sign in

REVIEW 2 major objections 6 minor 20 references

DV365: Extremely Long User History Modeling at Instagram

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DV365 shows that offline multi-slicing of up to 70,000 user events into one embedding yields >0.4% ranking gains on top of production HSTU models.

desk verdict DV365's central deployment result is solid and important, but the paper overstates the evidence against order-aware long-history modeling. read the letter →

arxiv 2506.00450 v1 pith:P57LUBHH submitted 2025-05-31 cs.IR cs.LG

Wenhan Lyu , Devashish Tyagi , Yihang Yang , Ziwei Li , Ajay Somani , Karthikeyan Shanmugasundaram , Nikola Andrejevic , Ferdi Adeputra
show 4 more authors
Curtis Zeng Arun K. Singh Maxime Ransan Sagar Jain
This is my paper · ORCID
classification cs.IRcs.LG
keywords usermodelingrepresentationlearningembeddinglongsequencerecommendersystemmulti-slicingandsummarizationoffline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the expensive, hard-to-scale step of feeding extremely long user histories through an attentive sequence model can be replaced by an offline computation that is both cheap and surprisingly effective. It describes DV365, a user embedding trained once on up to 70,000 events (40,000 on average) of a user's interaction history, using a multi-slicing and summarization strategy: the history is split into dozens of sub-sequences by action type, watch time, and recency, each reduced to a pooled embedding, then compressed by a small neural encoder. The authors report that adding this embedding to production Instagram and Threads models, which already use the HSTU sequence encoder, improves ranking quality by more than 0.4% offline on average and increased Instagram App time spent by 0.7% across 15 product launches. If correct, the paper shows that a single 'foundation' user embedding can capture the stable part of a user's long-term interest and be shared across many downstream models at a fraction of the serving cost.

What carries the argument

The central object is the multi-slicing and summarization (MSS) strategy: a rule-based feature engineering method that converts a user's raw interaction timeline into roughly 200 sub-sequences, each filtered by a criterion such as action type (like, share, comment), watch-time bucket, watch-ratio bucket, or recency window (3 days, 7 days, full horizon), and then reduces each sub-sequence by mean-pooling or weighted mean-pooling into a single token embedding. These pooled token embeddings are transposed to a token-wise view and compressed by a Funnel Summarization Arch (FSA), a Funnel Transformer whose pooling operates on the token dimension, plus a parallel linear compression encoder, yielding 58 embeddings of dimension 256. The user encoder is trained inside a DLRM backbone simulation network with multi-task ranking losses and a distant-interest objective that removes the most recent 24 hours of history from the input. This machinery carries the argument because it makes 40,000-event histories tractable offline: the expensive computations happen once per user on a recurring schedule, and the compressed embedding is quantized to 4-bit, stored for 3 billion users, and served as a lightweight feature to many downstream models.

What would settle it

Train an HSTU (or similarly attentive) encoder directly on the full 35,000-step explicit and implicit timelines, or on a distilled version of the same data, and compare its offline NE and downstream online metrics against the DV365 embedding on the same Reels ranking tasks; if the attentive model matches or beats DV365's reported >0.4% average NE gain, the claim that multi-slicing summarization is sufficient would be falsified.

Watch

Extended reading notes

Core claim

DV365 establishes that an offline user embedding built from count-based, multi-sliced summarization of extremely long user histories delivers significant incremental quality on top of production recommendation models that already encode shorter histories with an attentive sequence encoder (HSTU). The central discovery is that once the history is sliced along action type, dwell-time, watch-ratio, and time-bucket dimensions and each slice is mean-pooled, a compact neural summarizer (Funnel Summarization Arch) can condense 200 pooled embeddings into 58 quantized embeddings that downstream rankers and retrievers can consume as features. Trained with a backbone simulation network and a 'distant interest' objective that removes the most recent 24 hours of history, the embedding is designed to capture stable interest that is robust to staleness. The paper reports that this single upstream embedding, launched in 15 models, improved average offline NE by over 0.4% in ranking, increased retrieval hit rates by 2–8%, and contributed a 0.7% increase in Instagram App time spent.

Load-bearing premise

The design assumes that count-based mean-pooling over carefully chosen slices of the long history captures essentially all of the incremental signal, so that an order-aware attentive encoder over the full 35,000-step sequence would add little beyond what DV365 already gets.

Editorial extensions

If this is right

  • Adding DV365 to production ranking models that already use HSTU improves average offline NE by over 0.4%, with a linear projection as the adopted adaptation.
  • Retrieval models see hit-rate improvements of 2% to 8% across tasks when the embedding is integrated with a gating network adapter.
  • The embedding is robust to staleness: fixed embeddings seven days old retain roughly the same NE gain as freshly updated ones, with cosine similarity above 90% between versions a week apart.
  • A single upstream embedding, refreshed roughly every six hours, can serve 15 heterogeneous downstream models across Instagram and Threads, with an accumulated 0.7% increase in Instagram App time spent.
  • The offline approach avoids an estimated 100 MW of real-time feature-extraction power and 11 PB of low-latency key-value storage that end-to-end long-sequence modeling would require.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication left implicit by the paper is that for stable long-term interest, counting statistics over well-chosen slices may carry most of the signal that attentive sequence models extract from short histories, so similar offline pooling embeddings could be built for other platforms without scaling attention to tens of thousands of steps.
  • Because the 24-hour-gap objective and staleness results imply the embedding changes slowly, a testable extension is to use DV365-style embeddings for cold-start or infrequent users, combining the stable-interest signal with freshly computed short-term signals.
  • The 200 hand-selected slicing criteria were chosen by exhaustive offline experiments, so an automated or learned feature-selection process over the slice space could yield a data-driven variant that adapts when content distributions shift.
  • The reported 0.7% App-time gain is an aggregate over 15 launches, so a natural follow-up is to decompose the gain by surface, such as Reels, Feed, Explore, and Threads, to guide where further long-history investment pays off.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper describes DV365, an offline user-embedding system at Instagram that encodes extremely long user histories (up to 70,000 events, average 40,000) via a multi-slicing and summarization (MSS) strategy. Explicit and implicit timelines are sliced by action type, watch-time/ratio thresholds, and time buckets; the resulting roughly 200 pooled categorical features are compressed by a Funnel Summarization Architecture and served as a quantized embedding to 15 downstream ranking and retrieval models. The authors report average offline NE improvements over 0.4% in ranking, hit-rate gains of 2-8% in retrieval, and a cumulative 0.7% increase in Instagram app time across launches. They argue that offline embedding avoids the training/serving and feature-infrastructure costs of end-to-end long-sequence modeling.

Significance. If the reported gains are reliable, the paper is a valuable industrial demonstration that extremely long user histories can be exploited at low serving cost: one upstream foundation model serves many heterogeneous downstream models, with a concrete cost model (100 MW feature-extraction saving, 11 PB storage saving, and an estimated 3380 A100 training-equivalent saving). The multi-slicing design and the 24-hour gap objective are concrete, transferable ideas, and the 15-model, one-year production deployment is a rare and credible external validation. The main risk is that the evidence for the central design principle—that count-based pooling captures sufficient long-tail signal—is weaker than the evidence for the headline incremental gain, because the only sequence-encoder comparison is length-confounded and does not report quantitative deltas. The paper would be stronger with uncertainty quantification for the offline metrics and with a matched-length attentive baseline.

major comments (2)
  1. [Appendix C / Section 3.2] The conclusion that attentive sequence encoders add nothing beyond multi-slicing is not supported by the reported experiment. The text states that HSTU was scaled to 1,000 steps while the multi-slicing baseline used 35,000 steps per timeline, and no NE deltas are given for either position-weighted pooling or HSTU. This is a length-confounded comparison, so the sentence 'gains coming from additional sequence model encoded embedding fully diminished' does not follow. Because the abstract and conclusion claim that the multi-slicing strategy 'captures sufficient information' from the long history, this missing matched-length baseline is load-bearing. Please either run an attentive encoder at comparable length (or with a long-context approximation) and report the deltas, or explicitly restrict the claim to the tested configurations (e.g., HSTU at 1k, position-weighted pooling at 35k) and soften the 'sufficient information' phrasing.
  2. [Section 3, Tables 2-4] The paper asserts that an NE delta of >0.1% is considered significant, but it reports no error bars, confidence intervals, or significance tests for any of the offline NE or hit-rate deltas. Combined with Section 2.2's statement that the 200 derived features were selected by exhaustive search on offline NE, the point estimates in Tables 2-4 may overstate the deployable gain. Please report the number of evaluation days, variance across days or repeated runs, and any feature-selection holdout procedure; if the deltas are single point estimates from production data, state this and justify the significance assertion.
minor comments (6)
  1. [Section 2.4] The word 'downstrewam' should be 'downstream'.
  2. [Section 2.3.2] The stated compression factor of 50 is inconsistent with the stated 200x256 fp32 input and 58x17 long-integer output; please clarify the byte accounting (e.g., the 4-bit packing format and whether the 17 long integers are per-embedding or per-output-vector).
  3. [Appendix C] The cross-reference 'As mentioned in 6' is unclear; it should refer to the time-slicing description in Section 2.2.
  4. [Section 3.5] The phrase 'improved 0.7% of Instagram App time spent' should be 'increased Instagram app time by 0.7%' or similar.
  5. [Table 2] The table formatting makes some entries run together (e.g., '-0.500-0.659'); please add spacing for readability.
  6. [Section 2.2] The grouping criteria mention 'watch_time > 15s' and 'watch_time < 3s' but these thresholds are not defined in the preceding equations; please align the notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's claims are empirical and benchmarked against external production baselines.

full rationale

The paper's central claims are empirical rather than derived: DV365 is an offline user embedding produced by a trained upstream model, and its value is measured by adding it as a feature to downstream production models (DLRM, HSTU, MoL) and reporting NE, hit-rate, and A/B test outcomes. No equation defines the embedding in terms of the target metric, and no fitted parameter is renamed as a prediction. The choice of multi-slicing and mean pooling is presented as a scalability-motivated design and is compared, albeit imperfectly, against attentive encoders in Appendix C. The feature-selection process ('selected 200 derived features based on offline eval metrics improvements') is a standard model-selection concern and is not a circular reduction because the final evidence comes from downstream production evaluations, not from the selection criterion itself. Citations to HSTU and MoL are external baselines from other teams, not self-citations that carry the argument. The Appendix C length-confounded comparison (HSTU at 1,000 steps vs multi-slicing at 35,000) weakens the claim that order-aware attention is unnecessary, but this is an experimental validity issue, not circularity. Overall, the paper is self-contained against external benchmarks and exhibits no significant circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on domain assumptions about user interest decomposition and transferability, plus many hand-chosen hyper-parameters. No new physical entities are introduced; DV365 is an engineered representation.

free parameters (7)
  • 200 derived features = 200
    Selected by exhaustive search on offline NE improvements; the specific feature definitions are not enumerated.
  • Maximum timeline length per source = 35,000 (explicit), 35,000 (implicit)
    Chosen based on ROI; caps history to 70k total.
  • Time-slicing buckets = 3 days, 7 days
    Additional time windows chosen for long-term stable interest.
  • Watch-time filtering hyper-parameters = T_lb, T_ub, alpha_lb, alpha_ub, beta, gamma, s0 (values not reported)
    Tunable parameters in Eqs. (3)-(5); no final values disclosed.
  • 24-hour gap = 24 hours
    Distant interest objective removes most recent 24h to enforce stability; chosen by design.
  • FSA output configuration = 58 embeddings x 256 dims, 4-bit quantized
    Compression target that balances quality and serving cost.
  • Loss weight a = not reported
    Hyper-parameter balancing binary and regression losses in Eq. (8).
assumptions (4)
  • domain assumption User interests decompose into stable and emerging interest
    Motivates offline embedding and 24h gap; not directly validated except via staleness experiments.
  • domain assumption Improvement in foundation model transfers to downstream models through exported embeddings
    Explicitly stated in Section 3.1; downstream results provide partial validation for the final system, but feature selection used this assumption.
  • domain assumption Mean pooling of multi-sliced features captures sufficient long-term stable interest signal; sequential order adds little at 35k scale
    Supported only by unbalanced comparison (HSTU at 1k vs multi-slicing at 35k) in Appendix C.
  • domain assumption HSTU baseline is representative of SOTA attentive sequence models
    Used as the strong baseline; if HSTU is not representative, the claimed incrementality over advanced attentive models may be overstated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DV365: Extremely Long User History Modeling at Instagram." pith.science (2026). https://pith.science/paper/P57LUBHH

@misc{pith2026250600450,
  author       = {Pith},
  title        = {Pith review of: DV365: Extremely Long User History Modeling at Instagram},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P57LUBHH}},
  note         = {Machine review of arXiv:2506.00450}
}
read the original abstract

Long user history is highly valuable signal for recommendation systems, but effectively incorporating it often comes with high cost in terms of data center power consumption and GPU. In this work, we chose offline embedding over end-to-end sequence length optimization methods to enable extremely long user sequence modeling as a cost-effective solution, and propose a new user embedding learning strategy, multi-slicing and summarization, that generates highly generalizable user representation of user's long-term stable interest. History length we encoded in this embedding is up to 70,000 and on average 40,000. This embedding, named as DV365, is proven highly incremental on top of advanced attentive user sequence models deployed in Instagram. Produced by a single upstream foundational model, it is launched in 15 different models across Instagram and Threads with significant impact, and has been production battle-proven for >1 year since our first launch.

Figures

Figures reproduced from arXiv: 2506.00450 by the authors.

Figure 1
Figure 1. Architecture of model training and embedding exporting. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Architecture of the recommendation system architecture incorporating DV365 embeddings. New components [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. This figure illustrates the relationship between em [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Distribution of cosine similarity between two user [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 7 canonical work pages

  1. [1]

    Yue Cao, XiaoJiang Zhou, Jiaqi Feng, Peihao Huang, Yao Xiao, Dayao Chen, and Sheng Chen. 2022. Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction. arXiv:2205.10249 [cs.IR] https://arxiv.org/abs/ 2205.10249

    arXiv 2022

  2. [2]

    Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. 2023. TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou. arXiv:2302.02352 [cs.IR] https://arxiv.org/abs/2302.02352

    arXiv 2023

  3. [3]

    Zihang Dai, Guokun Lai, Yiming Yang, and Quoc V. Le. 2020. Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing. https: //api.semanticscholar.org/CorpusID:219401850

    work page 2020

  4. [4]

    Pinterest Engineering. 2022. How pinterest leverages Realtime user actions in recommendation to boost Homefeed engagement volume. https://medium.com/pinterest-engineering/how-pinterest-leverages-realtime- user-actions-in-recommendation-to-boost-homefeed-engagement-volume- 165ae2e8cde8

    work page 2022

  5. [5]

    Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, and Joaquin Quiñonero Candela. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. InProceedings of the Eighth International Workshop on Data Mining for Online Advertising(New York, NY, USA)(ADKDD’14). Association for Comp...

    arXiv 2014

  6. [6]

    Tongwen Huang, Qingyun She, Zhiqiang Wang, and Junlin Zhang. 2020. GateNet: Gating-Enhanced Deep Network for Click-Through Rate Prediction. https: //api.semanticscholar.org/CorpusID:220381423

    work page 2020

  7. [7]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recom- mendation. arXiv:1808.09781 [cs.IR] https://arxiv.org/abs/1808.09781

    arXiv 2018

  8. [8]

    Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang, Zhihao Jia, Andrew Tulloch, Srinivas Sridharan, Xing Liu, Mustafa Ozdal, Jade Nie, Jongsoo Park, Liang Luo, Jie (Amy) Yang, Leon Gao, Dmytro Ivchenko, Aarti Basant, Yuxi Hu, Jiyan Yang, Ehsan K. Ardestani, Xiaodong Wang, Rakesh Komuravelli, Ching-Hsiang Chu, Serhat Yilmaz, Huayu Li, Jiyuan Qian, Zhuobo Feng, Y...

    arXiv 2022

Show all 20 references
  1. [9]

    Nikil Pancha, Andrew Zhai, Jure Leskovec, and Charles Rosenberg. 2022. PinnerFormer: Sequence Modeling for User Representation at Pinterest. arXiv:2205.04507 [cs.LG] https://arxiv.org/abs/2205.04507

    2022 arXiv

  2. [10]

    Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’19). ACM, New Yor...

    2019

  3. [11]

    Pi Qi, Xiaoqiang Zhu, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, and Kun Gai. 2020. Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction. arXiv:2006.05639 [cs.IR] https://arxiv.org/abs/2006.05639

    2020 arXiv

  4. [12]

    Kan Ren, Jiarui Qin, Yuchen Fang, Weinan Zhang, Lei Zheng, Weijie Bian, Guorui Zhou, Jian Xu, Yong Yu, Xiaoqiang Zhu, and Kun Gai. 2019. Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction. In Proceedings of the 42nd International ACM SIGIR...

    2019

  5. [13]

    Wikipedia. 2024. https://en.wikipedia.org/wiki/Online_machine_learning

    2024

  6. [14]

    Bei Yang, Jie Gu, Ke Liu, Xiaoxiao Xu, Renjun Xu, Qinghui Sun, and Hong Liu

  7. [15]

    Jiaqi Zhai, Zhaojie Gong, Yueming Wang, Xiao Sun, Zheng Yan, Fu Li, and Xing Liu. 2023. Revisiting Neural Retrieval on Accelerators. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(Long Beach, CA, USA)(KDD ’23). Association for Computing ...

    2023

  8. [16]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Michael He, Yinghai Lu, and Yu Shi. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. arXiv:2402.17152 [cs.LG] h...

    2024 arXiv

  9. [17]

    Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2018. Deep Interest Evolution Network for Click-Through Rate Prediction. arXiv:1809.03672 [stat.ML] https://arxiv.org/abs/1809.03672

    2018 arXiv

  10. [18]

    fresh embeddings

    Guorui Zhou, Chengru Song, Xiaoqiang Zhu, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. arXiv:1706.06978 [stat.ML] https://arxiv.org/abs/ 1706.06978 A Detailed Cost Estimation A.1 Stats o...

    2018 arXiv

  11. [20]

    To further assess the stability of the DV365 embeddings over time, we analyzed the cosine similarity between different embed- ding versions

    These results underscore the DV365 embeddings’ robustness to staleness, ensuring reliable performance even when updates are delayed or entirely absent. To further assess the stability of the DV365 embeddings over time, we analyzed the cosine similarity between different embed-...

    2025

  12. [2023]

    arXiv:2110.11337 [cs.LG] https://arxiv.org/abs/2110.11337

    Empowering General-purpose User Representation with Full-life Cycle Behavior Modeling. arXiv:2110.11337 [cs.LG] https://arxiv.org/abs/2110.11337

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.