Pith. sign in

REVIEW 4 major objections 6 minor 46 references

Learning from the Future: Privileged Self-Distillation for Sequential Recommendation

T0 review · 4 major / 6 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Future clicks in logged sequences can teach a causal recommender richer preferences without ever being used at serving time.

desk verdict Clean single-stage training trick that turns future suffix into soft labels for causal seq-rec; gains look real, gate story is the soft spot. read the letter →

arxiv 2607.27055 v1 pith:6PNBYR45 submitted 2026-07-29 cs.IR

Jiakai Tang , Yang Zhang , See-Kiong Ng , Xu Chen , Wen Chen , Jian Wu , Han Zhu This is my paper · ORCID
classification cs.IR
keywords SequentialRecommendationPrivilegedInformationSelf-DistillationKnowledgeDistillationCausalAttentionNext-itemPrediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sequential recommenders are usually trained with one-hot next-item labels from the past only, which matches deployment but leaves almost no signal about how non-target items should rank. This paper treats the future part of each logged sequence as privileged training information: it can shape what the model learns, but must not enter the predictor users actually call. The method, Privileged Self-Distillation, runs one Transformer under two attention masks—one that sees past and future, one that sees only the past—and distills the future-aware soft distribution into the causal student. A gate keeps only the soft targets the past can plausibly support, and a momentum-averaged teacher steadies the moving targets. The deployed model stays identical in size and cost to the original backbone, yet ranking quality rises consistently across public benchmarks and several architectures. A sympathetic reader cares because the extra supervision is already sitting in every interaction log and requires no new features, teachers, or serving changes.

What carries the argument

Privileged Self-Distillation (PSD): dual attention masks on one backbone produce a future-aware teacher and a causal student; an advantage-reachability gate (batch percentile on teacher–student KL) admits only low-discrepancy soft targets, and a momentum-averaged teacher supplies stable distillation targets, all trained in one stage.

What would settle it

Train PSD with the gate ablated versus full gating on the same backbone and data; if gating does not improve causal ranking—or if teacher soft labels that disagree most with the student still transfer well when forced—then the reachability-proxy claim fails.

Watch

Extended reading notes

Core claim

The paper claims that future suffix interactions are usable as training-only privileged information for sequential recommendation: by evaluating one shared backbone under a future-aware teacher mask and a prefix-only student mask, then transferring the teacher’s soft next-item distribution through gated KL distillation with a momentum teacher, the causal student absorbs richer relative preferences than one-hot labels provide, while inference remains strictly causal and unchanged in cost.

Load-bearing premise

The method assumes that a small teacher–student disagreement mostly means the future signal is already recoverable from the past, so discarding high-disagreement positions removes unreachable noise rather than useful hard cases the student still needs.

Editorial extensions

If this is right

  • Any Transformer-style sequential recommender can gain from future-suffix soft labels without changing its serving graph or adding parameters.
  • One-hot next-item training systematically underuses logged trajectories; privileged soft targets are a drop-in richer alternative.
  • Co-evolving same-backbone teachers can outperform separately pretrained static teachers for this form of privileged signal.
  • The same dual-mask plus gated distillation pattern is offered as a general training idea for other sequential prediction tasks that log full trajectories but must act causally.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gate’s KL proxy is only a coarse filter, replacing it with an explicit prefix-only recoverability score could further shrink the teacher–student gap that remains even at full future visibility.
  • The same privileged-suffix idea may apply to session-based ranking, next-basket prediction, or offline RL from logged trajectories wherever future context is observed in training logs but forbidden at act time.
  • Because the student never sees a one-hot loss, failures of the teacher early in training could starve the student of any hard anchor; curriculum schedules that briefly mix hard labels might be a natural stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper argues that future interactions in logged sequences are training-only privileged information for sequential recommendation, and proposes Privileged Self-Distillation (PSD): a single Transformer backbone is run under a privileged (past+future) mask as teacher and a causal (prefix-only) mask as student; the teacher is grounded by next-item CE, while the student is trained solely by gated KL distillation from a momentum-averaged teacher. An advantage-reachability gate keeps only batch-wise low teacher–student KL positions. The deployed model remains the causal student with unchanged inference cost. Across Video Games, CDs & Vinyl, and Yelp, and on SASRec, BERT4Rec, and UniSRec, PSD reports consistent gains over the backbone and over RD, TCE/RCE, S4Rec, and CSRec variants (Table 3), with supporting diagnostics on future visibility, oracle splits, win-rate dynamics, and hyperparameter sensitivity.

Significance. If the empirical gains hold under stronger statistical scrutiny, PSD is a practically useful training recipe: it needs no extra parameters, side features, or multi-stage teacher pretraining, and leaves serving identical to the backbone. Framing the logged suffix as privileged information (rather than as bidirectional input) cleanly separates learning-time and inference-time information and is a useful conceptual contribution for sequential recommenders. Strengths include full-corpus ranking, chronological splits, multi-backbone evaluation, and diagnostic analyses (Figs. 3–6) that partially support the mechanism story. The work is orthogonal to architecture design and could transfer to other causal sequential predictors.

major comments (4)
  1. [Section 3.2, Eqs. (11)–(13)] Section 3.2, Eqs. (11)–(13): the advantage-reachability gate is load-bearing for the claim that PSD transfers only prefix-supported privileged signal. The paper equates small batch-wise d_t = KL(sg(p̄_T) || p_S) with “reachable” advantage and large d_t with suffix-only or premature targets, then drops the latter. This is not validated. Low d_t is where teacher and student already agree (often easy positions); high d_t is where privileged soft structure differs most. Fig. 6 shows some δ < 1 helps on Video Games, which is also consistent with generic hard-example or noise filtering. Fig. 3’s residual teacher–student gap motivates filtering but does not show that the dropped slice is the future-only slice. Please add a direct check (e.g., correlate gated-out positions with suffix dependence via teacher ablation that masks the suffix, or compare gate vs. random/high-d_t retention) or soften
  2. [Table 3, Section 4.2] Table 3: all headline claims (“consistent and substantial,” ~19.8% average over backbones, large BERT4Rec lifts) rest on single-point metrics with no error bars, no multi-seed means/stds, and no significance tests. Several margins over the strongest baseline are modest (e.g., UniSRec on CDs & Vinyl). For a journal claim of consistent superiority across three datasets and three backbones, report at least mean±std over multiple runs (or paired tests) on the main setting; otherwise the ranking of close methods is hard to trust.
  3. [Section 4.2, Table 3 (Bert4Rec)] Section 4.2 / BERT4Rec rows: the largest gains are on BERT4Rec, where the paper itself notes a severe train–serve mismatch (masked bidirectional training vs causal serving). PSD forces the student to be causal while still giving the teacher future context, so part of the lift may be “fix the objective to causal next-item prediction” rather than privileged-future distillation per se. Please disentangle these (e.g., causal-CE-only BERT4Rec student, or PSD without suffix on the teacher) so the privileged-future contribution is isolated from train–serve realignment.
  4. [Section 3.1, Eq. (16), Section 3.4] Section 3.1 and Eq. (16): the student receives no one-hot CE, only gated KD from the teacher. This is a strong design choice and central to the “soft preference structure” story, but there is no ablation against (i) student CE only, (ii) student CE + KD, or (iii) ungated KD. Without that, it is unclear whether gains require dropping student CE, the gate, EMA, or simply richer soft targets from a dual forward pass. A compact ablation table on one backbone/dataset would pin down which pieces are necessary.
minor comments (6)
  1. [Figure 1] Figure 1 caption and axis labels are clear, but the main text could state the exact protocol for “future visibility %” (random suffix tokens vs. contiguous suffix prefix) so the diagnostic is reproducible.
  2. [Table 1, Section 3.5] Table 1 is helpful; consider adding a row for whether the teacher is co-evolving vs frozen, since that is a stated advantage of PSD over RD.
  3. [Figure 2, Algorithm 1] Notation: p̄_T / ¯p^T and θ̄ appear in the figure and Algorithm 1 with slight typographic inconsistency (bar placement); unify with Eqs. (14)–(15).
  4. [Section 4.1.4] Implementation details fix lr, batch size, and dim for all methods but tune dropout/weight decay per method; briefly confirm that Base and PSD used the same search grid so gains are not from unequal regularization budgets.
  5. [Section 5.2] Related work on LUPI / privileged information (Vapnik; Lopez-Paz et al.) is cited; a one-sentence contrast with non-recommendation LUPI distillation would help non-RecSys readers.
  6. [Section 1, Section 3.3] Minor prose: “deployment costraint” → “constraint” (contribution bullet); “A verage” → “Average” in “Exponential Moving A verage”.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical training method evaluated on held-out ranking metrics; gains are not forced by definition or self-citation.

full rationale

PSD is a training procedure (dual attention masks, teacher CE + gated student KL, EMA teacher) whose claimed contribution is improved HR/NDCG on chronological held-out splits for SASRec, BERT4Rec, and UniSRec. The evaluation targets are standard next-item ranking metrics independent of the training losses (Eqs. 9–16). Hyperparameters δ and α are tuned on validation, which is ordinary model selection, not a fitted constant renamed as a prediction. Privileged information and distillation are cited from external literature (Vapnik, Hinton, Mean Teacher, etc.); no load-bearing uniqueness theorem or author-only prior result forces the architecture or the reported lifts. The advantage-reachability gate is a heuristic filter on batch KL, not a quantity defined to equal the test metric. Residual teacher–student gaps and ablations are diagnostic, not circular closures. Any skepticism about whether low-d_t truly measures prefix-reachability is a correctness/interpretation concern, not circularity of the derivation chain.

Assumptions & free parameters 3 free parameters · 5 assumptions · 2 invented entities

Load-bearing content is methodological and empirical rather than axiomatic physics-style derivation. Claims rest on standard next-item softmax modeling, the LUPI-style split between train-only future and serve-time prefix, and engineering choices (percentile gate, EMA) whose justification is experimental.

free parameters (3)
  • gate percentile δ = dataset/backbone-dependent; e.g. peaks near 0.8 on Video Games in Fig. 6
    Controls which fraction of batch positions enter distillation; searched over {0.6,0.7,0.8,0.9,0.99,1.0} and material to the reachability mechanism.
  • EMA rate α = dataset-dependent; e.g. 0.8 on Video Games, 0.99 on Yelp in sensitivity plot
    Smoothing factor for momentum teacher parameters; searched over {0.6,0.7,0.8,0.9,0.99}.
  • shared training hyperparameters (lr, batch size, dim, dropout, weight decay) = lr=1e-3, batch=2048, dim=256; dropout/wd tuned on grids
    Fixed or tuned choices that affect all compared methods; not the scientific object but required to realize reported numbers.
assumptions (5)
  • domain assumption Serving-time sequential recommendation may condition only on the observed prefix (causal attention).
    Stated in §1–§2 as the deployment constraint that forbids using the suffix as input at inference.
  • domain assumption Future suffix interactions carry positive conditional mutual information about the target given the prefix (Eq. 3 ≥ 0), i.e., are informative privileged information.
    Motivates the teacher; supported diagnostically by Fig. 1 future-visibility gains, not proved for all users/items.
  • domain assumption Softmax next-item cross-entropy / KL on Transformer logits is an adequate training surrogate for ranking quality (HR/NDCG).
    Standard sequential-rec practice underlying L_CE^T and L_KD^S.
  • ad hoc to paper Low teacher–student KL within a mini-batch indicates privileged advantage reachable from the prefix; high KL indicates non-transferable or premature targets.
    Defines the advantage-reachability gate (§3.2); central inductive bias of PSD, validated only empirically.
  • ad hoc to paper EMA of shared parameters yields stabler privileged targets than the instantaneous teacher without harming co-evolution.
    §3.3; borrowed from Mean Teacher/MoCo-style practice and ablated via α sensitivity.
invented entities (2)
  • Privileged Self-Distillation (PSD) dual-mask teacher/student on one backbone
    purpose: Create an informationally privileged teacher and a deployable causal student without extra parameters or a separately pretrained teacher.
    Core proposed framework (§3.1, Fig. 2); entity is a training procedure, not a physical object.
  • Advantage-reachability gate
    purpose: Filter distillation positions so the student only imitates teacher soft labels deemed supported by the prefix.
    Introduced in §3.2 via batch percentile of KL; mechanism-specific to this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning from the Future: Privileged Self-Distillation for Sequential Recommendation." pith.science (2026). https://pith.science/paper/6PNBYR45

@misc{pith2026260727055,
  author       = {Pith},
  title        = {Pith review of: Learning from the Future: Privileged Self-Distillation for Sequential Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6PNBYR45}},
  note         = {Machine review of arXiv:2607.27055}
}
read the original abstract

Sequential recommenders are commonly trained with one-hot next-item labels under a causal (prefix-only) objective aligned with inference. While deployment-compatible, this supervision offers little insight into relative preferences among non-target items. Yet logged interaction sequences contain an additional supervisory source: interactions following the target often reveal how user intent evolves, making the target easier to interpret. We treat these future interactions as training-only privileged information, available during learning but not at inference. This raises a natural question: can future interactions provide richer supervision while keeping training aligned with inference-time prediction? We propose Privileged Self-Distillation (PSD), a framework that separates learning-time information from inference-time information. PSD applies two attention masks to the same backbone: a future-aware view yields a privileged teacher distribution conditioned on past and future interactions, while a prefix-only view yields the student distribution used for deployment. Distilling the privileged distribution converts future interactions into training-only supervision rather than inference-time inputs. Since both views share a backbone, the teacher's advantage is purely informational, not architectural, removing the need for a separately pretrained teacher and letting its supervision adapt as the student evolves. PSD further uses an advantage-reachability gate to focus distillation on teacher signals likely supported by the observed prefix, along with a momentum-averaged teacher for stable targets. The framework is optimized end-to-end in a single stage, leaving the deployed model and inference cost unchanged. Experiments across public benchmarks and diverse backbones show consistent improvements.

Figures

Figures reproduced from arXiv: 2607.27055 by the authors.

Figure 1
Figure 1. Relative improvement in HR@20, NDCG@20, and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of PSD. The same Transformer backbone is evaluated under two attention masks: the causal mask restricts [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Oracle analysis comparing PSD with its backbone [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Win-rate dynamics of the privileged teacher and [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 2 canonical work pages

  1. [1]

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. 2024. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. InInternational Conference on Learning Representations (ICLR)

    2024

  2. [2]

    Tesfaye Fenta Boka, Zhendong Niu, and Rama Bastola Neupane. 2024. A sur- vey of sequential recommendation systems: Techniques, evaluation, and future directions.Information Systems125 (2024), 102427

    2024

  3. [3]

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9650–9660

    2021

  4. [4]

    Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and An- ima Anandkumar. 2018. Born again neural networks. InInternational Conference on Machine Learning (ICML). PMLR, 1607–1616

    2018

  5. [5]

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021. Knowl- edge distillation: A survey.International Journal of Computer Vision (IJCV)129, 6 (2021), 1789–1819

    2021

  6. [6]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge Distillation of Large Language Models. InInternational Conference on Learning Representations (ICLR)

    2024

  7. [7]

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Mo- mentum contrast for unsupervised visual representation learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9729–9738

    2020

  8. [8]

    Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk

Show all 46 references
  1. [9]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531(2015)

    2015 arXiv

  2. [10]

    Yupeng Hou, Jiacheng Li, Xiangjun Fu, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. 2026. Bridging language and items for retrieval and recommenda- tion: Benchmarking LLMs as semantic encoders. InProceedings of the 64th Annual Meeting of the Association for Computational...

    2026

  3. [11]

    Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards universal sequence representation learning for recommender systems. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Dis- covery and Data Mining (KDD). 585–593

    2022

  4. [12]

    SeongKu Kang, Junyoung Hwang, Wonbin Kweon, and Hwanjo Yu. 2020. DE- RRD: A knowledge distillation framework for recommender system. InProceed- ings of the 29th ACM International Conference on Information and Knowledge Management (CIKM). 605–614

    2020

  5. [13]

    SeongKu Kang, Junyoung Hwang, Wonbin Kweon, and Hwanjo Yu. 2021. Topol- ogy distillation for recommender system. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 829–839

    2021

  6. [14]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Rec- ommendation. In2018 IEEE International Conference on Data Mining (ICDM). 197–206

    2018

  7. [15]

    Jae-woong Lee, Minjin Choi, Jongwuk Lee, and Hyunjung Shim. 2019. Collabora- tive distillation for top-N recommendation. In2019 IEEE International Conference on Data Mining (ICDM). IEEE, 369–378

    2019

  8. [16]

    David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. 2016. Unifying Distillation and Privileged Information. InInternational Conference on Learning Representations (ICLR)

    2016

  9. [17]

    Amir M Mansourian, Rozhan Ahmadi, Masoud Ghafouri, Amir Mohammad Babaei, Elaheh Badali Golezani, Zeynab Yasamani Ghamchi, Vida Ramezanian, Alireza Taherian, Kimia Dinashi, Amirali Miri, et al . 2025. A comprehensive survey on knowledge distillation.arXiv preprint arXiv:2503.12...

    2025

  10. [18]

    Li-Wei Pan, Wei-Ke Pan, Mei-Yan Wei, Hong-Zhi Yin, and Zhong Ming. 2026. A survey on sequential recommendation.Frontiers of Computer Science20, 3 (2026), 2003606

    2026

  11. [19]

    Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive Learn- ing for Representation Degeneration Problem in Sequential Recommendation. InProceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM). 813–823

    2022

  12. [20]

    Tran, Jonah Samost, Maciej Kula, Ed H

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. 2023. Recommender Systems with Generative Retrieval. InAdvances in Neural In...

    2023

  13. [21]

    Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factor- izing personalized Markov chains for next-basket recommendation. InProceed- ings of the 19th International Conference on World Wide Web (WWW). 811–820

    2010

  14. [22]

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. FitNets: Hints for thin deep nets. In International Conference on Learning Representations (ICLR)

    2015

  15. [23]

    Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang

  16. [24]

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2818–2826

    2016

  17. [25]

    Jiakai Tang, Xu Chen, Wen Chen, Jian Wu, Yuning Jiang, and Bo Zheng. 2026. Parallel Latent Reasoning for Sequential Recommendation.arXiv preprint arXiv:2601.03153(2026)

    2026

  18. [26]

    Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Jian Wu, and Yuning Jiang. 2026. Think before recommend: Unleashing the latent reasoning power for sequential recommendation.IEEE Transactions on Knowledge and Data Engineering (TKDE)(2026)

    2026

  19. [27]

    Jiaxi Tang and Ke Wang. 2018. Personalized top-N sequential recommendation via convolutional sequence embedding. InProceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM). 565–573

    2018

  20. [28]

    Jiaxi Tang and Ke Wang. 2018. Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 2289–2298. doi:10.1145/3219819.3220021

    2018

  21. [29]

    Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. InAdvances in Neural Information Processing Systems (NeurIPS)

    2017

  22. [30]

    Vladimir Vapnik and Akshay Vashist. 2009. A New Learning Paradigm: Learning Using Privileged Information.Neural Networks22, 5–6 (2009), 544–557

    2009

  23. [31]

    Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z Sheng, and Mehmet Orgun. 2019. Sequential recommender systems: challenges, progress and prospects.arXiv preprint arXiv:2001.04830(2019)

    2019 arXiv

  24. [32]

    Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021. Denoising Implicit Feedback for Recommendation. InProceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM). 373–381. doi:10.1145/3437963.3441800

    2021

  25. [33]

    Shaowei Wei, Zhengwei Wu, Xin Li, Qintong Wu, Zhiqiang Zhang, Jun Zhou, Lihong Gu, and Jinjie Gu. 2024. Leave No One Behind: Online Self-Supervised Self-Distillation for Sequential Recommendation. InProceedings of the ACM Web Conference (WWW). doi:10.1145/3589334.3645590

    2024

  26. [34]

    Shiguang Wu, Xin Xin, Pengjie Ren, Zhumin Chen, Jun Ma, Maarten de Rijke, and Zhaochun Ren. 2024. Learning Robust Sequential Recommenders through Confident Soft Labels.ACM Transactions on Information Systems (TOIS)(2024). doi:10.1145/3700876

    2024 doi

  27. [35]

    Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive Learning for Sequential Recommendation. InProceedings of the 38th IEEE International Conference on Data Engineering (ICDE). 1259–1273

    2022

  28. [36]

    Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun, Jian Wu, Hanxiao Sun, and Wenwu Ou. 2019. Privileged Features Distillation at Taobao Recommendations.arXiv preprint arXiv:1907.05171(2019). doi:10.48550/arXiv.1907.05171

    2019 doi arXiv

  29. [37]

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024. A survey on knowledge distillation of large language models.arXiv preprint arXiv:2402.13116(2024)

    2024 arXiv

  30. [38]

    Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to go next for recommender systems? id- vs. modality-based recommender models revisited. InProceedings of the 46th International ACM SIGIR Conference on Research and Devel...

    2023

  31. [39]

    Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. 2019. Be your own teacher: Improve the performance of con- volutional neural networks via self distillation. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 3713–3722

    2019

  32. [40]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou

  33. [41]

    Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. 2018. Deep mutual learning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4320–4328

    2018

  34. [42]

    Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. InProceed- ings of the 29th ACM International Conference on Inf...

    2020

  35. [43]

    Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. 2022. Filter-enhanced MLP is all you need for sequential recommendation. InProceedings of the ACM Web Conference (WWW). 2388–2399

    2022

  36. [2016]

    InInter- national Conference on Learning Representations (ICLR)

    Session-based recommendations with recurrent neural networks. InInter- national Conference on Learning Representations (ICLR)

  37. [2019]

    InProceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM)

    BERT4Rec: Sequential Recommendation with Bidirectional Encoder Rep- resentations from Transformer. InProceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM). 1441–1450

  38. [2025]

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models.arXiv preprint arXiv:2506.05176(2025)

    2025 arXiv

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.