Next-User Retrieval: Enhancing Cold-Start Recommendations via Generative Next-User Modeling

Xiao Yang; Yang Huo; Yi Shen; Yu-Ting Lan; Zuotao Liu

REVIEW 3 major objections 5 minor 1 cited by

Next-User Retrieval: Enhancing Cold-Start Recommendations via Generative Next-User Modeling

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that generating the next user to interact with a cold item improves retrieval, with production gains in Douyin.

desk verdict A credible industrial cold-start system whose own ablations show the gains come from item-feature prefixing, not from the generative next-user mechanism the title promises. read the letter →

arxiv 2506.15267 v1 pith:FFNNPJR4 submitted 2025-06-18 cs.IR

Yu-Ting Lan , Yang Huo , Yi Shen , Xiao Yang , Zuotao Liu This is my paper · ORCID

classification cs.IR

keywords itemcold-startlookalikemodelinggenerativerecommendationnext-userretrievalcausalattentioncontrastivelossHNSWapproximatenearestneighborshort-video

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to solve item cold-start in large short-video platforms by reframing lookalike recommendation as a generative task: instead of matching a new item to similar users with rules or pairwise models, it trains a transformer to take a cold item's features and its sequence of recent positive interactions (likes and comments) and generate an embedding for the next user most likely to interact. The authors argue that this makes feedback transferable to items that have few or zero interactions, and that it integrates with industrial retrieval through HNSW nearest-neighbor search. They report offline gains over a traditional lookalike baseline and online A/B gains in Douyin: +0.0142% daily active users, +0.1144% creator publications, and +7.05% interactions. A sympathetic reader would care because the result is a production-scale demonstration that generative modeling of users, not just items, can move a cold-start metric that matters for creator retention.

What carries the argument

The carrying object is a transformer encoder-decoder over sequential UID embeddings, the chronological list of user IDs who liked or commented on an item, with three modifications. Causal attention makes each generated UID depend only on earlier UIDs, encoding a unidirectional relationship among interacting users; unmasked prefix prompt embeddings (item ID and category) let zero-interaction items still generate a first user; and a learnable [CLS] token appended after the sequence switches the model from generating UID tokens to producing a single next-user embedding in the same space as requesting users. The losses are a contrastive loss that aligns generated and true next-user embeddings with the dot product used in HNSW serving, a cross-entropy loss on exposed-but-not-interacted samples, and an auxiliary regression loss on the generated UID sequence.

What would settle it

Rerun the online A/B test with the transformer-generated next-user embedding replaced by an average-pooled embedding of the same interacted-user sequence; if the +0.0142% DAU and +0.1144% publication gains persist, the generative next-user model is not doing the causal work the paper claims.

Watch

Extended reading notes

Core claim

The central claim is that the next user who will interact with an item can be treated as a token to be generated, and that the sequence of users who already interacted with the item, combined with item features as prefix prompts, carries enough signal to predict that next user even when the item itself is new. Formally, the model maximizes the likelihood of the real next user $u$ given sequential user IDs $(u_1,\dots,u_n)$, item features $f_i$, and next-user features $f_u$; at serving time, candidate cold items are retrieved by matching the requesting user's embedding against generated next-user embeddings via HNSW. The paper claims this outperforms traditional lookalike methods, which pool or pairwise-compare user similarities, and that each design choice, including prefix prompts, a learnable [CLS] token, causal attention, and a contrastive-plus-cross-entropy-plus-auxiliary loss, contributes to offline Recall@Top20/Top50 and to the online metrics.

Load-bearing premise

The method assumes that users interact with an item in a causally ordered chain, so that a later user's like or comment is causally influenced by earlier users; if the order is just timestamp noise, the generated next-user embedding may encode correlation rather than causation.

Editorial extensions

If this is right

Cold-start items no longer need to accumulate their own interaction history to be recommendable: the sequence of users who already engaged, possibly just one or a few, plus item features is enough to generate a retrieval embedding.
Because the generated next-user embeddings are served through the same HNSW approximate-nearest-neighbor index as the existing retrieval stream, the approach can be dropped into a production recommender without a new serving stack.
The reported online metrics link the retrieval change to creator-side outcomes: more daily active users and more publications, consistent with the paper's argument that early feedback encourages creators to stay.
The ablations indicate the dominant offline signal is the prefix-prompt-conditioned sequence, with removing prefix prompts costing 14.15% of Recall@Top20, while the causal mask and [CLS] token contribute only fractions of a percent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the causal-order assumption is not real, the method may still work as a learned pooling of interacted users; the paper's own ablation, a 0.52% Recall drop without causal attention, leaves room for that interpretation.
A natural testable extension is to swap item and user roles and apply next-item generation to cold users, since the same sequence-to-embedding machinery does not depend on which side is cold.
The contrastive loss is deliberately aligned with dot-product serving, so the framework could generalize to any embedding-based retrieval index, not only cold-start items.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, quick read of 2506.15267. The paper is a plausible industrial write-up from ByteDance: they train a transformer to take a sequence of users who positively interacted with a cold-start video, plus item features as prefix prompts, and generate an embedding for the 'next user' likely to interact. Retrieval is then done by HNSW against this generated user embedding. That framing—predicting users rather than items—is genuinely new relative to the generative retrieval work they cite (which predicts item IDs), and the offline/online results they report are consistent with a real deployed system. The A/B test on Douyin is the kind of evidence most academic cold-start papers never have.

What works: the experimental setup is a production system with hundreds of billions of training samples; the online lift, while small (+0.0142% DAU, +0.1144% publications), is directionally consistent across variants; and the prefix-prompt and sequence-length ablations show those design choices matter. They are honest about the non-significant metrics in Table 2.

Where it falls short: the stress-test note is right. The paper's own ablations put the causal 'next-user' mechanism in the minor role—removing causal attention costs 0.5% Recall@Top20, while masking the prefix prompt costs 14%. And no variant isolates the generative next-user objective itself: the 'Traditional Lookalike' baseline is sum-pooling without prefix features or the transformer, so the 27% gap conflates architecture, features, and objective. A non-generative transformer encoder with the same prefix inputs and a BCE exposure-vs-interaction loss might well close that gap. The unidirectional-ordering assumption is speculative but, as the ablation shows, not load-bearing; the load-bearing claim—that next-user generation is the source of improvement—is untested. Reporting is also thin: no absolute metrics, no error bars, no significance tests for Table 1, no hyperparameters, no code.

Minor: template placeholders ('Conference'17'), typos ('prmopts', 'Reall@topk'), and the 'significant improvements' phrasing overstates metrics that are tiny even for a platform of that scale.

Bottom line: this is a credible engineering report from a major platform, and the direction is worth knowing about. As a scientific claim about generative next-user modeling, the experiment doesn't yet support it. A serious referee could force the revision it needs—better baselines, an ablation that turns the next-user loss on/off while holding the architecture and prefix features fixed, and fuller reporting. I'd accept it for review but expect the revision to do the real work. Reading group? Maybe—the stress-test conversation is instructive.

Referee Report

3 major / 5 minor

Summary. The paper proposes Next-User Retrieval, a transformer-based framework for item cold-start recommendation in Douyin. The method encodes a chronological sequence of users who positively interacted with a cold item as UID embeddings, prepends item ID and category features as prefix prompts, appends a learnable [CLS] token, and applies causal attention to generate an embedding of the 'next' user likely to interact. Training combines a contrastive loss over the generated next-user embedding with a cross-entropy loss over exposure-but-no-interaction samples and an auxiliary reconstruction loss. At serving, the generated next-user embedding is indexed with HNSW and matched against requesting-user embeddings. Offline results report relative Recall@Top20/50 gains over a sum-pooled lookalike baseline, and an online A/B test reports increases of +0.0142% in daily active users and +0.1144% in publications; the authors state that the system has been integrated into Douyin's main recommendation service.

Significance. The industrial setting is a strength: an online A/B test with external downstream outcomes (DAU, publications, interactions) is more convincing than a purely offline study, and the paper addresses an important practical problem with a clean framing as generative next-user retrieval. If the attribution of the gains to the generative next-user objective were established, the deployment at Douyin's scale would be a notable result for the cold-start literature. The paper does not exhibit circularity: training supervises next-user embeddings against ground-truth users, and the online test is an external outcome. However, the current experimental design does not isolate the generative objective from the transformer backbone, prefix prompts, and the three-loss combination, so the central mechanism remains unvalidated as the driver of the reported improvements.

major comments (3)

[Table 1 / Section 2.4 (Eq. 4)] The central claim that generative next-user modeling is the source of the improvement is not tested, because no variant ablates the next-user generation objective. Variants 4 and 5 remove the CLS token and causal attention, but every variant other than Traditional Lookalike still trains with L_contrastive, L_CE, and L_auxiliary and still generates a next-user embedding. Variant 1 changes the architecture, prefix features, and loss simultaneously, so the reported -27.20%/-22.11% gap conflates all of those factors. Please add an ablation in which the same transformer with the same prefix prompts and CLS token is trained with a standard exposure-versus-interaction BCE or ranking loss, or at least report individual ablations of the three loss terms in Eq. (4), to show that the generative next-user supervision, rather than the richer backbone and prefix features, drives the gains.
[Table 2] The online results contain an internal inconsistency that weakens the significance claim: the 'Half the Sequence Length' variant matches the full model on Publications (+0.1144%) and exceeds it on Interactions (+7.4535% vs. +7.0515%), even though the offline ablation in Table 1 shows a -4.33%/-1.98% drop from halving the sequence. The paper neither explains this discrepancy nor reports confidence intervals or p-values; the symbol '-' only indicates 'not statistically significant' without giving the significance test used. Please report uncertainty bounds and reconcile the online and offline ordering; if 'Half the Sequence Length' is statistically indistinguishable from the full model on the key metrics, the conclusion that the full configuration is best needs qualification.
[Section 3, Offline Results / Table 1] The offline evaluation is under-specified. Recall@Top20/Top50 is reported only as relative differences for the variants, with no absolute values, candidate pool size, evaluation sample size, or variance. The paper also omits essential experimental details: the temperature tau, the loss weights lambda_1..3, embedding dimension, number of layers and heads, learning rate, and optimizer. Without these details the offline comparisons cannot be reproduced or assessed. Please provide the missing configuration and, at a minimum, confidence intervals or significance tests for the key comparisons.

minor comments (5)

[Eqs. (5) and (7)] The mathematical typesetting is corrupted in places: the denominator in Eq. (5) contains a stray '˝' symbol, and Eq. (7) has a formatting error in the sum. Please regenerate the equations carefully.
[Section 2.3] The motivation for the unidirectional relation is not clearly stated: 'users typically engage in actions such as commenting only after encountering some interesting comments' does not by itself justify a unidirectional ordering of the sequence of users who interacted with an item. Since the w/o-causal-attention ablation shows a small effect, this is not a load-bearing issue, but the statement should be revised to present the assumption as an inductive bias rather than an established observation.
[Section 3] There are several typos: 'Reall@topk' should be 'Recall@topk', 'converged,.' has a stray comma, and Eq. (3) contains 'prmopts' instead of 'prompts'.
[Section 3, offline evaluation setup] The paper should clarify whether the offline evaluation uses a time-based split and whether the ground-truth users in Eq. (1) overlap with users seen during training; this affects the interpretation of the Recall numbers for the cold-start scenario.
[General] The manuscript uses the anonymous ACM template with placeholder venue ('Conference\'17'), placeholder DOI, and page-count uncertainties. These must be updated to the final submission format.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular dependency: the training target, the retrieval embedding, and the online success metrics are externally distinct.

full rationale

Next-User Retrieval's derivation chain is a standard supervised learning pipeline rather than a circular one. Equation (1) defines the training objective as maximizing the likelihood of the real next user u given the sequence of previously interacted users and item prefix features; the contrastive loss in Eq. (5) trains the generated next-user embedding to be similar to the ground-truth interacted user embedding and dissimilar to negatives. At serving, the item-side generated next-user embeddings are matched against the requesting user's embedding via HNSW, so the target user is not fed into the model at inference time. The offline Recall@Top20/Top50 metrics are the usual supervised evaluation of next-user retrieval, and the online A/B test against daily active users and creator publications is an external business outcome, not a transformed version of the training labels. No equation is shown to equal another by construction, and no fitted parameter is relabeled as a prediction. The only author-overlap citation, reference [10] used to identify Douyin, is background context and carries no load-bearing argument. The unidirectional-attention premise is an inductive-bias assumption whose limited contribution is empirically ablated in Table 1, variant 5, so it is not a self-definitional justification. The skeptic's concern that the prefix-prompt ablation accounts for most of the gain is an attribution or internal-validity question, not evidence of circularity. The paper is self-contained against an external production benchmark.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method depends on several behavioral assumptions about user interaction order and sample informativeness, plus a learnable token to reconcile feature domains. None are independently validated beyond the reported online gains.

free parameters (4)

Contrastive loss temperature tau
Appears in Eq. (5) contrastive loss; no value reported.
Loss weights lambda1, lambda2, lambda3
Appear in Eq. (4) combined loss; no values or tuning procedure reported.
Maximum sequence length = 50
Section 2.2: limited to 50 to meet storage and latency constraints.
Embedding dimension d
Used throughout Section 2; no value reported.

assumptions (4)

domain assumption Users interact with items in a unidirectional causal order, so the next user is predictable from preceding users.
Stated in Section 2.3: 'there exists a unidirectional relationship'; the causal attention mechanism relies on this. No evidence is provided.
domain assumption Exposed-but-not-interacted samples are informative high-quality negatives.
Section 2.4, Cross-Entropy Loss: these samples passed the recommendation funnel, 'indicating they are relatively high-quality and informative compared to random negative samples.'
domain assumption A learnable [CLS] token can bridge the feature domain gap between sparse sequential UID embeddings and fully contextual requesting user embeddings.
Section 2.3, Learnable [CLS] token.
domain assumption Dot-product similarity between a requesting user's embedding and the generated next-user embedding is a valid retrieval signal.
Section 2.1 and 2.4 use dot product for HNSW retrieval and contrastive loss.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Next-User Retrieval: Enhancing Cold-Start Recommendations via Generative Next-User Modeling." pith.science (2026). https://pith.science/paper/FFNNPJR4

@misc{pith2026250615267,
  author       = {Pith},
  title        = {Pith review of: Next-User Retrieval: Enhancing Cold-Start Recommendations via Generative Next-User Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FFNNPJR4}},
  note         = {Machine review of arXiv:2506.15267}
}

read the original abstract

The item cold-start problem is critical for online recommendation systems, as the success of this phase determines whether high-quality new items can transition to popular ones, receive essential feedback to inspire creators, and thus lead to the long-term retention of creators. However, modern recommendation systems still struggle to address item cold-start challenges due to the heavy reliance on item and historical interactions, which are non-trivial for cold-start items lacking sufficient exposure and feedback. Lookalike algorithms provide a promising solution by extending feedback for new items based on lookalike users. Traditional lookalike algorithms face such limitations: (1) failing to effectively model the lookalike users and further improve recommendations with the existing rule- or model-based methods; and (2) struggling to utilize the interaction signals and incorporate diverse features in modern recommendation systems. Inspired by lookalike algorithms, we propose Next-User Retrieval, a novel framework for enhancing cold-start recommendations via generative next-user modeling. Specifically, we employ a transformer-based model to capture the unidirectional relationships among recently interacted users and utilize these sequences to generate the next potential user who is most likely to interact with the item. The additional item features are also integrated as prefix prompt embeddings to assist the next-user generation. The effectiveness of Next-User Retrieval is evaluated through both offline experiments and online A/B tests. Our method achieves significant improvements with increases of 0.0142% in daily active users and +0.1144% in publications in Douyin, showcasing its practical applicability and scalability.

Figures

Figures reproduced from arXiv: 2506.15267 by the authors.

**Figure 1.** The framework of Next-User Retrieval. The dark and light cells in the attention mask indicate whether the token is [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

A Survey of Real-World Recommender Systems: Challenges, Constraints, and Industrial Perspectives
cs.IR 2025-09 conditional novelty 4.0 of 10

A survey of A/B-validated industrial recommender systems, split into transaction-oriented and content-oriented categories, with a discussion of the academia-industry gap.

Reference graph

Works this paper leans on

14 extracted references · 9 canonical work pages · cited by 1 Pith paper

[1]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations.Proceedings of the 10th ACM conference on recom- mender systems(2016), 191–198

work page 2016
[2]

Chi Liu, Jiangxia Cao, Rui Huang, Kai Zheng, Qiang Luo, Kun Gai, and Guorui Zhou. 2024. KuaiFormer: Transformer-Based Retrieval at Kuaishou.arXiv preprint arXiv:2411.10057(2024)

arXiv 2024
[3]

Yudan Liu, Kaikai Ge, Xu Zhang, and Leyu Lin. 2019. Real-time attention based look-alike model for recommender system.Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(2019), 2765– 2773

work page 2019
[4]

Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.IEEE Transactions on Pattern Analysis and Machine Intelligence42, 4 (2018), 824–836

2018
[5]

Ashish Mangalampalli, Adwait Ratnaparkhi, Andrew O Hatch, Abraham Bagher- jeiran, Rajesh Parekh, and Vikram Pudi. 2011. A feature-pair-based associative classification approach to look-alike modeling for conversion-oriented user- targeting in tail campaigns.Proceedings of the 20th international conference companion on World wide web(2011), 85–86

work page 2011
[6]

Yang Peng, Changzheng Liu, and Wei Shen. 2023. Finding Lookalike Customers for E-Commerce Marketing.arXiv preprint arXiv:2301.03147(2023)

work page Pith review arXiv 2023
[7]

Md Mostafizur Rahman, Daisuke Kikuta, Yu Hirate, and Toyotaro Suzumura. 2024. Graph-Based Audience Expansion Model for Marketing Campaigns.Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval(2024), 2970–2975

work page 2024
[8]

Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al

Show all 14 references

[9]

Jianqiang Shen, Sahin Cem Geyik, and Ali Dasdan. 2015. Effective audience extension in online advertising.Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining(2015), 2099–2108

2015
[10]

Jing Yan, Liu Jiang, Jianfei Cui, Zhichen Zhao, Xingyan Bin, Feng Zhang, and Zuotao Liu. 2024. Trinity: Syncretizing Multi-/Long-Tail/Long-Term Interests All in One.Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(2024), 6095–6104

2024
[11]

Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, et al. 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommenda- tions.International Conference on Machine Learning...

2024
[12]

Weizhi Zhang, Yuanchen Bei, Liangwei Yang, Henry Peng Zou, Peilin Zhou, Aiwei Liu, Yinghui Li, Hao Chen, Jianling Wang, Yu Wang, et al . 2025. Cold- Start Recommendation towards the Era of Large Language Models (LLMs): A Comprehensive Survey and Roadmap.arXiv preprint arXiv:25...

2025 arXiv
[13]

Yongchun Zhu, Yudan Liu, Ruobing Xie, Fuzhen Zhuang, Xiaobo Hao, Kaikai Ge, Xu Zhang, Leyu Lin, and Juan Cao. 2021. Learning to expand audience via meta hybrid experts and critics for recommendation and advertising.Proceedings of the 27th ACM SIGKDD Conference on Knowledge Dis...

2021
[2023]

Recommender systems with generative retrieval.Advances in Neural Information Processing Systems36 (2023), 10299–10315

2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations.Proceedings of the 10th ACM conference on recom- mender systems(2016), 191–198

work page 2016

[2] [2]

Chi Liu, Jiangxia Cao, Rui Huang, Kai Zheng, Qiang Luo, Kun Gai, and Guorui Zhou. 2024. KuaiFormer: Transformer-Based Retrieval at Kuaishou.arXiv preprint arXiv:2411.10057(2024)

arXiv 2024

[3] [3]

Yudan Liu, Kaikai Ge, Xu Zhang, and Leyu Lin. 2019. Real-time attention based look-alike model for recommender system.Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(2019), 2765– 2773

work page 2019

[4] [4]

Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.IEEE Transactions on Pattern Analysis and Machine Intelligence42, 4 (2018), 824–836

2018

[5] [5]

Ashish Mangalampalli, Adwait Ratnaparkhi, Andrew O Hatch, Abraham Bagher- jeiran, Rajesh Parekh, and Vikram Pudi. 2011. A feature-pair-based associative classification approach to look-alike modeling for conversion-oriented user- targeting in tail campaigns.Proceedings of the 20th international conference companion on World wide web(2011), 85–86

work page 2011

[6] [6]

Yang Peng, Changzheng Liu, and Wei Shen. 2023. Finding Lookalike Customers for E-Commerce Marketing.arXiv preprint arXiv:2301.03147(2023)

work page Pith review arXiv 2023

[7] [7]

Md Mostafizur Rahman, Daisuke Kikuta, Yu Hirate, and Toyotaro Suzumura. 2024. Graph-Based Audience Expansion Model for Marketing Campaigns.Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval(2024), 2970–2975

work page 2024

[8] [8]

Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al