Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Generative Representational Learning of Foundation Models for Recommendation

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RecFound claims to be the first recommendation foundation model to train three embedding tasks and ten generative tasks in one shared backbone, reporting the best average rank on both task branches—ahead of larger LLMs and finetuned…

desk verdict A real systems contribution with a new benchmark, but the SOTA claim needs entity-leakage analysis and error bars before I'd trust the numbers. read the letter →

arxiv 2506.11999 v3 pith:GGDHLROK submitted 2025-06-13 cs.IR cs.CL

Zheli Zhou , Chenxu Zhu , Jianghao Lin , Bo Chen , Ruiming Tang , Weinan Zhang , Yong Yu This is my paper · ORCID
classification cs.IRcs.CL
keywords recommendationfoundationmodelsgenerativerepresentationallearningmulti-tasktrainingmixtureoflow-rankexpertsconvergence-awaresampleschedulingmodelmergingembeddingandtasks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that one foundation model can serve the two halves of recommendation usually handled by separate models: embedding tasks that power retrieval (representing users, queries, and items as vectors) and generative tasks that produce text (next-item prediction, user profiles, sentiment, product descriptions, and answer generation). Previous recommendation foundation models trained mostly the generative side; RecFound is claimed to be the first to train 3 embedding and 10 generative tasks together in one backbone. To make that joint training work, the authors contribute a dataset covering all 13 tasks, a task-routed mixture of low-rank experts, a convergence-aware sample scheduler, and a checkpoint-merging step. If the claim holds, a single model could replace the two-model setup typical of deployed recommendation systems, and the reported state-of-the-art average ranks (1.30 on generative tasks, 1.33 on embedding tasks) would be the first evidence that retrieval and text capabilities reinforce rather than damage each other in a recommendation foundation model.

What carries the argument

Three modules carry the argument. (1) Task-wise Mixture of Low-Rank Experts (TMoLE): each attention projection is augmented with $N=6$ parallel low-rank adapters of rank $r=16$, partitioned into two embedding experts, two generative experts, and two shared experts; a shared MLP router maps a learned task embedding to a gating vector $v_i = \mathrm{Softmax}(\mathrm{Mask}(\mathrm{MLP}(e_i)))$ that zeroes out experts irrelevant to the sample's task, and the projection output is $o = W_0 x + \sum_{j=1}^{N} v_i A_j B_j x$. The task mask is what lets related tasks share knowledge while unrelated tasks keep separate pathways. (2) Step-wise Convergence-oriented Sample Scheduler (S2Sched): at each step it fits a linear regression to the last $L=64$ validation losses per task to get normalized slopes, computes inter-task and intra-task non-convergence rates and a balancing weight $\beta_b(t)$ shared within embedding or generative task groups, and reallocates the batch budget $n^{(b)}_i(t) = \lfloor B^{(b)} \omega_{i,b}(t) \rfloor$ toward the slower-converging tasks, without touching loss weights. (3) Model Merge: TIES-Merging over low-rank experts across checkpoints to resolve conflicting parameter updates. The embedding branch is trained with InfoNCE on mean-pooled last-layer representations and the generative branch with token-level cross-entropy, both on the same backbone.

What would settle it

Measure entity overlap across task boundaries: for each pair of tasks, count how many user IDs, item IDs, or review texts occur in one task's training samples and the other task's test samples. If, for instance, an item that appears in a U2I or I2I training sample also appears as the ground-truth next item in an SR test sample (or in its candidate set), re-run the comparison on a strictly entity-disjoint re-split; if RecFound's margins shrink or invert, the state-of-the-art claim rests on leakage rather than multi-task transfer.

Watch

Extended reading notes

Core claim

The paper's central claim is that RecFound achieves the best average performance on both branches of its own benchmark: average rank 1.30 across the ten generative tasks and 1.33 across the three embedding tasks, ahead of finetuned unified baselines (GritLM, eCeLLM) and of far larger untuned models (Qwen2.5-32B-Instruct, Mixtral-8x7B). Ablations show that each of the three modules contributes: removing S2Sched drops the average generative score from 0.703 to 0.679 and the average embedding score from 0.775 to 0.762, and removing TMoLE drops them to 0.678 and 0.759; the model-merge step adds a small further gain. The authors also observe that the router learns task structure by itself, assigning similar routing distributions to related tasks and divergent ones to unrelated tasks, and that the scheduler's per-task sample ratios rise and fall alternately, which they interpret as protection against forgetting.

Load-bearing premise

The load-bearing premise is that the RecFound dataset's train/validation/test split is leakage-free across tasks: the same user or item, drawn from the shared source corpora, must never appear in one task's training data and another task's test data; the paper states only that it split the dataset into training, validation, and testing sets and never shows that overlapping entities are partitioned consistently.

Editorial extensions

If this is right

  • A single trained model can serve both the retrieval (embedding) and user-facing text (generative) stages of a recommendation pipeline, removing the need to maintain separate embedding and generation models.
  • Task-grouped routing over low-rank experts is sufficient to reconcile knowledge sharing and conflict across 13 recommendation tasks in a 7B backbone, at parameter-efficient cost.
  • Adaptively reallocating sample budgets by validation-loss slope outperforms both uniform sampling and direct loss reweighting when tasks use incomparable loss functions (InfoNCE versus cross-entropy).
  • The three modules are each load-bearing: ablations that remove S2Sched and TMoLE independently degrade both branches, so the state-of-the-art result is attributed to the combination rather than to any single component.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the authors did not run: re-split the dataset so each user and item appears in exactly one of train/validation/test across all 13 tasks, since all tasks are carved from the same five source corpora and the paper never shows that overlapping entities are partitioned consistently. If RecFound's margins shrink on that split, part of the reported gains would be entity memorization rather
  • The TMoLE-plus-S2Sched recipe is not recommendation-specific: any multi-task LLM setup with incommensurable losses and uneven convergence (search-plus-QA, retrieval-augmented generation) could reuse the same task-masked expert grouping and slope-based sample reallocation.
  • The router keys on a fixed task-ID embedding table, so the framework cannot absorb new tasks without retraining; the authors' own limitation note (sensitivity to task data imbalance, no continual task expansion) points at where the design would need to grow—for example, deriving the mask from instruction text or adding experts dynamically.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. RecFound proposes a unified generative-representational framework for recommendation foundation models. The paper contributes a new benchmark dataset covering 13 tasks (3 embedding and 10 generative) assembled from Amazon Reviews, MovieLens, Shopping Queries, and related sources; a Task-wise Mixture of Low-rank Experts (TMoLE) module to separate and share knowledge across tasks; a Step-wise Convergence-Oriented Sample Scheduler (S2Sched) that adjusts per-task sampling ratios from validation loss slopes; and a Model Merge module based on TIES-Merging to balance checkpoint specialization. Experiments compare RecFound against general-purpose LLMs, GritLM, eCeLLM, and an embedding-only baseline, reporting improved average ranks on both generative and embedding tasks, with ablations supporting the three proposed components. The authors state they will release code, data, and checkpoints.

Significance. If the claims hold, RecFound is a useful step toward recommendation foundation models that handle both embedding and generative tasks in one backbone, and the released dataset could become a reference benchmark for the community. The paper's strengths are its broad task coverage, the explicit attempt to handle knowledge conflict and convergence imbalance in multi-task training, and the promised public release of code, data, and checkpoints. The significance is currently tempered, however, because the SOTA claims rest entirely on the authors' own new benchmark, the train/test split is not shown to be entity-disjoint across tasks, and the main tables contain an unresolved numerical inconsistency. The central ideas are plausible, but the empirical evidence needs to be made internally consistent and leakage-free before the claims can be accepted.

major comments (5)
  1. [§3, Tables 2 and 3] The RecFound dataset is split into training, validation, and testing sets without any stated user- or item-disjoint partitioning, even though all 13 tasks are derived from overlapping sources (Amazon Reviews, MovieLens, Shopping Queries). Because the same user or item can appear in the training data of one task and the test data of another task, the shared backbone can memorize entities (item titles, user sequences) seen in other tasks' training splits, which is especially acute for retrieval-oriented tasks such as U2I, I2I, and SR. The paper should report cross-task entity-overlap statistics and re-run the main comparison on splits where no test entity appears in any task's training set, or otherwise justify that such leakage cannot affect the reported SOTA gains.
  2. [§5.2 vs Appendix C, Tables 3 and 6] The same configuration 'RecFound' reports inconsistent numbers in Table 3 and Table 6. Table 3 gives Item Emb 0.824, Query Emb 0.988, User Emb 0.570, while Table 6 gives 0.823, 0.993, 0.510 for the same model; the embedding average 0.775 in Table 4 matches Table 6, not Table 3. This discrepancy affects the main results and the ablation comparison, so it must be resolved before the SOTA claim can be assessed.
  3. [§4.3, Eqs. (3)-(6)] The sign convention in the non-convergence estimator is unclear and appears inconsistent with the stated intent. If α_i(t) is the validation loss slope, then a task that is still improving has negative α, while a task whose loss is rising (for example due to overfitting) has positive α. Eq. (3) maps positive α to negative γ, so the statement that 'higher γ indicates less convergence' does not hold under the usual sign convention, and Eq. (6)'s -β γ_inter term would tend to up-weight tasks with negative slopes. The authors should define the normalized slope precisely, state what positive and negative α mean, and show explicitly that the scheduler up-weights genuinely non-converged tasks. In addition, the Softmax_t in Eq. (5) is not defined: the index set over which the softmax is computed is missing.
  4. [§5.2, Tables 2, 3, 5, 6] All reported metrics are single point estimates without error bars or significance tests. Several headline differences are small (e.g., AVE 0.344 vs 0.341, AG 0.853 vs 0.843, SA 0.533 vs 0.532 in Table 2), so without variance estimates or significance tests it is not possible to determine whether RecFound's SOTA claim is statistically meaningful. Please add multiple seeds, confidence intervals, or significance tests for the main comparisons and the ablations.
  5. [§4.4] The Model Merge module is not specified at the implementation level. The text says TIES-Merging is applied 'over the learned low-rank experts across different checkpoints,' but it does not state how many checkpoints are merged, at which training steps they are selected, whether the base backbone weights are also merged, or how the merged experts interact with the trained router. Without these details, the 'w/o M.M.' ablation and the main RecFound results cannot be fully reproduced.
minor comments (4)
  1. [§4.4, §5.2, §4.3.2] There are several typos: 'perfromance' in §4.4, 'S2Shed' in §5.2, and 'noramlized' in §4.3.2.
  2. [Table 2] In the QR column, eCeLLM and RecFound both report F1 = 0.973, but only RecFound is bolded; if ties are allowed, both should be marked, and the text should not describe RecFound as best on that task without qualification.
  3. [Eq. (1) and surrounding text] The statement that Mask(·) suppresses 'irrelevant task dimensions' by assigning them −∞ is loose; it is the logits before the softmax that are set to −∞, not dimensions of the routing distribution itself.
  4. [§5.1] The description of S2Sched says 'we uniformly sample 128 validation data instances' but does not clarify whether this sample is drawn per task or per task type; please specify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RecFound is an empirical benchmark paper whose SOTA claims rest on held-out test evaluation, not on fitted inputs, definitional equivalences, or a self-citation chain.

full rationale

This is an empirical systems paper; there is no derivation chain whose conclusion is built into its premises. The core claims are that RecFound obtains state-of-the-art results on its constructed dataset, that TMoLE handles knowledge sharing and conflict, and that S2Sched balances convergence across tasks. Each module is defined by equations (Eqs. 1-6) whose inputs are task embeddings, routing masks, and validation-loss slopes; the reported test metrics come from the testing split described in Section 3 and Tables 2-3, and no parameter is fitted to the test labels and then reported as a prediction. The dataset is constructed by the authors and used to evaluate their own method, which is a standard benchmark-construction setting; it weakens external comparability with independently established benchmarks but is not circular reasoning. Self-citations to prior SJTU/Huawei work (e.g., [18, 29, 31, 55, 59-62]) appear in related-work context and do not carry the central SOTA claim. The limitation section's admission of sensitivity to task-specific data imbalance and lack of continual task expansion is a robustness caveat, not evidence of circularity. A possible entity overlap between training and test across the thirteen tasks (since all tasks derive from Amazon Reviews, MovieLens, and Shopping Queries) is a data-leakage risk that should be investigated, but it is an evaluation validity concern rather than a circular reduction of the kind this pass flags.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The framework rests on hand-set hyperparameters, on the assumption that the new RecFound benchmark measures what it claims, and on an unstated leakage-free split. No new physical or metaphysical entities are introduced; TMoLE and S2Sched are trainable modules rather than entities with independent falsifiable handles.

free parameters (6)
  • S2Sched temperature tau = 10
    Hand-set hyperparameter in the balancing weight Eq. (5); affects how quickly the scheduler shifts between inter-task and intra-task non-convergence signals.
  • History window L = 64
    Window length for computing normalized validation loss slope via linear regression in Section 4.3.1.
  • Warmup ratio ς = 10%
    Fraction of training during which no scheduling is applied, set in Section 5.1.
  • Number of experts per projection = N=6, with 2 embedding, 2 generative, 2 shared
    Architectural choice for TMoLE, Section 5.1; no sensitivity analysis is provided.
  • LoRA rank and alpha = r=16, alpha=64, dropout=0.1
    LoRA configuration for TMoLE adapters, adopted from standard LoRA defaults with no tuning study.
  • Validation sample size for loss estimation = 128
    Number of validation instances uniformly sampled per task per step to compute normalized validation loss, Section 5.1.
assumptions (5)
  • domain assumption The RecFound dataset, assembled from public sources, is a coherent and representative benchmark for recommendation foundation models.
    Section 3 states the dataset unifies 13 tasks but offers no external validation that these tasks measure general recommendation ability.
  • ad hoc to paper Validation loss slope over the last L steps is a reliable measure of task convergence and can guide sample scheduling.
    Section 4.3.1 builds the entire scheduler on this assumption, but no evidence is given that validation loss slope predicts final task performance.
  • domain assumption No cross-task entity leakage occurs between training and test splits of the combined dataset.
    Section 3 only says the data is split into training, validation, and testing; it does not describe de-duplicating users or items across tasks.
  • domain assumption InfoNCE loss for embedding tasks and cross-entropy loss for generative tasks can be jointly optimized on a shared decoder-only backbone without destructive interference.
    Section 4.1 states the two branches share the same backbone and gradients are jointly backpropagated, but no analysis of gradient conflict magnitude is provided.
  • standard math Standard mathematical operations such as softmax normalization and linear regression behave as expected.
    Used throughout S2Sched in Eqs. (3)-(6); no specialized mathematical claims are made.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative Representational Learning of Foundation Models for Recommendation." pith.science (2026). https://pith.science/paper/GGDHLROK

@misc{pith2026250611999,
  author       = {Pith},
  title        = {Pith review of: Generative Representational Learning of Foundation Models for Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GGDHLROK}},
  note         = {Machine review of arXiv:2506.11999}
}
read the original abstract

Developing a single foundation model with the capability to excel across diverse tasks has been a long-standing objective in the field of artificial intelligence. As the wave of general-purpose foundation models sweeps across various domains, their influence has significantly extended to the field of recommendation systems. While recent efforts have explored recommendation foundation models for various generative tasks, they often overlook crucial embedding tasks and struggle with the complexities of multi-task learning, including knowledge sharing & conflict resolution, and convergence speed inconsistencies. To address these limitations, we introduce RecFound, a generative representational learning framework for recommendation foundation models. We construct the first comprehensive dataset for recommendation foundation models covering both generative and embedding tasks across diverse scenarios. Based on this dataset, we propose a novel multi-task training scheme featuring a Task-wise Mixture of Low-rank Experts (TMoLE) to handle knowledge sharing & conflict, a Step-wise Convergence-oriented Sample Scheduler (S2Sched) to address inconsistent convergence, and a Model Merge module to balance the performance across tasks. Experiments demonstrate that RecFound achieves state-of-the-art performance across various recommendation tasks, outperforming existing baselines.

Figures

Figures reproduced from arXiv: 2506.11999 by the authors.

Figure 1
Figure 1. The Method of RecFound. In TMoLE structure, we assume we use 2 embedding experts, 2 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Similarity heatmaps of the routing distribution across different tasks. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. The evolving sample ratio per batch of different tasks during the training. The training step [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Large Foundation Model for Ads Recommendation

    cs.LG 2025-08 conditional novelty 5.0 of 10

    Tencent's LFM4Ads transfers user, item, and user-item cross representations from a pre-trained foundation model into downstream ad models via feature, module, and model-level mechanisms, reporting a 2.45% platform-wid...

Reference graph

Works this paper leans on

69 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [1]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Eleanor Adlam, Atticus von Werra, Zachary Mahoney, Aran Neeraj, Eric Roberts, Stephen Casais, Roger Mouawad, Joel Batten, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  2. [2]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and et al

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  3. [3]

    Deep variational embedding representation on neural collaborative filtering for recommender systems

    Li Chen et al. Deep variational embedding representation on neural collaborative filtering for recommender systems. Applied Sciences, 12(9):4168, 2022

    work page 2022

  4. [4]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171–4186, 2019

    2019

  5. [5]

    L. Dong, Y . Shi, and T. Liu. Neural review generation for personalized recommendations. In CIKM, 2017

    work page 2017

  6. [6]

    Sequence-to-sequence learning for review generation and recommendation

    Lifu Dong, Tianbao Zhang, Siwei Xu, Yun Li, Shengqiang Liu, Jun Guo, Yi Zhu, and Kewei Wang. Sequence-to-sequence learning for review generation and recommendation. In Pro- ceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1109–1112, 2017

    work page 2017

  7. [7]

    Loramoe: Alleviate world knowledge forgetting in large language models via moe-style plugin, 2024

    Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Jun Zhao, Wei Shen, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. Loramoe: Alleviate world knowledge forgetting in large language models via moe-style plugin, 2024

    work page 2024

  8. [8]

    Coba: Conver- gence balancer for multitask finetuning of large language models, 2024

    Zi Gong, Hang Yu, Cong Liao, Bingchang Liu, Chaoyu Chen, and Jianguo Li. Coba: Conver- gence balancer for multitask finetuning of large language models, 2024

    work page 2024

Show all 69 references
  1. [9]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407, 2024

    2024

  2. [10]

    Amazonqa: A review-based question answering task

    Mansi Gupta, Nitish Kulkarni, Raghuveer Chanda, Anirudha Rayasam, and Zachary C Lipton. Amazonqa: A review-based question answering task. arXiv preprint arXiv:1908.04364, 2019

    1908 arXiv

  3. [11]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 5(4), December 2015

    2015

  4. [12]

    Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering

    Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web, pages 507–517, 2016

    2016

  5. [13]

    Session-based recommendations with recurrent neural networks

    Balázs Hidasi, Alexandros Karatzoglou, Dalius Baltrunas, and Dániel Tikk. Session-based recommendations with recurrent neural networks. In International Conference on Learning Representations, 2015

    2015

  6. [14]

    Bridging language and items for retrieval and recommendation

    Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation. arXiv preprint arXiv:2403.03952, 2024

    2024 arXiv

  7. [15]

    Towards universal sequence representation learning for recommender systems

    Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 585–593, 2022

    2022

  8. [16]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021

    2021

  9. [17]

    Foun- dation models for recommender systems: A survey and new perspectives

    Chengkai Huang, Tong Yu, Kaige Xie, Shuai Zhang, Lina Yao, and Julian McAuley. Foun- dation models for recommender systems: A survey and new perspectives. arXiv preprint arXiv:2402.11143, 2024. 10

    2024 arXiv

  10. [18]

    A comprehensive survey on retrieval methods in recommender systems

    Junjie Huang, Jizheng Chen, Jianghao Lin, Jiarui Qin, Ziming Feng, Weinan Zhang, and Yong Yu. A comprehensive survey on retrieval methods in recommender systems. arXiv preprint arXiv:2407.21022, 2024

    2024

  11. [19]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023

  12. [20]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne...

    2024

  13. [21]

    Unsupervised tag recommendation for popular and cold products

    Anand Konjengbam, Nagendra Kumar, and Manish Singh. Unsupervised tag recommendation for popular and cold products. Journal of Intelligent Information Systems, 54:545–566, 2020

    2020

  14. [22]

    Evaluation of entity resolution approaches on real-world match problems

    Hanna Köpcke, Andreas Thor, and Erhard Rahm. Evaluation of entity resolution approaches on real-world match problems. Proceedings of the VLDB Endowment, 3(1-2):484–493, 2010

    2010

  15. [23]

    Beyond distillation: Task-level mixture-of-experts for efficient inference, 2021

    Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh- Thang Luong, and Orhan Firat. Beyond distillation: Task-level mixture-of-experts for efficient inference, 2021

    2021

  16. [24]

    Supervised transfer learning for product information question answering

    Tuan Manh Lai, Trung Bui, Nedim Lipka, and Sheng Li. Supervised transfer learning for product information question answering. arXiv preprint arXiv:1901.02539, 2019

    1901 arXiv

  17. [25]

    Unigen: A unified generative framework for retrieval and question answering with large language models

    Xiaoxi Li, Yujia Zhou, and Zhicheng Dou. Unigen: A unified generative framework for retrieval and question answering with large language models. 2023. arXiv:2312.11036

    2023 arXiv

  18. [26]

    Ecomgpt: Instruction-tuning large language models with chain-of-task tasks for e-commerce

    Yangning Li, Shirong Ma, Xiaobin Wang, Shen Huang, Chengyue Jiang, Hai-Tao Zheng, Pengjun Xie, Fei Huang, and Yong Jiang. Ecomgpt: Instruction-tuning large language models with chain-of-task tasks for e-commerce. In Proceedings of the AAAI Conference on Artificial Intelligence...

  19. [27]

    Towards general text embeddings with multi-stage contrastive learning

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023

    2023 arXiv

  20. [28]

    Variational autoencoders for collaborative filtering

    Dawen Liang, Rahul Krishnan, Matthew D Hoffman, and Tony Jebara. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference , pages 689–697, 2018

    2018

  21. [29]

    Clickprompt: Ctr models are strong prompt generators for adapting language models to ctr prediction

    Jianghao Lin, Bo Chen, Hangyu Wang, Yunjia Xi, Yanru Qu, Xinyi Dai, Kangning Zhang, Ruiming Tang, Yong Yu, and Weinan Zhang. Clickprompt: Ctr models are strong prompt generators for adapting language models to ctr prediction. In Proceedings of the ACM Web Conference 2024, page...

    2024

  22. [30]

    How can recommender systems benefit from large language models: A survey

    Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Hao Zhang, Yong Liu, Chuhan Wu, Xiangyang Li, Chenxu Zhu, et al. How can recommender systems benefit from large language models: A survey. ACM Transactions on Information Systems, 43(2):1–47, 2025

    2025

  23. [31]

    Rella: Retrieval-enhanced large language models for lifelong sequential behavior comprehension in recommendation

    Jianghao Lin, Rong Shan, Chenxu Zhu, Kounianhua Du, Bo Chen, Shigang Quan, Ruiming Tang, Yong Yu, and Weinan Zhang. Rella: Retrieval-enhanced large language models for lifelong sequential behavior comprehension in recommendation. In Proceedings of the ACM Web Conference 2024, ...

    2024

  24. [32]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision , pages 2980–2988, 2017. 11

    2017

  25. [33]

    Fim: A fine-grained and multi-aspect recommendation model based on large language model

    Haoxi Liu, Yunjun Zhang, Xiaochen Yu, Xinxing Wang, Peng Cao, Jian Zhang, Yu Chen, and Haitao Huang. Fim: A fine-grained and multi-aspect recommendation model based on large language model. arXiv preprint arXiv:2308.04571, 2023

    2023 arXiv

  26. [34]

    Dora: Weight-decomposed low-rank adaptation, 2024

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation, 2024

    2024

  27. [35]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Li, Artem Kochanov, Omer Mahoney, Piotr Dollar, Veselin Stoyanov, et al. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  28. [36]

    Embedding in recommender systems: A survey

    Yuxin Liu et al. Embedding in recommender systems: A survey. arXiv preprint arXiv:2310.18608, 2023

    2023

  29. [37]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  30. [38]

    Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models, 2024

    Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models, 2024

    2024

  31. [39]

    MuDoCo: Corpus for multidomain coreference resolution and referring expression generation

    Scott Martin, Shivani Poddar, and Kartikeya Upasani. MuDoCo: Corpus for multidomain coreference resolution and referring expression generation. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Is...

    2020

  32. [40]

    Image-based recommendations on styles and substitutes

    Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 43–52, 2015

    2015

  33. [41]

    Sfr-embedding-mistral: Enhance text retrieval with transfer learning

    Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. Sfr-embedding-mistral: Enhance text retrieval with transfer learning. Salesforce AI Research Blog, 2024

    2024

  34. [42]

    Efficient estimation of word representations in vector space

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In ICLR Workshop, 2013

    2013

  35. [43]

    Generative representational instruction tuning, 2025

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning, 2025

    2025

  36. [44]

    ecellm: Generalizing large language models for e-commerce from large-scale, high-quality instruction data

    Bo Peng, Xinyi Ling, Ziru Chen, Huan Sun, and Xia Ning. ecellm: Generalizing large language models for e-commerce from large-scale, high-quality instruction data. In Proceedings of the 41st International Conference on Machine Learning. PMLR

  37. [45]

    Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543, 2014

    2014

  38. [46]

    Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Ken- ton Lee, and Luke Zettlemoyer

    Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Ken- ton Lee, and Luke Zettlemoyer. Deep contextualized word representations. arXiv preprint arXiv:1802.05365, 2018

    2018 arXiv

  39. [47]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. OpenAI Blog, 2018

    2018

  40. [48]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Technical Report, 2019

    2019

  41. [49]

    Benchmark datasets for entity resolution

    Rahm, Erhard. Benchmark datasets for entity resolution. https://dbs.uni-leipzig.de/ research/projects/benchmark-datasets-for-entity-resolution , 2010. 12

    2010

  42. [50]

    Shopping queries dataset: A large-scale esci benchmark for improving product search

    Chandan K Reddy, Lluís Màrquez, Fran Valero, Nikhil Rao, Hugo Zaragoza, Sambaran Bandy- opadhyay, Arnab Biswas, Anlu Xing, and Karthik Subbian. Shopping queries dataset: A large-scale esci benchmark for improving product search. arXiv preprint arXiv:2206.06588, 2022

    2022 arXiv

  43. [51]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pages 3982–3992, 2019

    2019

  44. [52]

    Large language models are zero-shot recommenders

    Shijie Shen, Yunjia Ma, Jian Retrieval Zhang, Zhuoyi Zhang, Zihan Wang, Liangkun Li, Shuyu Deng, Yunfan Zhang, Jiaxing Zhang, Shaohan Jiang, et al. Large language models are zero-shot recommenders. arXiv preprint arXiv:2301.00794, 2023

    2023 arXiv

  45. [53]

    Llama-e: Empowering e-commerce authoring with object-interleaved instruction following, 2024

    Kaize Shi, Xueyao Sun, Dingxian Wang, Yinlin Fu, Guandong Xu, and Qing Li. Llama-e: Empowering e-commerce authoring with object-interleaved instruction following, 2024

    2024

  46. [54]

    Quality metrics in recommender systems: Do we calculate metrics consistently? In Proceedings of the 15th ACM conference on recommender systems, pages 708–713, 2021

    Yan-Martin Tamm, Rinchin Damdinov, and Alexey Vasilev. Quality metrics in recommender systems: Do we calculate metrics consistently? In Proceedings of the 15th ACM conference on recommender systems, pages 708–713, 2021

    2021

  47. [55]

    Flip: Fine-grained alignment between id-based models and pretrained lan- guage models for ctr prediction

    Hangyu Wang, Jianghao Lin, Xiangyang Li, Bo Chen, Chenxu Zhu, Ruiming Tang, Weinan Zhang, and Yong Yu. Flip: Fine-grained alignment between id-based models and pretrained lan- guage models for ctr prediction. In Proceedings of the 18th ACM Conference on Recommender Systems, pa...

    2024

  48. [56]

    Irgan: A minimax game for unifying generative and discriminative information retrieval models

    Jun Wang, Lantao Yu, Xiang Zhang, Yunhai Gong, Yingguo Xu, Bo Wang, Shi Zhang, Zhongfei Zhang, Yongfneg Li, and Chen Haifeng. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR Confe...

    2017

  49. [57]

    Unirec: Unified and fine-grained recommendation with large language model

    Xinxing Wang, Haoxi Liu, Nan Xu, Xiaochen Yu, Ruiming Wu, Yunjun Zhang, Jian Zhang, Haitao Huang, and Yu Chen. Unirec: Unified and fine-grained recommendation with large language model. arXiv preprint arXiv:2401.14614, 2024

    2024 arXiv

  50. [58]

    Robust training objectives improve embedding-based retrieval in industrial recommendation systems

    Yan Wu et al. Robust training objectives improve embedding-based retrieval in industrial recommendation systems. arXiv preprint arXiv:2409.14682, 2023

    2023 arXiv

  51. [59]

    Towards open-world recommendation with knowledge augmentation from large language models

    Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. Towards open-world recommendation with knowledge augmentation from large language models. In Proceedings of the 18th ACM Conference on Recommender System...

    2024

  52. [60]

    Memocrs: Memory-enhanced sequential conversational recommender systems with large language models

    Yunjia Xi, Weiwen Liu, Jianghao Lin, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. Memocrs: Memory-enhanced sequential conversational recommender systems with large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Manage...

    2024

  53. [61]

    Efficient and deployable knowledge infusion for open-world recommendations via large language models

    Yunjia Xi, Weiwen Liu, Jianghao Lin, Muyan Weng, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Yong Yu, et al. Efficient and deployable knowledge infusion for open-world recommendations via large language models. ACM Transactions on Recommender Systems, 2024

    2024

  54. [62]

    A decoding acceleration framework for industrial deployable llm-based recommender systems

    Yunjia Xi, Hangyu Wang, Bo Chen, Jianghao Lin, Menghui Zhu, Weiwen Liu, Ruiming Tang, Weinan Zhang, and Yong Yu. A decoding acceleration framework for industrial deployable llm-based recommender systems. arXiv preprint arXiv:2408.05676, 2024

    2024 arXiv

  55. [63]

    C-pack: Packaged resources to advance general chinese embedding, 2023

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding, 2023

    2023

  56. [64]

    Scaling up open tagging from tens to thousands: Comprehension empowered attribute value extraction from product title

    Huimin Xu, Wenting Wang, Xin Mao, Xinyu Jiang, and Man Lan. Scaling up open tagging from tens to thousands: Comprehension empowered attribute value extraction from product title. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages ...

    2019

  57. [65]

    Ties-merging: Resolving interference when merging models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36:7093–7115, 2023

    2023

  58. [66]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  59. [67]

    X. Yang, Y . Zhang, and B. Li. Behavior-driven item description generation. In WWW, 2018

    2018

  60. [68]

    Llasa: Large language and e-commerce shopping assistant, 2024

    Shuo Zhang, Boci Peng, Xinping Zhao, Boren Hu, Yun Zhu, Yanjia Zeng, and Xuming Hu. Llasa: Large language and e-commerce shopping assistant, 2024

    2024

  61. [69]

    [INST]". For Llama-3-8B-Instruct and Qwen-2.5-Instruct series, the prompt is wrapped with

    Gaojie Zhou, Yupeng Xu, Wayne Xin Ding, Ying Wang, Zijian Zhang, Longxing Wang, Tianyu Yao, Yujie Zhu, Jiaqi Song, Zhengyu Bi, et al. Large language models for recommendation: A survey. arXiv preprint arXiv:2401.12397, 2024. 14 A Broader Impact As a unified generative represen...

    1995 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.