Learning from Online User Feedback for Shopping Agents

Haobo Zhang; Kelong Mao; Simiu Gu; Sulong Xu; Zhicheng Dou

REVIEW 4 major objections 6 minor 32 references

Learning from Online User Feedback for Shopping Agents

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A shopping agent can be trained automatically from real user logs, using purchases as outcome rewards and conversational corrections as token-level teaching signals.

desk verdict Real-log learning loop with a plausible design, but the LLM-judge circularity and a missing variance estimate leave the headline dialogue gains unproven. read the letter →

arxiv 2608.11604 v1 pith:FGMGU6JC submitted 2026-08-12 cs.AI

Haobo Zhang , Kelong Mao , Sulong Xu , Simiu Gu , Zhicheng Dou This is my paper · ORCID

classification cs.AI

keywords shoppingagentsconversationalrecommendationonlineuserfeedbackreinforcementlearningGRPOpolicydistillationLLMmininge-commerceinteractionlogs

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LOFA tries to show that deployed shopping agents can be improved directly from their own real interaction logs, with no human annotation. The central step is to treat two kinds of user feedback as complementary supervision: whether a conversation ended in a purchase, and what the user said in later turns when the agent's answer missed. Purchase outcomes are turned into a ranking reward, while conversational corrections are mined by a large language model into feedback categories and explanations that construct a privileged teacher. The teacher then guides the student's own generated replies through token-level distillation. If the claims hold, shopping agents could continuously retrain from the complaints and purchases recorded in their daily logs instead of relying on static offline datasets.

What carries the argument

The mechanism is a two-stage training loop. First, behavioral feedback learning turns sessions that ended in a purchase into reinforcement-learning contexts, where the agent samples recommendations and receives a format reward for valid output plus a ranking reward that favors putting the purchased item near the top; the update uses GRPO, a policy-gradient method that scores a sampled group of responses against the verifiable purchase signal. Second, directive feedback learning augments each turn with privileged future information — the agent's previous response, the user's next message, the mined feedback category, and a generated explanation — to form a teacher; the deployed student generates its own reply, and the teacher's token distribution over that same reply is used as dense supervision via reverse-KL divergence. The load-bearing identity is that sparse, noisy natural-language corrections become dense teacher signals only when the teacher sees the user's subsequent utterance and a category label for it.

What would settle it

Take a random sample of the directive-feedback test instances and have human annotators independently label the feedback category and judge whether each revised response resolves the user's complaint; recompute Success Rate with human labels. If LOFA's advantage over baselines shrinks to near zero or reverses under human judgment, the reported gains are artifacts of the automated judge rather than real user-satisfaction improvements.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that real online user logs contain two complementary supervision signals — outcome-level purchase signals and turn-level directive feedback (explicit criticism, implicit deduction, pure negative feedback, and comparative preference) — and that jointly learning from both, sequentially, outperforms learning from either alone. Concretely, optimizing the agent with GRPO over sessions ending in purchases substantially improves recommendation ranking metrics, while feedback-aware on-policy distillation from the user's next utterance substantially improves the rate at which revised responses resolve the user's stated concern. Combining the two in the order purchase-RL first, distillation second yields the best ranking and success-rate numbers across both a 4-billion- and an 8-billion-parameter backbone.

Load-bearing premise

The load-bearing premise is that the large language model used to mine feedback categories and to judge whether revised responses are successful is accurate enough that its decisions track real user satisfaction; the paper reports no human validation of either role, so if that model is biased or lenient, the reported gains could be artifacts.

Editorial extensions

If this is right

Deployed shopping agents can be retrained from their own logs on a schedule, because both purchase outcomes and conversational feedback are extracted automatically.
Purchase-level reinforcement learning and turn-level distillation improve different capabilities: RL mainly lifts recommendation ranking, while distillation mainly lifts response helpfulness; the combined LOFA pipeline gets both.
Training order matters: learning reliable outcome-level signals before fine-grained directive signals beats the reverse order, so the sequential pipeline is itself part of the method.
Reasoning traces are not required for feedback-driven optimization; the feedback-learning methods outperform supervised fine-tuning on response-only data while preserving the model's own reasoning.
All four directive feedback categories contribute, with implicit deduction providing the largest gains, so discarding or underweighting any one type would cost performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: because the same large language model both mines feedback categories and judges success, some of the reported gap may come from self-consistency rather than from genuinely better responses; an independent human-judged subset would reveal the true effect size.
Beyond the paper: the four-category feedback taxonomy and the privileged-teacher recipe should transfer to other deployed conversational assistants, where users routinely correct the agent in the next turn.
Beyond the paper: a natural extension is to feed richer outcome signals — returns, refunds, repeated queries, or cart abandonment after acceptance — into the reward, since purchase completion alone is a delayed and partial proxy for satisfaction.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know about this paper: it trains shopping agents directly on real JD interaction logs without human annotation, by pairing purchase-outcome GRPO with on-policy distillation from a teacher that sees the user's next-turn feedback. That combination is new, and the ranking half of the results looks solid. The dialogue-helpfulness half is shakier, because the only metric for it is a Success Rate judged by the same LLM that creates the training signal.

What's genuinely good: the data are real online logs, not synthetic. The purchase-outcome reward is verifiable, and the ranking gains on NDCG, Recall, and MAP are large and consistent across Qwen3-4B and Qwen3-8B. The feedback taxonomy (explicit criticism, implicit deduction, pure negative, comparative preference) is reasonable, and the ablations show each category contributes. The privileged-teacher construction is a sensible way to convert sparse user complaints into dense token-level supervision. The 'data flywheel' framing is practically valuable for deployment.

The soft spots are proportionate. The main evaluation of helpfulness and satisfaction alignment is the Success Rate computed by DeepSeek-V3.2, the same model used for feedback mining and for writing the teacher's explanations. That is a mild circularity, and there is no human agreement study or second judge to check it. The Comparative Preference test set has 17 instances, so a one-sample shift flips the ranking; yet the paper claims RL-to-OPD 'consistently outperforms' OPD-to-RL while Table 4 shows OPD-to-RL winning on Explicit Criticism and Comparative Preference. No confidence intervals or significance tests appear anywhere. Data and code are not released. These issues are fixable and they do not undermine the ranking results, which depend only on purchase outcomes.

I agree with the stress-test note: the declared helpfulness/alignment gains rest entirely on an unvalidated judge, and the 'consistently' claim overreaches. Still, the core framework is plausible and the empirical story on ranking is credible. This paper is for people who build deployed shopping agents and for researchers working on learning from interaction logs. It deserves a serious referee, but I'd send it back with a request for human-annotated SR on a sample, error bars, and a softened ordering claim. If the authors can supply those, it becomes a practical contribution.

Referee Report

4 major / 6 minor

Summary. The paper proposes LOFA, a framework for learning a shopping agent from real-world e-commerce interaction logs without human annotation. It combines two supervision sources: (i) explicit behavioral feedback in the form of purchase outcomes, optimized with GRPO under format and NDCG-based ranking rewards; and (ii) in-dialogue directive feedback, where an LLM (DeepSeek-V3.2) annotates each user follow-up into one of four actionable categories (explicit criticism, implicit deduction, pure negative feedback, comparative preference) plus topic shift/abandonment, together with an explanation. The annotated feedback is used to construct a teacher that sees privileged future feedback and distills corrections into the student via on-policy reverse-KL distillation. Experiments on JD.com 'Jingyan' logs compare Qwen3-8B variants (and Qwen3-4B) on ranking metrics and an LLM-judged Success Rate. The main findings are that RL improves ranking metrics, OPD improves Success Rate, and the combined RL→OPD pipeline gives the best overall numbers, with ablations showing the contribution of each reward and feedback category.

Significance. If the results hold, LOFA offers a practical route to exploit abundant online interaction logs for shopping-agent improvement without human annotation, treating purchase outcomes as verifiable rewards and conversational corrections as dense token-level supervision. The framework is novel in combining GRPO on real purchase data with feedback-aware on-policy distillation, and the paper clearly targets an industrial deployment scenario. Strengths include the use of real purchase outcomes as objective ranking targets, the category-level ablations showing that each directive-feedback type contributes, and the evaluation across two backbone scales. However, the reported helpfulness and alignment gains rest entirely on an LLM judge that is also used to create the training labels, and the paper lacks statistical grounding; the central claims therefore need additional validation before the conclusions can be accepted.

major comments (4)

[Section 4.2, observation (3); Table 4] The paper states that 'RL→OPD consistently outperforms OPD→RL' but Table 4 contradicts this on two of the four categories: on Explicit Criticism, OPD→RL achieves 0.5846 vs RL→OPD's 0.5692, and on Comparative Preference, OPD→RL achieves 0.5294 vs RL→OPD's 0.4706. The overall Success Rate margin (0.6022 vs 0.5924) is also small. This contradicts the 'consistently' claim and weakens the conclusion that outcome-level learning should precede directive learning. Please rephrase the claim, report per-category significance tests, or provide an analysis that reconciles the ordering.
[Sections 4.1.3, 3.3.1, 4.1.4] The Success Rate metric is computed by prompting DeepSeek-V3.2 (Section 4.1.3), and the same model is used in Section 3.3.1 to mine the feedback categories and explanations that build the teacher. Because the student is trained to imitate a teacher constructed from DeepSeek-labeled directives and is then evaluated by DeepSeek, the reported SR gains may reflect the judge's preferences rather than genuine user satisfaction. The paper reports no human agreement study, no alternative judge, and no error analysis for the judge. Please add a human-annotated sample of test instances scored by independent raters with agreement statistics, and/or report SR using a second independent evaluator; without this, the helpfulness and alignment claims are not verifiable.
[Section 4.1.4; Tables 3-6] The text says 'All results are averaged over multiple runs' but no variance, confidence intervals, or significance tests are reported anywhere. Most headline differences are small, for example overall SR 0.6022 vs 0.5924, NDCG@10 0.6512 vs 0.6386, and Recall@10 0.8911 vs 0.8595. Without measures of uncertainty, 'consistent improvements' cannot be distinguished from run-to-run noise. Please report means with standard deviations across runs and paired significance tests (e.g., bootstrap or permutation) for the main comparisons.
[Table 2; Sections 4.2-4.3] The Comparative Preference test set contains only 17 instances (Table 2). Consequently, the CP rows in Table 4 (e.g., 0.4706 vs 0.5294) represent differences of one or two responses and cannot support any reliable conclusion, including the claim that gains on Comparative Preference are 'slightly smaller.' Please either enlarge the CP test set or explicitly state that CP results are anecdotal and exclude CP from aggregate conclusions.

minor comments (6)

[Figure 1; Table 7] Figure 1 contains the typo 'snecific reason' and Table 7 case 2 contains 'understant'; both should be corrected.
[Section 4.2] The observations are enumerated (1) through (4) and then jump to (6), skipping (5); renumber the list.
[Tables 3 and 5] The column headers read 'MAP@1, 10, 20' but the tables list MAP@1, MAP@5, and MAP@10; align the headers with the actual columns.
[References] References [24] and [25] are identical duplicates of the same Christiano et al. paper; remove one of them.
[Section 4.1.2 and Table 3] The baseline 'Qwen3-8B-reflect' appearing in Table 3 is not defined in the compared-methods list, which only mentions a 'Self-Reflection' baseline; add a description or rename it for consistency.
[Section 3.3.3] The on-policy distillation objective is presented without an equation number, making it awkward to reference in later sections; consider numbering it.

Circularity Check

1 steps flagged · score 4.0 of 10

Mild circularity: DeepSeek-V3.2 both mines the directive-feedback labels that supervise the student and judges the Success Rate used to claim helpfulness gains; recommendation-quality results remain independent.

other [Sections 3.3.1–3.3.2 and 4.1.3–4.1.4]
"We denote the identified feedback category as c_t and the generated explanation as e_t, which are subsequently used to construct feedback-aware supervision. ... For directive-feedback mining and response evaluation, we use DeepSeek-V3.2 [33] as the judge model."

The teacher that supervises the student via on-policy distillation is built on privileged inputs (c_t, e_t) produced by DeepSeek-V3.2's feedback miner (Sec. 3.3.1-3.3.2). The Success Rate metric in Sec. 4.1.3 is then computed by prompting 'an external LLM evaluator'—the same DeepSeek-V3.2, per Sec. 4.1.4—to judge whether the revised response 'successfully resolves the identified issue.' Thus the training target (match a teacher conditioned on DeepSeek's interpretation of user feedback) and the evaluation target (DeepSeek's judgment of resolution) are generated by the same model.

full rationale

The core learning signals are anchored to real external data: purchase outcomes (Section 3.2) and actual user utterances (Section 3.3). The GRPO reward is NDCG@10 with the purchased item as target, so the recommendation-quality gains in Table 3 are not circular. The one substantive circularity concern is the shared DeepSeek-V3.2 judge/miner for the directive-feedback path: the same model generates the teacher's privileged category/explanation and evaluates SR, so the SR-based claims of helpfulness/alignment are partially self-confirming. This does not fully reduce to fitting the target metric because the student never sees c_t/e_t at inference and must generalize on held-out logs, but it weakens the independence of the SR evaluation. The only self-citation ([15]) is in related work and is not load-bearing. Overall, the derivation is largely self-contained; the circularity is limited to the SR evaluation channel.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central empirical claims rest on domain assumptions about the validity of purchase labels, the reliability of LLM-generated feedback labels and evaluations, and the usefulness of the privileged teacher. None of these are proven in the paper, and no human evaluation is reported.

assumptions (3)

domain assumption The purchased item i* is a valid single target for ranking quality.
Equation (2) sets NDCG@10 with i* as the only target; purchase decisions can be driven by price, stock, or external factors not reflecting recommendation quality.
ad hoc to paper DeepSeek-V3.2's feedback categorization and success judgments are accurate enough to serve as training labels and evaluation metrics.
Sections 3.3.1 and 4.1.3 use the same model family for mining and judging without human validation.
domain assumption The teacher, given privileged future feedback, produces responses that are genuinely better for the student to imitate.
Section 3.3.2 constructs the teacher by adding at, qt+1, ct, and et; if the teacher's revisions are not actually better, reverse-KL distillation in Section 3.3.3 could reinforce errors.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning from Online User Feedback for Shopping Agents." pith.science (2026). https://pith.science/paper/FGMGU6JC

@misc{pith2026260811604,
  author       = {Pith},
  title        = {Pith review of: Learning from Online User Feedback for Shopping Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FGMGU6JC}},
  note         = {Machine review of arXiv:2608.11604}
}

read the original abstract

Large language model-based shopping agents are increasingly deployed in real-world e-commerce platforms, generating massive amounts of user interaction logs that provide valuable supervision for improving these agents. However, existing approaches primarily rely on offline training signals, such as user-item interactions or synthetic preference data, while largely overlooking the rich supervision contained in users' natural conversational feedback. Moreover, the available online feedback is heterogeneous, sparse, and noisy, making it difficult to transform into reliable learning signals automatically. To address these challenges, we propose LOFA, a framework that enables shopping agents to learn directly from real online interaction logs without human annotation. LOFA combines reinforcement learning over verifiable purchase outcomes with feedback-aware on-policy distillation, which identifies users'in-dialogue directives and converts them into dense token-level supervision. These complementary objectives capture both collaborative behavioral patterns and user-specific preferences. Extensive experiments on real-world e-commerce logs demonstrate that LOFA consistently improves recommendation quality, response helpfulness, and user-satisfaction alignment over strong baselines, highlighting the effectiveness of learning shopping agents from real online user feedback.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 8 canonical work pages

[1]

Bert4rec: Sequential rec- ommendation with bidirectional encoder representations from transformer

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential rec- ommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, page 1441–1450, New York, NY, USA, 2019. Association for Computing Machine...

arXiv 2019
[2]

Wang-Cheng Kang and Julian J. McAuley. Self-attentive sequential recommendation. InICDM, pages 197–206. IEEE Computer Society, 2018

work page 2018
[3]

BPR: bayesian personalized ranking from implicit feedback.CoRR, abs/1205.2618, 2012

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. BPR: bayesian personalized ranking from implicit feedback.CoRR, abs/1205.2618, 2012

arXiv 2012
[4]

S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization

Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 1893–1902, New York, NY, USA, 2020. Ass...

arXiv 1902
[5]

Llasa: Large language and e-commerce shopping assistant.CoRR, abs/2408.02006, 2024

Shuo Zhang, Boci Peng, Xinping Zhao, Boren Hu, Yun Zhu, Yanjia Zeng, and Xuming Hu. Llasa: Large language and e-commerce shopping assistant.CoRR, abs/2408.02006, 2024. doi: 10.48550/ARXIV.2408.02006. URLhttps://doi.org/10.48550/arXiv.2408.02006

arXiv doi:10.48550/arxiv.2408.02006 2024
[6]

A shopping agent for addressing subjective product needs

Preetam Prabhu Srikar Dammu, Omar Alonso, and Barbara Poblete. A shopping agent for addressing subjective product needs. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, page 1032–1035, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400713293. doi: 10.1145/3701551.3704124. URLht...

work page arXiv 2025
[7]

Language model alignment for conversational shop- ping at amazon

Chen Luo, Dimitri Papadimitriou, Hariharan Muralidharan, Dhineshkumar Ramasubbu, Aakash Kolekar, Wenju Xu, Cong Xu, Anirudh Srinivasan, Mukesh Jain, and Qi He. Language model alignment for conversational shop- ping at amazon. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, page ...

arXiv 2025
[8]

In Ee-Peng Lim, Marianne Winslett, Mark Sanderson, Ada Wai-Chee Fu, Jimeng Sun, J

Feng-Lin Li, Minghui Qiu, Haiqing Chen, Xiongwei Wang, Xing Gao, Jun Huang, Juwei Ren, Zhongzhou Zhao, Weipeng Zhao, Lei Wang, Guwei Jin, and Wei Chu.AliMe Assist : An intelligent assistant for creating an innovative e-commerce experience. In Ee-Peng Lim, Marianne Winslett, Mark Sanderson, Ada Wai-Chee Fu, Jimeng Sun, J. Shane Culpepper, Eric Lo, Joyce C....

arXiv 2017

Show all 32 references

[9]

R2ec: Towards large recommender models with reasoning.CoRR, abs/2505.16994, 2025

Runyang You, Yongqi Li, Xinyu Lin, Xin Zhang, Wenjie Wang, Wenjie Li, and Liqiang Nie. R2ec: Towards large recommender models with reasoning.CoRR, abs/2505.16994, 2025

2025
[10]

Reason-to-recommend: Using interaction-of-thought reasoning to enhance LLM recommendation

Keyu Zhao, Fengli Xu, and Yong Li. Reason-to-recommend: Using interaction-of-thought reasoning to enhance LLM recommendation. CoRR, abs/2506.05069, 2025. doi: 10.48550/ARXIV.2506.05069. URLhttps://doi. org/10.48550/arXiv.2506.05069

2025 doi arXiv
[11]

Let me do it for you: Towards llm empowered recommendation via tool learning

Yuyue Zhao, Jiancan Wu, Xiang Wang, Wei Tang, Dingxian Wang, and Maarten de Rijke. Let me do it for you: Towards llm empowered recommendation via tool learning. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, S...

2024
[12]

A multi-agent conversational recommender system

Jiabao Fang, Shen Gao, Pengjie Ren, Xiuying Chen, Suzan Verberne, and Zhaochun Ren. A multi-agent conversational recommender system. CoRR, abs/2402.01135, 2024. doi: 10.48550/ARXIV.2402.01135. URL https://doi.org/10.48550/arXiv.2402.01135

2024 doi arXiv
[13]

Macrec: A multi-agent collaboration framework for recommendation

Zhefan Wang, Yuanqing Yu, Wendi Zheng, Weizhi Ma, and Min Zhang. Macrec: A multi-agent collaboration framework for recommendation. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 2760–2764, New ...

2024
[14]

Reason4rec: Large language models for recommendation with deliberative user preference alignment.CoRR, abs/2502.02061, 2025

Yi Fang, Wenjie Wang, Yang Zhang, Fengbin Zhu, Qifan Wang, Fuli Feng, and Xiangnan He. Reason4rec: Large language models for recommendation with deliberative user preference alignment.CoRR, abs/2502.02061, 2025

2025 arXiv
[15]

Recthinker: An agentic framework for tool-augmented reasoning in recommendation.CoRR, abs/2603.09843, 2026

Haobo Zhang, Yutao Zhu, Kelong Mao, Tianhao Li, and Zhicheng Dou. Recthinker: An agentic framework for tool-augmented reasoning in recommendation.CoRR, abs/2603.09843, 2026. doi: 10.48550/ARXIV.2603.09843. URLhttps://doi.org/10.48550/arXiv.2603.09843

2026 doi
[16]

Agentcf++: Memory-enhanced llm-based agents for popularity-aware cross-domain recommendations

Jiahao Liu, Shengkang Gu, Dongsheng Li, Guangping Zhang, Mingzhe Han, Hansu Gu, Peng Zhang, Tun Lu, Li Shang, and Ning Gu. Agentcf++: Memory-enhanced llm-based agents for popularity-aware cross-domain recommendations. InProceedings ofthe48thInternationalACMSIGIRConferenceonRes...

2025
[17]

User behavior simulation with large language model-based agents

Lei Wang, Jingsen Zhang, Hao Yang, Zhi-Yuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Hao Sun, Ruihua Song, Xin Zhao, Jun Xu, Zhicheng Dou, Jun Wang, and Ji-Rong Wen. User behavior simulation with large language model-based agents. ACM Trans. Inf. Syst., 43(2), Janua...

2025 doi
[18]

Simuser: Simulating user behavior with large language models for recommender system evaluation.CoRR, abs/2504.12722, 2025

Nicolas Bougie and Narimasa Watanabe. Simuser: Simulating user behavior with large language models for recommender system evaluation.CoRR, abs/2504.12722, 2025

2025 arXiv
[19]

A llm-based controllable, scalable, human-involved user simulator framework for conversational recommender systems

Lixi Zhu, Xiaowen Huang, and Jitao Sang. A llm-based controllable, scalable, human-involved user simulator framework for conversational recommender systems. InProceedings of the ACMon WebConference 2025, WWW ’25, page 4653–4661, New York, NY, USA, 2025. Association for Computi...

2025
[20]

Agentcf: Collaborative learning with autonomous language agents for recommender systems

Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. Agentcf: Collaborative learning with autonomous language agents for recommender systems. InProceedings of the ACM Web Conference 2024, WWW ’24, page 3679–3689, New York...

2024
[21]

Recmind: Large language model powered agent for recommendation

Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xi- aojiang Huang, and Yingzhen Yang. Recmind: Large language model powered agent for recommendation. In NAACL-HLT(Findings), volume NAACL 2024 ofFindings of ACL, pages 4351–4364. A...

2024
[22]

Personax: A recommendation agent- oriented user modeling framework for long behavior sequence

Yunxiao Shi, Wujiang Xu, Zeqi Zhang, Xing Zi, Qiang Wu, and Min Xu. Personax: A recommendation agent- oriented user modeling framework for long behavior sequence. InACL (Findings), pages 5764–5787. Association for Computational Linguistics, 2025

2025
[23]

Recommender ai agent: Integrating large language models for interactive recommendations.ACMTrans.Inf

Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. Recommender ai agent: Integrating large language models for interactive recommendations.ACMTrans.Inf. Syst., 43(4), June 2025. ISSN 1046-8188. doi: 10.1145/3731446. URLhttps://doi.org/10.1145/3731446

2025 doi
[25]

Christiano, Jan Leike, Tom B

Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. InNIPS, pages 4299–4307, 2017

2017
[26]

Manning, Stefano Ermon, and Chelsea Finn

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InNeurIPS, 2023

2023
[27]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024

2024 arXiv
[28]

On-policy distillation of language models: Learning from self-generated mistakes

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In ICLR. OpenReview.net, 2024

2024
[29]

A survey on knowledge distillation of large language models.CoRR, abs/2402.13116, 2024

Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models.CoRR, abs/2402.13116, 2024

2024 arXiv
[30]

Minillm: Knowledge distillation of large language models

Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. In ICLR. OpenReview.net, 2024

2024
[31]

Self-distilled reasoner: On-policy self-distillation for large language models.CoRR, abs/2601.18734, 2026

Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models.CoRR, abs/2601.18734, 2026

2026 arXiv
[32]

Qwen3 technical report.CoRR, abs/2505.09388, 2025

Qwen Team. Qwen3 technical report.CoRR, abs/2505.09388, 2025

2025 arXiv
[33]

Deepseek-v3.2: Pushing the frontier of open large language models.CoRR, abs/2512.02556, 2025

DeepSeek-AI. Deepseek-v3.2: Pushing the frontier of open large language models.CoRR, abs/2512.02556, 2025. 15

2025 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Bert4rec: Sequential rec- ommendation with bidirectional encoder representations from transformer

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential rec- ommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, page 1441–1450, New York, NY, USA, 2019. Association for Computing Machine...

arXiv 2019

[2] [2]

Wang-Cheng Kang and Julian J. McAuley. Self-attentive sequential recommendation. InICDM, pages 197–206. IEEE Computer Society, 2018

work page 2018

[3] [3]

BPR: bayesian personalized ranking from implicit feedback.CoRR, abs/1205.2618, 2012

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. BPR: bayesian personalized ranking from implicit feedback.CoRR, abs/1205.2618, 2012

arXiv 2012

[4] [4]

S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization

Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 1893–1902, New York, NY, USA, 2020. Ass...

arXiv 1902

[5] [5]

Llasa: Large language and e-commerce shopping assistant.CoRR, abs/2408.02006, 2024

Shuo Zhang, Boci Peng, Xinping Zhao, Boren Hu, Yun Zhu, Yanjia Zeng, and Xuming Hu. Llasa: Large language and e-commerce shopping assistant.CoRR, abs/2408.02006, 2024. doi: 10.48550/ARXIV.2408.02006. URLhttps://doi.org/10.48550/arXiv.2408.02006

arXiv doi:10.48550/arxiv.2408.02006 2024

[6] [6]

A shopping agent for addressing subjective product needs

Preetam Prabhu Srikar Dammu, Omar Alonso, and Barbara Poblete. A shopping agent for addressing subjective product needs. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, page 1032–1035, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400713293. doi: 10.1145/3701551.3704124. URLht...

work page arXiv 2025

[7] [7]

Language model alignment for conversational shop- ping at amazon

Chen Luo, Dimitri Papadimitriou, Hariharan Muralidharan, Dhineshkumar Ramasubbu, Aakash Kolekar, Wenju Xu, Cong Xu, Anirudh Srinivasan, Mukesh Jain, and Qi He. Language model alignment for conversational shop- ping at amazon. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, page ...

arXiv 2025

[8] [8]

In Ee-Peng Lim, Marianne Winslett, Mark Sanderson, Ada Wai-Chee Fu, Jimeng Sun, J

Feng-Lin Li, Minghui Qiu, Haiqing Chen, Xiongwei Wang, Xing Gao, Jun Huang, Juwei Ren, Zhongzhou Zhao, Weipeng Zhao, Lei Wang, Guwei Jin, and Wei Chu.AliMe Assist : An intelligent assistant for creating an innovative e-commerce experience. In Ee-Peng Lim, Marianne Winslett, Mark Sanderson, Ada Wai-Chee Fu, Jimeng Sun, J. Shane Culpepper, Eric Lo, Joyce C....

arXiv 2017