Pith. sign in

REVIEW 3 major objections 5 minor 48 references

RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RecUserSim, an LLM agent with a three-tier 'Rating-Action-Response' mechanism and tool-based refinement, generates realistic, diverse user dialogues and consistent explicit ratings for evaluating conversational recommender systems.

desk verdict RecUserSim is a genuinely more flexible LLM user simulator with real deployed use, but its realism and diversity claims lean on an unvalidated LLM judge and self-referential metrics. read the letter →

arxiv 2507.22897 v1 pith:EV3RU6MS submitted 2025-06-25 cs.HC cs.AI

Luyu Chen , Quanyu Dai , Zeyu Zhang , Xueyang Feng , Mingyu Zhang , Pengcheng Tang , Xu Chen , Yue Zhu
show 1 more author
Zhenhua Dong
This is my paper
classification cs.HCcs.AI
keywords usersimulatorconversationalrecommendersystemsLLMagentsevaluationdiversityratingmechanismpersonasimulationboundedrationality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RecUserSim is an LLM-based agent that plays the role of a user talking to a conversational recommender system, so the system can be tested without costly human trials. The paper claims this agent produces more realistic and diverse dialogues than existing LLM simulators, while also giving an explicit numeric rating for each recommendation that stays consistent even when the agent's underlying LLM is swapped for a smaller one. If true, that would give recommender-system developers a cheap, repeatable way to compare systems and tune them before deployment, which the paper demonstrates in an industrial food-assistant setting. The paper's evidence is 500-dialogue comparisons against two baselines, diversity analyses on sentence length, information richness, and formality, and cross-LLM rating correlations.

What carries the argument

The machinery is the three-tier 'Rating-Action-Response' mechanism, adapted from Bounded Rationality theory, together with a tool-augmented refinement module. In each turn the simulator first produces a multi-dimensional rating (language, action, recommendation, with subjective modifiers), then selects a subset of five user actions (request, clarify, feedback, inquire, end), then generates a response conditioned on the profile, history, ratings, and actions; finally the refinement module applies three tools—information richness, formality, and sentence length—each with a judger and refiner, to align the response with the persona. This design is what lets the agent show both individual realism and population diversity while still producing quantitative scores.

What would settle it

Run a blinded human study where judges see real user transcripts mixed with RecUserSim and baseline outputs and rate which are human; if RecUserSim is not chosen as more human-like than baselines at a significant rate, the realism claim fails. Alternatively, if RecUserSim ratings from two different base LLMs diverge sharply when evaluating the same fixed CRS, the rating-consistency claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a four-module LLM agent—profile, memory, action, refinement—can realistically role-play individual users and represent a diverse population, and that its explicit ratings are reliable evaluators of a conversational recommender. The load-bearing part is the three-tier 'Rating-Action-Response' action module: before saying anything, the agent scores the system's turn on language quality, action quality, and recommendation quality (with user-specific modifiers), then chooses one or more of five fine-grained actions, then writes a response in the user's linguistic pattern. A refinement module then enforces the persona's information richness, formality, and sentence length through dedicated judge-refiner tools. Experiments show RecUserSim wins pairwise comparisons against iEvalLM and CSHI on six subjective metrics judged by gpt-4o, produces more even distributions across the three language dimensions, and yields action ratings with Pearson correlations above 0.8 across gpt-4o, gpt-4o-mini, and glm-4-9b.

Load-bearing premise

The evidence that RecUserSim is realistic relies on a gpt-4o judge comparing outputs, and the diversity metrics are categories that RecUserSim's own refinement module defines, so the reported advantages could reflect the simulator's design rather than what humans perceive.

Editorial extensions

If this is right

  • RecUserSim's ratings can rank CRS instances: the paper shows scores track the underlying base LLM's quality (gpt-4o > gpt-4o-mini > gpt-3.5-turbo).
  • Because ratings correlate highly across base LLMs, evaluators can use smaller or cheaper models and retain a consistent evaluation signal.
  • Adjusting a persona's linguistic patterns shifts output distributions, so a developer can deliberately simulate a target user segment.
  • The simulator can operate in an industrial setting, as shown by its use in evaluating Huawei's Celia Food Assistant, where its scores align with human evaluators on the online version.
  • Even a small open-source model (glm-4-9b) inside RecUserSim beats the single-prompt baseline running on gpt-4o on most subjective metrics, suggesting the framework, not the base model, is what drives realism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to use RecUserSim as a training environment: its ratings could serve as reward signals for reinforcement learning of CRS policies, something the paper does not test.
  • The diversity claim is tied to categories the refinement module itself defines; testing on external or human-defined style categories would tell whether the diversity reflects the language axis or the evaluator's design.
  • The same judge-refiner pattern for controlling multiple output constraints could transfer to other multi-constraint generation tasks, such as summarization or dialogue systems beyond recommendation.
  • Comparing RecUserSim's ratings to actual human ratings over a wider range of systems would give the rating mechanism an absolute calibration, not just relative consistency.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces RecUserSim, an LLM agent-based user simulator for evaluating conversational recommender systems (CRS). The simulator comprises four modules: a profile module with conflict resolution, a memory module with unknown-preference excitation, a three-tier 'Rating-Action-Response' action module inspired by bounded rationality, and a tool-augmented refinement module controlling sentence length, information richness, and formality. The authors claim that RecUserSim produces more realistic and diverse dialogues than existing simulators (iEvalLM, CSHI), generates explicit multi-dimensional ratings that are consistent across base LLMs, and demonstrates practical value in an industrial deployment. The evaluation uses a gpt-4o judge for subjective pairwise comparisons, distribution-based diversity metrics, correlation analyses across ratings from different LLM backbones, and a human-annotated comparison in the Huawei Celia Food Assistant deployment.

Significance. If the claims are substantiated, RecUserSim would be a useful contribution to CRS evaluation, particularly its explicit rating mechanism for quantitative assessment and its modular design for controllable output generation. The paper has several strengths: it ships a complete framework with multiple mechanisms; it tests robustness across four base LLMs including a smaller open-source model; and it includes an industrial deployment with human evaluation, which is rare in this literature. The explicit rating module addressing action, language, and recommendation quality is a concrete step toward quantitative simulator-based CRS evaluation. However, the central empirical claims about realism and diversity currently rest on an LLM judge and on metrics that are partly defined by the simulator's own controls, and the rating reliability evidence is weaker than the text suggests. These issues must be addressed before the contribution can be fully accepted.

major comments (3)
  1. [§4.1.1, Figure 2] The subjective evaluation of realism, naturalness, and dialogue quality relies entirely on a gpt-4o-based judge for pairwise comparisons, with no human evaluation on this main scenario. Since the paper's central claim is that RecUserSim produces realistic, high-quality dialogues, the win rates in Figure 2 need to be validated against human judgments—for example, through a small-scale human study or an agreement analysis between the LLM judge and human annotators. Without such validation, the reported advantages over iEvalLM and CSHI may reflect systematic bias of the judge rather than genuine differences in user-simulation quality.
  2. [§4.1.2, Figure 3] The diversity metrics (sentence length, information richness, formality) are exactly the dimensions controlled by the refinement module in §3.4, and the categorization thresholds are set by that module. The diversity evaluation is therefore circular in an important sense: a uniform distribution on these metrics partially reflects the simulator's design rather than an independent measure of population diversity. To support the claim of diverse user population representation, the authors should either use metrics that are not derived from the module's own control dimensions, or validate the chosen categories against human perception of diversity with a user study.
  3. [§4.2, Tables 2 and 3; §4.3, Table 4] The claim of 'high consistency' of ratings across base LLMs is overstated for recommendation ratings. In Table 3, three of the six correlation coefficients are not statistically significant (e.g., gpt-4o vs gpt-4o-mini Pearson 0.62 and Spearman 0.51; gpt-4o vs glm-4-9b Pearson 0.53), and all correlations are computed over only six CRS models. Furthermore, the main rating-reliability study has no human ground truth, and the industrial comparison in Table 4 shows substantial discrepancies (e.g., Demo Action 3.37/3.30 vs human 2.16; Online Recommendation 2.99/2.59 vs human 3.53). These gaps should be discussed explicitly, and the reliability claims should be tempered or supplemented with additional validation against human judgments.
minor comments (5)
  1. [§3.3] The phrase 'Tree-Tier Action Mechanism' appears twice and should read 'Three-Tier Action Mechanism'; 'Tree-Tier' is a typo that also recurs in the heading of the subsection.
  2. [Throughout] There are several typos and inconsistent spellings: 'espsecially' in the action list of §3.3, 'polupations' in §4.1.2, 'Dalogue' in §2.1, and inconsistent use of 'iEvaLM' versus 'iEvalLM' (the cited work is iEvalLM).
  3. [References and text] The citation numbering for CSHI is inconsistent: the text cites [44] for the CSHI framework in §3.3 and [43] in the related-work paragraph, but the reference list appears to place the CSHI paper as [44] and the limitations paper as [43]. Please unify the citations and ensure the reference list matches the in-text numbering.
  4. [§4.1.1] The description of the data collection is slightly ambiguous: it states that 'each simulator interacts with the same CRS to generate 500 dialogues,' but it is unclear whether this means 500 dialogues per baseline simulator or 500 in total. Clarifying the number of dialogues per condition would help reproducibility.
  5. [§4.2] The correlation analysis reports only coefficients and significance stars, not confidence intervals or scatter plots; given the small number of models (n=6), reporting per-pair scatter plots or at least the underlying scores would make the strength of the correlations easier to assess.

Circularity Check

2 steps flagged · score 4.0 of 10

Diversity and rating-validation claims reduce in part to RecUserSim's own refinement thresholds and to a self-authored rating rubric; the core agent construction is otherwise non-circular.

  1. self definitional [Section 4.1.2 (Objective Evaluation of User Simulator Diversity) and Section 3.4 (Tool-augmented Refinement)]
    "Responses are then categorized using predefined thresholds set by the refinement module. By analyzing the distribution of these categories, we evaluate the diversity of simulator outputs. A more uniform distribution indicates greater response diversity, while a skewed distribution suggests a bias towards a specific type, indicating lower diversity. ... The refinement module consists of several specialized refinement tools, each fine-tuning outputs based on specific linguistic patterns including information richness, formality, and sentence length."

    The three diversity metrics are exactly the three linguistic patterns that RecUserSim's refinement module is designed to control, and the category thresholds are set by that same module. Therefore RecUserSim's uniform distributions over sentence length, information richness, and formality are not an independent measurement of population diversity; they are the simulator's own design targets realized through profile sampling and refinement. The claimed advantage over iEvalLM and CSHI on these axes reduces substantially to the fact that the baselines lack the same control module, rather than to an externally validated property of diversity.

  2. self citation load bearing [Section 3.3 (Multi-Dimensional Rating) and Section 4.3 (Industrial Deployment)]
    "Ratings are structured across three key dimensions as suggested by [4]: ... Six evaluators annotated these conversations based on the same criteria of RecUserSim."

    The rating dimensions used to evaluate CRS performance come from reference [4], which shares two authors (Quanyu Dai and Zhenhua Dong) with this paper. The only external check of the rating mechanism asks human annotators to score with the same criteria RecUserSim itself uses, so the agreement reported in Table 4 validates the simulator against its own rubric rather than an independent ground truth. The rating mechanism's claimed accuracy is therefore partly defined by the framework it is supposed to be validated against.

full rationale

The core architectural contributions of RecUserSim (profile module, memory module, action module, refinement module) are engineering designs and are not circular in themselves; they are evaluated through head-to-head LLM-judge comparisons and controlled experiments. However, two load-bearing evaluation claims have a built-in circular component. First, the diversity claim in Section 4.1.2 measures uniformity over sentence length, information richness, and formality using thresholds defined by RecUserSim's own refinement module, so the metric is aligned with the simulator's design targets by construction. Second, the rating reliability claim leans on rating dimensions from a self-authored prior work [4] and on a human evaluation that uses the same criteria as the simulator, weakening the independence of the validation. There is some external grounding in the industrial deployment with human annotators, and the observed discrepancies (e.g., Demo Action quality 3.37/3.30 for RecUserSim vs. 2.16 for humans) show the comparison is not purely cosmetic, but that evaluation still uses RecUserSim's own rubric. Given these partial reductions, a score of 4 reflects substantial non-circular engineering content alongside evaluation claims that are partially self-defined.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on handcrafted profile priors, rating modifier constants, and refinement thresholds, as well as on assumptions about LLM judge validity and Bounded Rationality as a model of user behavior. No new physical or conceptual entities are postulated.

free parameters (3)
  • Profile sampling distributions and dictionaries = not specified
    Profile module randomly samples attributes from predefined dictionaries with prior distributions (Section 3.2.1); these priors are handcrafted design choices that shape all simulation outputs.
  • Rating subjective modifiers = e.g., +1 for novelty, 1-5 base scores
    Recommendation quality score includes subjective modifiers of -1 to +1 based on user-specific tendencies (Section 3.3); these are hand-assigned constants, not fitted to data.
  • Refinement thresholds = e.g., 20 words, 2 key points
    Tool-augmented refinement uses predefined thresholds for sentence length and information richness (Section 3.4); these hand-chosen values directly influence controllability and diversity results.
assumptions (4)
  • domain assumption A gpt-4o-based LLM judge reliably evaluates dialogue quality, diversity, and realism compared to human judgment.
    Subjective comparisons and diversity metrics throughout Section 4.1 rely on gpt-4o as judge/evaluator, citing [41], but no human validation is provided for these main results.
  • domain assumption Bounded Rationality theory is an appropriate model of real user decision-making in conversational recommendation.
    The three-tier Rating-Action-Response mechanism is 'inspired by' Simon [30] (Section 3.3); the paper does not validate that this theory captures actual user behavior in the tested setting.
  • domain assumption Correlation of ratings across different base LLMs indicates rating reliability for CRS evaluation.
    Section 4.2 treats cross-LLM consistency as evidence of accuracy; this assumes that agreement between simulators reflects ground truth, not shared biases.
  • domain assumption The rating dimensions (language, action, recommendation quality) from self-cited prior work [4] are appropriate for CRS evaluation.
    The rating mechanism is built on dimensions suggested by the same authors' earlier framework [4]; whether these dimensions cover all relevant CRS aspects is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems." pith.science (2026). https://pith.science/paper/EV3RU6MS

@misc{pith2026250722897,
  author       = {Pith},
  title        = {Pith review of: RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EV3RU6MS}},
  note         = {Machine review of arXiv:2507.22897}
}
read the original abstract

Conversational recommender systems (CRS) enhance user experience through multi-turn interactions, yet evaluating CRS remains challenging. User simulators can provide comprehensive evaluations through interactions with CRS, but building realistic and diverse simulators is difficult. While recent work leverages large language models (LLMs) to simulate user interactions, they still fall short in emulating individual real users across diverse scenarios and lack explicit rating mechanisms for quantitative evaluation. To address these gaps, we propose RecUserSim, an LLM agent-based user simulator with enhanced simulation realism and diversity while providing explicit scores. RecUserSim features several key modules: a profile module for defining realistic and diverse user personas, a memory module for tracking interaction history and discovering unknown preferences, and a core action module inspired by Bounded Rationality theory that enables nuanced decision-making while generating more fine-grained actions and personalized responses. To further enhance output control, a refinement module is designed to fine-tune final responses. Experiments demonstrate that RecUserSim generates diverse, controllable outputs and produces realistic, high-quality dialogues, even with smaller base LLMs. The ratings generated by RecUserSim show high consistency across different base LLMs, highlighting its effectiveness for CRS evaluation.

Figures

Figures reproduced from arXiv: 2507.22897 by the authors.

Figure 1
Figure 1. An overview of RecUserSim, which consists of four modules: profile, memory, action, and refinement. The profile [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Subjective comparison of RecUserSim and baseline [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Controllability of RecUserSim on outputs based on users’ linguistic patterns. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 30 canonical work pages

  1. [1]

    Jafar Afzali, Aleksander Mark Drzewiecki, Krisztian Balog, and Shuo Zhang. 2023. UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recom- mender Systems. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 1160–1163

    work page 2023

  2. [2]

    Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He

  3. [3]

    Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. 2024. From Persona to Per- sonalization: A Survey on Role-Playing Language Agents. CoRR abs/2404.18231 (2024)

    arXiv 2024

  4. [4]

    Nuo Chen, Quanyu Dai, Xiaoyu Dong, Xiao-Ming Wu, and Zhenhua Dong. 2025. Evaluating Conversational Recommender Systems with Large Language Models: A User-Centric Evaluation Framework. arXiv preprint arXiv:2501.09493 (2025)

    arXiv 2025

  5. [5]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah

  6. [6]

    Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongx- iang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering ChatGPT’s Capabilities in Recommender Systems. In Proceedings of the 17th ACM Conference on Recom- mender Systems. 1126–1132

    work page 2023

  7. [7]

    Jiabao Fang, Shen Gao, Pengjie Ren, Xiuying Chen, Suzan Verberne, and Zhaochun Ren. 2024. A multi-agent conversational recommender system. arXiv preprint arXiv:2402.01135 (2024)

    arXiv 2024

  8. [8]

    Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A survey. AI Open 2 (2021), 100–126

    work page 2021

Show all 48 references
  1. [9]

    Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524 (2023)

    2023 arXiv

  2. [10]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

  3. [11]

    Emrul Hasan, Mizanur Rahman, Chen Ding, Jimmy Xiangji Huang, and Shaina Raza. 2024. Review-based Recommender Systems: A Survey of Approaches, Challenges and Future Perspectives. arXiv preprint arXiv:2405.05562 (2024)

    2024 arXiv

  4. [12]

    Rolf Jagerman, Krisztian Balog, and Maarten De Rijke. 2018. OpenSearch: Lessons Learned from an Online Evaluation Campaign. J. Data and Information Quality 10, 3 (2018)

    2018

  5. [13]

    Rolf Jagerman, Ilya Markov, and Maarten de Rijke. 2019. When People Change their Mind: Off-Policy Evaluation in Non-stationary Recommendation Environ- ments. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. 447–455

    2019

  6. [14]

    Dietmar Jannach and Ahtsham Manzoor. 2020. End-to-end learning for conver- sational recommendation: A long way to go? IntRS@ RecSys 20 (2020), 72–76

    2020

  7. [15]

    Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2021. A Survey on Conversational Recommender Systems. ACM Comput. Surv. 54, 5 (2021)

    2021

  8. [16]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20, 4 (2002), 422–446

    2002

  9. [17]

    Minjin Kim, Minju Kim, Hana Kim, Beong-woo Kwak, SeongKu Kang, Youngjae Yu, Jinyoung Yeo, and Dongha Lee. 2024. Pearl: A Review-driven Persona- Knowledge Grounded Conversational Recommendation Dataset. In Findings of the Association for Computational Linguistics: ACL 2024 . 1105–1120

    2024

  10. [18]

    Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems. In Proceedings of the 13th International Conference on Web Search and ...

    2020

  11. [19]

    Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020. Interactive Path Reasoning on Graph for Conver- sational Recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining ....

    2020

  12. [20]

    Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020. Interactive path reasoning on graph for conver- sational recommendation. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining ....

    2020

  13. [21]

    Lei Li, Yongfeng Zhang, Dugang Liu, and Li Chen. 2024. Large Language Models for Generative Recommendation: A Survey and Visionary Discussions. In Pro- ceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COL...

    2024

  14. [22]

    Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, and Tat- Seng Chua. 2019. Deep conversational recommender in travel. arXiv preprint arXiv:1907.00710 (2019)

    2019 arXiv

  15. [23]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. 74–81

    2004

  16. [24]

    We Need Structured Output

    Michael Xieyang Liu, Frederick Liu, Alexander J. Fiannaca, Terry Koo, Lucas Dixon, Michael Terry, and Carrie J. Cai. 2024. "We Need Structured Output": To- wards User-centered Constraints on Large Language Model Output. In Extended Abstracts of the CHI Conference on Human Fact...

    2024

  17. [25]

    Yuanxing Liu, Weinan Zhang, Yifan Chen, Yuchi Zhang, Haopeng Bai, Fan Feng, Hengbin Cui, Yongbin Li, and Wanxiang Che. 2023. Conversational Recommender System and Large Language Model Are Made for Each Other in E-commerce Pre-sales Dialogue. In Findings of the Association for ...

    2023

  18. [26]

    Julia Neidhardt, Rainer Schuster, Leonhard Seyfang, and Hannes Werthner. 2014. Eliciting the users’ unknown preferences. In Proceedings of the 8th ACM Confer- ence on Recommender Systems . 309–312

    2014

  19. [27]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics . 311–318

    2002

  20. [28]

    Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. 285–295

    2001

  21. [29]

    J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative filtering recommender systems. In The adaptive web: methods and strategies of web personalization. 291–324

    2007

  22. [30]

    Herbert A Simon. 1955. A behavioral model of rational choice. The quarterly journal of economics (1955), 99–118

    1955

  23. [31]

    Yueming Sun and Yi Zhang. 2018. Conversational Recommender System. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 235–244

    2018

  24. [32]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 6 (2024), 186345

    2024

  25. [33]

    Xiaolei Wang, Xinyu Tang, Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. 2023. Re- thinking the Evaluation for Conversational Recommendation in the Era of Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 10052–10065

    2023

  26. [34]

    Xiaolei Wang, Kun Zhou, Ji-Rong Wen, and Wayne Xin Zhao. 2022. Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 1929–1937

    2022

  27. [35]

    Ya Xu, Nanyu Chen, Addrian Fernandez, Omar Sinno, and Anmol Bhasin. 2015. From Infrastructure to Culture: A/B Testing Challenges in Large Scale Social Networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 2227–2236

    2015

  28. [36]

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2024. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408.15240 (2024)

    2024 arXiv

  29. [37]

    Shuo Zhang and Krisztian Balog. 2020. Evaluating Conversational Recommender Systems via User Simulation. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 1512–1520

    2020

  30. [38]

    Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. 2024. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501 (2024)

    2024 arXiv

  31. [39]

    Zeyu Zhang, Quanyu Dai, Luyu Chen, Zeren Jiang, Rui Li, Jieming Zhu, Xu Chen, Yi Xie, Zhenhua Dong, and Ji-Rong Wen. 2024. Memsim: A bayesian simulator for evaluating memory of llm-based personal assistants. arXiv preprint arXiv:2409.20163 (2024)

    2024 arXiv

  32. [40]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong W...

    2023 arXiv

  33. [41]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2024. Judging LLM-as-a-judge with MT- bench and Chatbot Arena. In Proceedings of the 37th Interna...

    2024

  34. [42]

    Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving Conversational Recommender Systems via Knowl- edge Graph based Semantic Fusion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Min...

    2020

  35. [43]

    Lixi Zhu, Xiaowen Huang, and Jitao Sang. 2024. How Reliable is Your Simulator? Analysis on the Limitations of Current LLM-based User Simulators for Conver- sational Recommendation. In Companion Proceedings of the ACM Web Conference

    2024

  36. [44]

    Lixi Zhu, Xiaowen Huang, and Jitao Sang. 2024. A LLM-based Controllable, Scalable, Human-Involved User Simulator Framework for Conversational Rec- ommender Systems. arXiv preprint arXiv:2405.08035 (2024)

    2024 arXiv

  37. [45]

    Jie Zou, Yifan Chen, and Evangelos Kanoulas. 2020. Towards Question-based Rec- ommender Systems. InProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . 881–890

    2020

  38. [2016]

    In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems

    Wide & Deep Learning for Recommender Systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems . 7–10

  39. [2023]

    ACM Trans

    Bias and Debias in Recommender System: A Survey and Future Directions. ACM Trans. Inf. Syst. 41, 3 (2023)

    2023

  40. [2024]

    ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.